Automatic synchronization method for CG data assets

By dynamically adjusting the compression ratio and incremental synchronization, combined with differential algorithms and key node identification, the problems of low transmission efficiency, insufficient concurrent processing, and poor user experience in CG data asset synchronization are solved, achieving efficient and smooth data synchronization.

CN119003475AActive Publication Date: 2024-11-22NANTONG TIANXING INFORMATION TECH CO LTD

Patent Information

Application Number
CN202411491991.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-24
Publication Date
2024-11-22
Estimated Expiration
2044-10-24

AI Technical Summary

Technical Problem

Existing technologies for synchronizing CG data assets suffer from problems such as low efficiency in large file transfer, insufficient concurrent processing capabilities, untimely detection of data changes, and poor user experience. In particular, it is difficult to efficiently synchronize large files in a multi-person collaborative environment.

Method used

A full synchronization method with dynamically adjusted compression ratio is adopted, combined with incremental synchronization and differential algorithms, to identify key nodes and display a transition interface. The K-nearest neighbor algorithm is used to optimize the synchronization process, and high-concurrency processing tools and environment are configured.

Benefits of technology

It improves the efficiency of large file transfer, reduces resource consumption, enhances high-concurrency processing capabilities, improves user experience, and ensures the smoothness and accuracy of the synchronization process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003475A_ABST
    Figure CN119003475A_ABST
Patent Text Reader

Abstract

The invention discloses a CG data asset automatic synchronization method, and relates to the field of computer image data synchronization, and the method comprises the steps: completing the early-stage preparation of data synchronization; based on an earlier-stage preparation result of data synchronization, performing primary full-amount synchronization on CG data assets to be synchronized, and dynamically adjusting a compression ratio in a full-amount synchronization process; incremental synchronization is configured and executed, and in the incremental synchronization process, change data in the CG data is synchronized based on a difference algorithm; key nodes in the synchronization process are identified based on a proximity algorithm, and a pre-configured transition interface between the key nodes in the synchronization process is displayed; and monitoring full-amount synchronization and increment synchronization processes, and recording monitoring information of each time of full-amount synchronization and increment synchronization. According to the invention, the transmission efficiency is improved, the resource consumption is reduced, the high-concurrency processing capability is enhanced, and the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer image data synchronization, and in particular to a method for automatic synchronization of CG data assets. Background Art

[0002] CG data assets (Computer Graphics Data Assets) generally refer to various digital resources used in the field of computer graphics. These resources may include but are not limited to 3D models, textures, animations, scenes, lighting settings, materials, special effects, etc. CG data assets are widely used in film and television production, game development, virtual reality, architectural visualization and other fields.

[0003] Automatic synchronization of CG data assets refers to the process of automatically updating and synchronizing these data assets between different storage locations or systems through automation technology and tools. This automatic synchronization ensures that when multiple teams or systems use the same data asset, they always access the latest version, thereby improving work efficiency and data consistency.

[0004] With the rapid development of computer graphics (CG) technology, the scale of CG data assets continues to expand, especially the use of large files (such as 3D models, textures, etc.) is becoming more and more common. The synchronization and management of these data assets has become an important technical challenge, mainly facing the following technical problems: (1) Low transmission efficiency of large files: Large files may face bandwidth bottlenecks during transmission, resulting in low transmission efficiency.

[0005] (2) Insufficient high-concurrency processing capabilities: In a multi-person collaborative environment, how to efficiently synchronize multiple large files is a difficult problem.

[0006] (3) Untimely detection of data changes: Traditional synchronization methods often require full transmission, resulting in a waste of resources.

[0007] (4) Poor user experience: During the data synchronization process, users may encounter interface freezes or jerky transitions.

[0008] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0009] In view of the problems in the related art, the present invention proposes a method for automatic synchronization of CG data assets to overcome the above-mentioned technical problems existing in the existing related art.

[0010] To this end, the specific technical solution adopted by the present invention is as follows: A method for automatic synchronization of CG data assets, the method comprising: S1. Acquire and classify the CG data assets to be synchronized and complete the preliminary preparation for data synchronization; the preliminary preparation includes acquiring the synchronization requirements of CG data, planning the display process of CG data and configuring the synchronization architecture of CG data; S2. Based on the preliminary preparation results of data synchronization, the CG data assets to be synchronized are synchronized for the first time, and the compression ratio is dynamically adjusted during the full synchronization process; S3. Configure and execute incremental synchronization, and during the incremental synchronization process, synchronize the changed data in the CG data based on the differential algorithm; S4, identifies key nodes in the synchronization process based on the proximity algorithm, and displays the transition interface between key nodes in the pre-configured synchronization process; the synchronization process includes full synchronization and incremental synchronization; S5. Monitor the full synchronization and incremental synchronization process, and record the monitoring information of each full synchronization and incremental synchronization, and the monitoring information includes the execution status of key nodes and the display status of the transition interface; Among them, the key nodes in the synchronization process are identified based on the proximity algorithm, and the transition interface between the key nodes in the pre-configured synchronization process is displayed, including: S41, collecting historical CG data synchronization task data, and using the K nearest neighbor algorithm to identify key nodes of the current CG data synchronization task to obtain a key node list; S42. According to the key node list, a pre-configured transition interface between key nodes in the synchronization process is displayed to the user.

[0011] Furthermore, the preliminary preparations for completing data synchronization include: S11. Collect and analyze the synchronization requirements of CG data and determine the synchronization goals and requirements; S12. Design the display process of CG data, including user interface and data display method; S13. Select and configure synchronization tools and environments suitable for large file transfer and high-concurrency processing.

[0012] Furthermore, the compression ratio is dynamically adjusted during the full synchronization process including: S21. Collecting historical performance indicators of CG data synchronization, where the performance indicators include network bandwidth utilization, CPU utilization, system memory utilization, and data transmission rate; S22, determining a compression ratio range, and filtering out candidate compression ratios; S23, defining an objective function to measure the quality of the compression ratio; defining constraints; S24, using heuristic rules to explore different compression ratios and perform local search to improve the current solution; S25. Pre-calculate the best compression ratio under different conditions according to the results of the heuristic search and the local search, and construct a compression ratio table; S26. Acquire the performance index of CG data synchronization in real time, and calculate the value of the objective function; match the corresponding optimal compression ratio.

[0013] Further, the compression ratio range is determined, and the candidate compression ratios are filtered out to include: S221, determining a lower limit and an upper limit of a compression ratio according to an application scenario; S222. Within the determined compression ratio range, select a number of compression ratios at equal intervals as candidate compression ratios.

[0014] Furthermore, incremental synchronization is configured and performed, and during the incremental synchronization process, the changed data in the CG data is synchronized based on the differential algorithm, including: S31, calculate the file checksum of the source data and the target data, and compare the file checksum of the source data and the target data to obtain the changed data in the process of CG data incremental synchronization; S32, using a differential algorithm to generate a differential data file for the changed data in the CG data incremental synchronization process; S33, synchronizing the generated differential data file to the target location, and updating the target file at the target location using the differential data file.

[0015] Furthermore, the historical CG data synchronization task data is collected, and the K nearest neighbor algorithm is used to identify the key nodes of the current CG data synchronization task, and the key node list obtained includes: S411, obtaining a determination standard for a key node, wherein the determination standard for a key node includes that a CPU usage rate is continuously higher than a first threshold, a memory usage rate is continuously higher than a second threshold, a disk I / O rate is continuously higher than a third threshold, and a network bandwidth utilization rate is continuously higher than a fourth threshold, and the duration of the duration exceeds a preset duration; S412, according to the determination standard of the key nodes, collecting the historical CG data synchronization task data, and annotating the historical CG data synchronization task data to obtain an annotated data set; S413, selecting a distance measurement algorithm and the number of nearest neighbors, and determining the number of selected nearest neighbors through cross-validation; S414, constructing an index structure of the labeled data set, and storing the labeled data set; S415, collecting current CG data synchronization task data, using K nearest neighbor algorithm to analyze the current CG data synchronization task data, and identifying key nodes; S416. Generate a key node list according to the prediction result of the K nearest neighbor algorithm, where the key node list includes the information and position of each key node.

[0016] Furthermore, the current CG data synchronization task data is collected, and the current CG data synchronization task data is analyzed using the K nearest neighbor algorithm. Identifying key nodes includes the following steps: S4151. Using the selected distance measurement algorithm, calculate the distance between the nodes in the current CG data synchronization task data and the nodes in the labeled data set to obtain a distance list; S4152. According to the distance list, select several neighbors with the closest distance; assign a weight to each neighbor according to the distance, and perform weighted voting; S4153. Determine whether the current data point is a key node based on the voting result.

[0017] Furthermore, when assigning weights to each neighbor based on the distance and performing weighted voting, the weight assignment formula is: ; In the formula, w i Indicates i The weight of the neighbors, d ( x , y i ) indicates the current node x With Neighbors y i The distance between α represents the parameter used to adjust the weight sensitivity; Take key nodes and non-key nodes as categories, and initialize a counter to accumulate weighted votes for each category; Traverse the neighbor nodes and calculate the weight of each neighbor node based on the distance; according to the category and weight of each neighbor node, update the counter of the corresponding category, and select the category with the largest counter value as the prediction result.

[0018] Further, according to the key node list, the transition interface between the key nodes in the pre-configured synchronization process is displayed to the user, including: S421, pre-configure a transition effect from any node to the next node to obtain a user interface; S422. Deploy the user interface to the production environment, monitor user feedback on the user interface, and optimize and adjust the user interface based on the feedback.

[0019] Furthermore, the full and incremental synchronization processes are monitored, and the monitoring information for each full and incremental synchronization is recorded, including: S51. Configure monitoring tools and record the execution status of key nodes during each full synchronization and incremental synchronization process, and the execution status includes success, failure and execution time; S52: Obtain the display effect of the transition interface and record user feedback.

[0020] The beneficial effects of the present invention are: (1) Improved transmission efficiency: The present invention can optimize data transmission according to network conditions by dynamically adjusting the compression ratio, significantly improving the synchronization speed of large files. The dynamic adjustment method can flexibly cope with different network and system environments, improving the versatility and adaptability of the method.

[0021] (2) Reduce resource consumption: The present invention utilizes incremental synchronization and differential algorithms to transmit only the changed parts, thereby reducing bandwidth usage and storage requirements, and reducing the occupation of system resources.

[0022] (3) Enhanced high-concurrency processing capabilities: The present invention configures tools and environments suitable for high-concurrency processing, allowing multiple users to synchronize data at the same time and improve collaboration efficiency.

[0023] (4) Improve user experience: The present invention ensures that the user's operation during the synchronization process is more natural and smooth through the identification of key nodes and the smooth transition interface display, reducing the troubles in use. At the same time, the proximity algorithm is used to analyze historical data, which can accurately identify the key nodes in the synchronization process, enhance the intelligence level of key node identification during CG data synchronization, and make quick and accurate judgments when facing complex real-time data.

[0024] (5) Identification of key nodes: By identifying key nodes during the CG data synchronization process, the present invention can prioritize the allocation of computing and network resources during the data synchronization process, thereby improving synchronization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a flowchart of a method for automatic synchronization of CG data assets according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0028] According to an embodiment of the present invention, a method for automatic synchronization of CG data assets is provided.

[0029] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to the CG data asset automatic synchronization method of an embodiment of the present invention, the CG data asset automatic synchronization method includes: S1. Acquire and classify the CG data assets to be synchronized, and complete the preliminary preparation for data synchronization; the preliminary preparation includes obtaining the synchronization requirements of CG data, planning the display process of CG data, and configuring the synchronization architecture (tools and environment) of CG data.

[0030] In one embodiment, the preliminary preparations for completing data synchronization include: S11. Collect and analyze the synchronization requirements of CG data, determine the synchronization goals and requirements; for example, the synchronization frequency, data volume, data type (such as model, texture, animation, etc.). Clarify the target location of synchronization.

[0031] S12. Design the display process of CG data, including user interface and data display method; the user interface includes layout, navigation and interaction method, etc. The data display method includes determining the display type of CG data, selecting appropriate data display technology, data loading method and the interaction method between users and CG data, etc.

[0032] S13. Select and configure synchronization tools and environments suitable for large file transfer and high-concurrency processing; the synchronization tools use commonly used data synchronization and transmission tools, and the environment includes network environment, storage environment, computing resources and security configuration.

[0033] S2. Based on the preliminary preparation results of data synchronization, the CG data assets to be synchronized are synchronized for the first time, and the compression ratio is dynamically adjusted during the full synchronization process.

[0034] In one embodiment, dynamically adjusting the compression ratio during full synchronization includes: S21. Collect the performance indicators of CG data synchronization in history, and the performance indicators include network bandwidth utilization rate, to understand the bandwidth utilization; central processing unit utilization rate, to understand the CPU load; system memory utilization rate, to understand the memory occupancy; data transmission rate, to understand the efficiency of data transmission.

[0035] S22. Determine a reasonable compression ratio range and filter out candidate compression ratios.

[0036] S23. Define an objective function to measure the quality of the compression ratio; for example, the objective function may be a weighted sum of data transmission efficiency and system resource utilization. Define constraints, such as upper limits on network bandwidth utilization, CPU utilization, and system memory utilization.

[0037] S24, using heuristic rules to explore different compression ratios and perform local search to improve the current solution; Heuristic rules: The initial compression ratio is selected using heuristic rules based on historical data and current performance indicators. For example, if the current network bandwidth usage is high, a lower compression ratio can be selected.

[0038] Local Search: Based on the initial compression ratio, a local search is performed to try different compression ratios to find the solution with the optimal objective function value. The objective function values ​​under different compression ratios are compared to select the optimal compression ratio.

[0039] S25. Pre-calculate the best compression ratio under different conditions according to the results of the heuristic search and the local search, and construct a compression ratio table, wherein the best compression ratio under each condition and the corresponding objective function value are recorded. The pre-calculated best compression ratio and the corresponding condition are recorded in the compression ratio table.

[0040] S26. Acquire the performance index of CG data synchronization in real time and calculate the value of the objective function; match the corresponding optimal compression ratio and apply the corresponding optimal compression ratio.

[0041] It should be noted that during the synchronization process, the current performance indicators are obtained in real time, including network bandwidth utilization, system CPU utilization, system memory utilization, and data transmission rate. Based on the performance indicators obtained in real time, the objective function value under the current conditions is calculated. Based on the calculated objective function value, the corresponding optimal compression ratio is searched in the compression ratio table. If there is no completely matching condition, the closest compression ratio can be found by using interpolation or approximate matching methods. The best compression ratio found is applied to the current synchronization task, and the data compression ratio is dynamically adjusted to improve data transmission efficiency and system resource utilization.

[0042] In one embodiment, determining the compression ratio range and filtering out candidate compression ratios includes: S221, determining a lower limit and an upper limit of a compression ratio according to an application scenario; Lower limit: Determine the lower limit of the compression ratio based on the minimum requirements for data transmission and the characteristics of the data compression algorithm. For example, the lowest compression ratio can be set to 0, that is, no compression.

[0043] Upper limit: Determine the upper limit of the compression ratio based on the maximum compression capability of the data compression algorithm and the limitations of system resources. For example, the highest compression ratio can be set to 100, which is the maximum compression.

[0044] S222. In the determined compression ratio range, select several compression ratios at equal intervals as candidate compression ratios; for example, if the compression ratio range is 0 to 100, 0, 20, 40, 60, 80, and 100 may be selected as candidate compression ratios. Ensure that the candidate compression ratios can cover the entire compression ratio range so as to select the best compression ratio under different conditions.

[0045] S3. Configure and execute incremental synchronization, and during the incremental synchronization process, synchronize the changed parts of the CG data based on the differential algorithm.

[0046] In one embodiment, incremental synchronization is configured and performed, and during the incremental synchronization process, synchronizing the changed data in the CG data based on the differential algorithm includes: S31. Calculate the file checksum (for example, MD5, SHA-1) of the source data and the target data, and compare the file checksum of the source data and the target data to obtain the changed data during the CG data incremental synchronization process.

[0047] S32. For the changed data in the incremental synchronization process of CG data, a differential data file is generated using a differential algorithm (for example, rsync algorithm, Delta encoding, etc.). The rsync algorithm is an efficient algorithm for file synchronization and transmission, and Delta encoding (differential encoding) is a data compression technology that reduces the amount of data stored and transmitted by recording the differences between data.

[0048] S33, synchronizing the generated differential data file to the target location, and updating the target file at the target location using the differential data file to make it consistent with the source file. The differential data can be applied using the reverse operation of the differential algorithm to merge the differential data into the target file to make it consistent with the source file.

[0049] S4. Identify key nodes in the synchronization process based on the proximity algorithm and display the transition interface between key nodes in the pre-configured synchronization process; the synchronization process includes full synchronization and incremental synchronization.

[0050] Identify key nodes in the synchronization process based on the proximity algorithm and display the pre-configured transition interface between key nodes in the synchronization process, including: S41. Collect historical CG data synchronization task data, and use the K nearest neighbor algorithm to identify key nodes of the current CG data synchronization task to obtain a key node list.

[0051] S42. According to the key node list, a pre-configured transition interface between key nodes in the synchronization process is displayed to the user.

[0052] In one embodiment, historical CG data synchronization task data is collected, and the K nearest neighbor algorithm is used to identify key nodes of the current CG data synchronization task, and the key node list obtained includes: S411. Obtain the criteria for determining key nodes, and the criteria for determining key nodes include that the CPU usage rate is continuously higher than the first threshold, the memory usage rate is continuously higher than the second threshold, the disk I / O rate is continuously higher than the third threshold, and the network bandwidth utilization rate is continuously higher than the fourth threshold, and the duration exceeds a preset duration. For example, the CPU usage rate is continuously higher than 80%. The memory usage rate is continuously higher than 75%. The disk I / O usage rate is continuously higher than 70%. The network bandwidth utilization rate is continuously higher than 90%. The duration exceeds 5 minutes.

[0053] S412. According to the criteria for determining the key nodes, the historical CG data synchronization task data are collected, and the historical CG data synchronization task data are annotated to obtain an annotated data set; that is, which moments are the key nodes are annotated.

[0054] S413, selecting a distance measurement algorithm and the number of nearest neighbors, and determining the number of selected nearest neighbors through cross-validation; Commonly used distance measurement algorithms include Euclidean distance, Manhattan distance, cosine similarity, etc. Select an appropriate distance measurement algorithm based on the data characteristics. Determine the optimal number of nearest neighbors (K value) through cross-validation. Try different K values ​​and select the K value with the best prediction effect.

[0055] S414, constructing an index structure of the labeled data set and storing the labeled data set; for example, using a KD tree, a ball tree or other efficient data structures to construct an index structure of the labeled data set to speed up the nearest neighbor search.

[0056] S415. Collect the current CG data synchronization task data, use the K nearest neighbor algorithm to analyze the current CG data synchronization task data, and identify key nodes.

[0057] S416. Generate a key node list according to the prediction result of the K nearest neighbor algorithm, where the key node list includes the information and position of each key node.

[0058] In one embodiment, collecting current CG data synchronization task data, using K nearest neighbor algorithm to analyze the current CG data synchronization task data, and identifying key nodes includes the following steps: S4151. Using the selected distance measurement algorithm, calculate the distance between the nodes in the current CG data synchronization task data and the nodes in the labeled data set to obtain a distance list; S4152. According to the distance list, select several neighbors with the closest distance; assign a weight to each neighbor according to the distance, and perform weighted voting; S4153. Determine whether the current data point is a key node based on the voting result.

[0059] In one embodiment, when weights are assigned to each neighbor based on the distance and weighted voting is performed, the weight assignment formula is: ; In the formula, w i Indicates i The weight of a neighbor. During the CG data asset synchronization process, this weight determines the importance of the neighbor node in identifying whether the current node is a key node. The higher the weight, the greater the influence of the neighbor node on whether the current node is a key node.

[0060] d ( x , y i ) indicates the current node x With Neighbors y i The distance between nodes is the distance between nodes. The choice of distance metric depends on the characteristics of the data and the specific requirements of the synchronization task. The smaller the distance, the more similar the two nodes are in the feature space, and the greater the influence of the neighbor node on the current node.

[0061] α Represents the parameter used to adjust the weight sensitivity. α When the value of is large, the influence of distance on weight is more significant, that is, a slight increase in distance will significantly reduce the weight, which makes the model more strict in selecting neighbor nodes. On the contrary, when α When the value of is small, the weight is less sensitive to the change of distance, and the model is more relaxed in selecting neighbor nodes. During the CG data asset synchronization process, you can adjust the α to optimize the recognition of key nodes.

[0062] Key nodes and non-key nodes are considered as categories, and a counter is initialized to accumulate the weighted votes of each category.

[0063] Traverse the neighbor nodes and calculate the weight of each neighbor node based on the distance; according to the category and weight of each neighbor node, update the counter of the corresponding category, and select the category with the largest counter value as the prediction result.

[0064] Collect the current CG data synchronization task data, use the K nearest neighbor algorithm to analyze the current CG data synchronization task data, and identify key nodes. This process includes selecting a distance measurement algorithm, calculating distances, selecting nearest neighbors, assigning weights, weighted voting, and determining key nodes based on voting results, thereby improving the accuracy and efficiency of the data synchronization process.

[0065] In one embodiment, according to the key node list, presenting the pre-configured transition interface between key nodes in the synchronization process to the user includes: S421. Pre-configure a transition effect (including animation, fade-in, fade-out, sliding, etc.) from any node to the next node to obtain a user interface with a smooth transition effect.

[0066] S422. Deploy the user interface to the production environment, monitor user feedback on the user interface, and optimize and adjust the user interface based on the feedback.

[0067] S5. Monitor the full synchronization and incremental synchronization process, and record the monitoring information of each full synchronization and incremental synchronization, and the monitoring information includes the execution status of key nodes and the display status of the transition interface.

[0068] In one embodiment, monitoring the full synchronization and incremental synchronization process and recording the monitoring information of each full synchronization and incremental synchronization includes: S51. Configure monitoring tools to record the execution status of key nodes during full synchronization and incremental synchronization.

[0069] During each full and incremental synchronization process, the execution status of key nodes is recorded, and the execution status includes success, failure, and execution time; this information is recorded in a log file or database for subsequent analysis and optimization.

[0070] S52, obtaining the display effect of the transition interface, ensuring that the user interface meets expectations, and recording user feedback. For example, user feedback is collected through questionnaires, user interviews, etc., and recorded in a database or log file.

[0071] In order to facilitate understanding of the above technical solutions of the present invention, the working principle or operation mode of the present invention in the actual process is described in detail below.

[0072] 1. Preliminary preparation Target data asset types: including models (A), textures (B), animations (C). Synchronization frequency: set to daily (D) to ensure that the latest data is updated in a timely manner. Data volume: The total amount of data to be synchronized is expected to be approximately 100GB (E), including multiple file formats.

[0073] Synchronization target location: Choose a cloud storage solution (F) to facilitate cross-regional access and data security. User interface layout: Use a responsive grid layout (G) to ensure good display on different devices. Data display method: Use 3D view (H) combined with list view (I) to facilitate users to intuitively understand data. Interaction method: Support drag and drop (J) and single-click selection (K) to improve user experience.

[0074] Tool selection: Use rsync (L) for efficient data synchronization, support incremental synchronization, and reduce the amount of data transferred. Network environment: Ensure a high-speed Internet connection (M) with a bandwidth of at least 100Mbps to meet the needs of large file transfers. Storage environment: Choose a reliable cloud storage service (N) to ensure high availability and security of data.

[0075] Computing resources: Configure at least a 4-core CPU and 16 GB of memory (O) to ensure sufficient computing power during the synchronization process.

[0076] 2. Initial full synchronization 2.1. Collect historical performance metrics Network bandwidth usage: 70% (P) is monitored, ensuring that the bandwidth is not saturated. CPU usage: 50% (Q) is monitored, indicating that the current load is reasonable. Memory usage: 60% (R) is monitored, which can support more data processing. Data transmission rate: 5MB / s (S) is monitored, providing a basis for subsequent optimization.

[0077] 2.2. Determine a reasonable compression ratio range Lower limit: Based on the minimum requirement for data transmission, this value is set to 0% (T), indicating that no compression is performed.

[0078] Upper limit: Based on the maximum compression capability of the data compression algorithm, set to 100% (U) to ensure maximum utilization of storage space.

[0079] 2.3. Define the objective function Objective function example: Efficiency (V) = Data transfer rate (S) / [CPU usage (Q) + Memory usage (R)], which is used to measure the effectiveness of the current compression ratio.

[0080] 2.4. Dynamically adjust the compression ratio Initial compression ratio: Set to 50% (W) to balance compression effect and transmission efficiency.

[0081] Process adjustment: During the synchronization process, performance indicators are monitored in real time. If the network bandwidth usage is high, the compression ratio is adjusted to 30% (X) to improve data transmission efficiency.

[0082] 2.5. Constructing a compression ratio table Recording conditions and corresponding compression ratios: Condition A (P, Q, R): The best compression ratio is found to be 40% (Y), which is suitable for situations with high network bandwidth.

[0083] Condition B (M, Q, R): The best compression ratio is found to be 30% (Z), which is suitable for situations where system resources are tight.

[0084] 2.6. Obtain performance indicators in real time During the synchronization process, the network bandwidth usage, CPU usage, memory usage and data transfer rate are continuously monitored, and the objective function value is calculated to match and apply the optimal compression ratio.

[0085] 3. Incremental synchronization 3.1. Calculate file checksum Source data file checksum: Use the MD5 algorithm to generate the checksum (A1) of source data file A. Target data file checksum: Generate the checksum (B1) of target data file B. Comparison result: It is found that file C (such as texture file) has changed and needs to be synchronized.

[0086] 3.2. Using the difference algorithm The rsync algorithm is used to generate the difference of file D, extract the changed part, and create a differential data file E, which is about 50MB in size, which improves the transmission efficiency compared to large files.

[0087] 3.3 Synchronous differential data Synchronize the differential data file E to the target location F, and use the reverse operation of rsync to apply the differential data to the target file to ensure that the target data is consistent with the source data.

[0088] 4. Identify key nodes 4.1. Using K nearest neighbor algorithm Collect historical synchronization task data and generate a key node list (G1), including successful and failed execution status.

[0089] 4.2. Display transition interface Pre-configure transition effects to enhance the user experience and ensure that the user interface is smooth and natural when switching between key nodes.

[0090] 5. Monitor the synchronization process 5.1. Configure monitoring tools Select and configure monitoring tools to record the execution status of key nodes during full and incremental synchronization. Status information, including success, failure, and execution time, is recorded in a log file (I1) for subsequent analysis and optimization.

[0091] 5.2. Get user feedback Through questionnaire surveys and user interviews, users’ feedback on the transition interface and data synchronization process is collected, and the feedback results are recorded in the database (J1). The interface and process can be optimized and adjusted based on user feedback later.

[0092] In summary, the present invention can optimize data transmission according to network conditions by dynamically adjusting the compression ratio, and significantly improve the synchronization speed of large files. The dynamic adjustment method can flexibly respond to different network and system environments, and improve the versatility and adaptability of the method. The present invention uses incremental synchronization and differential algorithms to only transmit the changed parts, which reduces bandwidth usage and storage requirements, and reduces the occupation of system resources. The present invention configures tools and environments suitable for high-concurrency processing, so that multiple users can synchronize data at the same time, improving collaboration efficiency. The present invention ensures that the user's operation during the synchronization process is more natural and smooth through the identification of key nodes and the smooth transition interface display, reducing the trouble in use. At the same time, the proximity algorithm is used to analyze historical data, which can accurately identify the key nodes in the synchronization process, enhance the intelligence of key node identification during CG data synchronization, and make fast and accurate judgments when facing complex real-time data. By identifying key nodes in the CG data synchronization process, the present invention can prioritize the allocation of computing and network resources in the data synchronization process to improve synchronization efficiency.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatic synchronization of CG data assets, characterized in that: The CG data asset automatic synchronization method includes: S1. Acquire and classify the CG data assets to be synchronized and complete the preliminary preparation for data synchronization; the preliminary preparation includes acquiring the synchronization requirements of CG data, planning the display process of CG data and configuring the synchronization architecture of CG data; S2. Based on the preliminary preparation results of data synchronization, the CG data assets to be synchronized are synchronized for the first time, and the compression ratio is dynamically adjusted during the full synchronization process; S3. Configure and execute incremental synchronization, and during the incremental synchronization process, synchronize the changed data in the CG data based on the differential algorithm; S4, identifies key nodes in the synchronization process based on the proximity algorithm, and displays the transition interface between key nodes in the pre-configured synchronization process; the synchronization process includes full synchronization and incremental synchronization; S5. Monitor the full synchronization and incremental synchronization process, and record the monitoring information of each full synchronization and incremental synchronization, and the monitoring information includes the execution status of key nodes and the display status of the transition interface; The method of identifying key nodes in the synchronization process based on a proximity algorithm and displaying a pre-configured transition interface between key nodes in the synchronization process includes: S41, collecting historical CG data synchronization task data, and using the K nearest neighbor algorithm to identify key nodes of the current CG data synchronization task to obtain a key node list; S42. According to the key node list, a pre-configured transition interface between key nodes in the synchronization process is displayed to the user.

2. A method for automatic synchronization of CG data assets according to claim 1, characterized in that: The preliminary preparations for completing data synchronization include: S11. Collect and analyze the synchronization requirements of CG data and determine the synchronization goals and requirements; S12. Design the display process of CG data, including user interface and data display method; S13. Select and configure synchronization tools and environments suitable for large file transfer and high-concurrency processing.

3. The method for automatic synchronization of CG data assets according to claim 1, characterized in that: The dynamic adjustment of the compression ratio during the full synchronization process includes: S21. Collecting historical performance indicators of CG data synchronization, where the performance indicators include network bandwidth utilization, CPU utilization, system memory utilization, and data transmission rate; S22, determining a compression ratio range, and filtering out candidate compression ratios; S23, defining an objective function to measure the quality of the compression ratio; defining constraints; S24, using heuristic rules to explore different compression ratios and perform local search to improve the current solution; S25. Pre-calculate the best compression ratio under different conditions according to the results of the heuristic search and the local search, and construct a compression ratio table; S26. Acquire the performance index of CG data synchronization in real time, and calculate the value of the objective function; match the corresponding optimal compression ratio.

4. A method for automatic synchronization of CG data assets according to claim 3, characterized in that: Determining the compression ratio range and filtering out candidate compression ratios includes: S221, determining a lower limit and an upper limit of a compression ratio according to an application scenario; S222. Within the determined compression ratio range, select a number of compression ratios at equal intervals as candidate compression ratios.

5. A method for automatic synchronization of CG data assets according to claim 4, characterized in that: The configuration and execution of incremental synchronization, and in the incremental synchronization process, synchronizing the changed data in the CG data based on the differential algorithm includes: S31, calculate the file checksum of the source data and the target data, and compare the file checksum of the source data and the target data to obtain the changed data in the process of CG data incremental synchronization; S32, using a differential algorithm to generate a differential data file for the changed data in the CG data incremental synchronization process; S33, synchronizing the generated differential data file to the target location, and updating the target file at the target location using the differential data file.

6. The method for automatic synchronization of CG data assets according to claim 1, characterized in that: The historical CG data synchronization task data is collected, and the key nodes of the current CG data synchronization task are identified using the K nearest neighbor algorithm, and the key node list obtained includes: S411, obtaining a determination standard for a key node, wherein the determination standard for a key node includes that a CPU usage rate is continuously higher than a first threshold, a memory usage rate is continuously higher than a second threshold, a disk I / O rate is continuously higher than a third threshold, and a network bandwidth utilization rate is continuously higher than a fourth threshold, and the duration of the duration exceeds a preset duration; S412, according to the determination standard of the key nodes, collecting the historical CG data synchronization task data, and annotating the historical CG data synchronization task data to obtain an annotated data set; S413, selecting a distance measurement algorithm and the number of nearest neighbors, and determining the number of selected nearest neighbors through cross-validation; S414, constructing an index structure of the labeled data set, and storing the labeled data set; S415, collecting current CG data synchronization task data, using K nearest neighbor algorithm to analyze the current CG data synchronization task data, and identifying key nodes; S416. Generate a key node list according to the prediction result of the K nearest neighbor algorithm, where the key node list includes the information and position of each key node.

7. A method for automatic synchronization of CG data assets according to claim 6, characterized in that: The collecting of current CG data synchronization task data, analyzing the current CG data synchronization task data using the K nearest neighbor algorithm, and identifying key nodes comprises the following steps: S4151. Using the selected distance measurement algorithm, calculate the distance between the nodes in the current CG data synchronization task data and the nodes in the labeled data set to obtain a distance list; S4152. According to the distance list, select several neighbors with the closest distance; assign a weight to each neighbor according to the distance, and perform weighted voting; S4153. Determine whether the current data point is a key node based on the voting result.

8. A method for automatic synchronization of CG data assets according to claim 7, characterized in that: When assigning weights to each neighbor based on the distance and performing weighted voting, the weight assignment formula is: ; In the formula, w i Indicates i The weight of the neighbors, d ( x , y i ) indicates the current node x With Neighbors y i The distance between α represents the parameter used to adjust the weight sensitivity; Take key nodes and non-key nodes as categories, and initialize a counter to accumulate weighted votes for each category; Traverse the neighbor nodes and calculate the weight of each neighbor node based on the distance; According to the category and weight of each neighbor node, the counter of the corresponding category is updated, and the category with the largest counter value is selected as the prediction result.

9. A method for automatic synchronization of CG data assets according to claim 8, characterized in that: The step of displaying the transition interface between the key nodes in the pre-configured synchronization process to the user according to the key node list includes: S421, pre-configure the transition effect from any node to the next node to obtain a user interface; S422. Deploy the user interface to the production environment, monitor user feedback on the user interface, and optimize and adjust the user interface based on the feedback.

10. The method for automatic synchronization of CG data assets according to claim 1, characterized in that: The monitoring of the full synchronization and incremental synchronization process and recording of the monitoring information of each full synchronization and incremental synchronization include: S51. Configure monitoring tools and record the execution status of key nodes during each full synchronization and incremental synchronization process, and the execution status includes success, failure and execution time; S52: Obtain the display effect of the transition interface and record user feedback.

Citation Information

Patent Citations

  • Method for achieving automatic synchronization of multi-source heterogeneous data resources

    CN103617176A

  • Data synchronization method and system based on log analysis

    CN116361256A

  • Cross-environment data synchronization method and device

    CN118568181A

  • Data synchronization method and apparatus

    WO2022143081A1

  • Data synchronization method and apparatus

    WO2023109286A1

Cited By

  • Column storage database compression synchronization method

    CN121166820A