A method for automatic synchronization of CG data assets

Abstract

The present invention discloses a method for automatic synchronization of CG data assets, which relates to the field of computer image data synchronization, and includes: completing preliminary preparations for data synchronization; based on the preliminary preparation results of data synchronization, performing initial full synchronization of the CG data assets to be synchronized, and dynamically adjusting the compression ratio during the full synchronization process; configuring and executing incremental synchronization, and during the incremental synchronization process, synchronizing the changed data in the CG data based on a differential algorithm; identifying key nodes in the synchronization process based on a proximity algorithm, and displaying the transition interface between each key node in the pre-configured synchronization process; monitoring the full synchronization and incremental synchronization processes, and recording the monitoring information of each full synchronization and incremental synchronization. The present invention improves transmission efficiency, reduces resource consumption, enhances high-concurrency processing capabilities, and improves user experience.

Description

Technical Field

[0001] The present invention relates to the field of computer image data synchronization, and in particular to a method for automatic synchronization of CG data assets. Background Art

[0002] CG data assets (Computer Graphics Data Assets) generally refer to various digital resources used in the field of computer graphics. These resources can include, but are not limited to, 3D models, textures, animations, scenes, lighting settings, materials, special effects, and more. CG data assets are widely used in film and television production, game development, virtual reality, architectural visualization, and other fields.

[0003] Automatic CG data asset synchronization refers to the process of automatically updating and synchronizing these data assets across different storage locations or systems using automated technologies and tools. This automatic synchronization ensures that when multiple teams or systems use the same data asset, they always access the latest version, thereby improving work efficiency and data consistency.

[0004] With the rapid development of computer graphics (CG) technology, the scale of CG data assets continues to expand, especially the use of large files (such as 3D models and textures). The synchronization and management of these data assets has become a major technical challenge, mainly facing the following technical issues:

[0005] (1) Low transmission efficiency of large files: Large files may face bandwidth bottlenecks during transmission, resulting in low transmission efficiency.

[0006] (2) Insufficient high-concurrency processing capabilities: In a multi-person collaborative environment, how to efficiently synchronize multiple large files is a difficult problem.

[0007] (3) Data change detection is not timely: Traditional synchronization methods often require full transmission, resulting in waste of resources.

[0008] (4) Poor user experience: During the data synchronization process, users may encounter interface freezes or unsmooth transitions.

[0009] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0010] In response to the problems in the related art, the present invention proposes a method for automatic synchronization of CG data assets to overcome the above-mentioned technical problems existing in the existing related art.

[0011] To this end, the specific technical solutions adopted in the present invention are as follows:

[0012] A method for automatically synchronizing CG data assets, the method comprising:

[0013] S1. Acquire and classify the CG data assets to be synchronized and complete the preliminary preparations for data synchronization. The preliminary preparations include obtaining the synchronization requirements for CG data, planning the display process for CG data, and configuring the synchronization architecture for CG data.

[0014] S2. Based on the preliminary preparation results of data synchronization, perform the initial full synchronization of the CG data assets to be synchronized, and dynamically adjust the compression ratio during the full synchronization process;

[0015] S3. Configure and execute incremental synchronization, and during the incremental synchronization process, synchronize the changed data in the CG data based on the differential algorithm;

[0016] S4. Identify key nodes in the synchronization process based on a proximity algorithm and display the transition interface between key nodes in the pre-configured synchronization process. The synchronization process includes full synchronization and incremental synchronization.

[0017] S5. Monitor the full and incremental synchronization processes and record monitoring information for each full and incremental synchronization, including the execution status of key nodes and the display status of the transition interface.

[0018] Among them, the key nodes in the synchronization process are identified based on the proximity algorithm, and the transition interface between the key nodes in the pre-configured synchronization process is displayed, including:

[0019] S41. Collect historical CG data synchronization task data, and use the K nearest neighbor algorithm to identify key nodes of the current CG data synchronization task to obtain a key node list;

[0020] S42. According to the key node list, a pre-configured transition interface between key nodes in the synchronization process is displayed to the user.

[0021] Furthermore, the preliminary preparations for completing data synchronization include:

[0022] S11. Collect and analyze the synchronization requirements of CG data and determine the synchronization goals and requirements;

[0023] S12. Design the display process of CG data, including user interface and data display method;

[0024] S13. Select and configure synchronization tools and environments suitable for large file transfer and high-concurrency processing.

[0025] Furthermore, dynamic adjustment of the compression ratio during full synchronization includes:

[0026] S21. Collect historical performance indicators of CG data synchronization, including network bandwidth usage, CPU usage, system memory usage, and data transmission rate;

[0027] S22. Determine a compression ratio range and filter out candidate compression ratios;

[0028] S23. Define an objective function to measure the quality of the compression ratio; define constraints;

[0029] S24, using heuristic rules to explore different compression ratios and perform local search to improve the current solution;

[0030] S25. Pre-calculating optimal compression ratios under different conditions based on the results of the heuristic search and the local search, and constructing a compression ratio table;

[0031] S26. Acquire the performance indicators of CG data synchronization in real time, calculate the value of the objective function, and match the corresponding optimal compression ratio.

[0032] Furthermore, the compression ratio range is determined, and candidate compression ratios are filtered out, including:

[0033] S221. Determine a lower limit and an upper limit of the compression ratio according to the application scenario;

[0034] S222: Within the determined compression ratio range, select several compression ratios at equal intervals as candidate compression ratios.

[0035] Furthermore, incremental synchronization is configured and executed, and during the incremental synchronization process, the changed data in the CG data is synchronized based on the differential algorithm, including:

[0036] S31. Calculate the file checksum of the source data and the target data, and compare the file checksum of the source data and the target data to obtain the changed data in the CG data incremental synchronization process;

[0037] S32. Using a differential algorithm to generate a differential data file for the changed data during the CG data incremental synchronization process;

[0038] S33: Synchronize the generated differential data file to the target location, and update the target file at the target location using the differential data file.

[0039] Furthermore, we collected historical CG data synchronization task data and used the K nearest neighbor algorithm to identify key nodes of the current CG data synchronization task. The key node list we obtained included:

[0040] S411. Obtaining criteria for determining a key node, where the criteria for determining a key node include a CPU usage continuously exceeding a first threshold, a memory usage continuously exceeding a second threshold, a disk I / O continuously exceeding a third threshold, and a network bandwidth usage continuously exceeding a fourth threshold, and the duration of the continuous indication exceeds a preset time period;

[0041] S412. According to the criteria for determining key nodes, historical CG data synchronization task data is collected and annotated to obtain an annotated data set.

[0042] S413, selecting a distance measurement algorithm and the number of nearest neighbors, and determining the number of selected nearest neighbors through cross-validation;

[0043] S414: Construct an index structure for the labeled data set and store the labeled data set;

[0044] S415, collecting current CG data synchronization task data, using K nearest neighbor algorithm to analyze the current CG data synchronization task data, and identifying key nodes;

[0045] S416. Generate a key node list based on the prediction result of the K-nearest neighbor algorithm. The key node list includes information and location of each key node.

[0046] Furthermore, the current CG data synchronization task data is collected and analyzed using the K nearest neighbor algorithm to identify key nodes, including the following steps:

[0047] S4151. Using the selected distance measurement algorithm, calculate the distance between the nodes in the current CG data synchronization task data and the nodes in the labeled data set to obtain a distance list;

[0048] S4152. Select several neighbors with the closest distance based on the distance list; assign a weight to each neighbor based on the distance, and perform weighted voting;

[0049] S4153. Determine whether the current data point is a key node based on the voting result.

[0050] Furthermore, when assigning weights to each neighbor based on distance and performing weighted voting, the weight assignment formula is:

[0051] ;

[0052] Where, w i Indicates the i The weight of the neighbors, d ( x , yi ) represents the current node x With neighbors y i The distance between α Represents the parameter used to adjust the weight sensitivity;

[0053] Treat key nodes and non-key nodes as categories and initialize a counter to accumulate weighted votes for each category;

[0054] Traverse the neighbor nodes and calculate the weight of each neighbor node based on the distance; according to the category and weight of each neighbor node, update the counter of the corresponding category, and select the category with the largest counter value as the prediction result.

[0055] Furthermore, based on the key node list, the transition interface between each key node in the pre-configured synchronization process is displayed to the user, including:

[0056] S421, pre-configure the transition effect from any node to the next node to obtain a user interface;

[0057] S422. Deploy the user interface to the production environment, monitor user feedback on the user interface, and optimize and adjust the user interface based on the feedback.

[0058] Furthermore, the monitoring process of full and incremental synchronization is monitored, and the monitoring information of each full and incremental synchronization is recorded, including:

[0059] S51. Configure monitoring tools and record the execution status of key nodes during each full and incremental synchronization process. The execution status includes success, failure, and execution time.

[0060] S52: Obtain the display effect of the transition interface and record user feedback.

[0061] The beneficial effects of the present invention are:

[0062] (1) Improved transmission efficiency: By dynamically adjusting the compression ratio, the present invention can optimize data transmission based on network conditions, significantly increasing the synchronization speed of large files. This dynamic adjustment method can flexibly adapt to different network and system environments, improving the versatility and adaptability of the method.

[0063] (2) Reduce resource consumption: The present invention uses incremental synchronization and differential algorithms to transmit only the changed parts, which reduces bandwidth usage and storage requirements and reduces the occupation of system resources.

[0064] (3) Enhanced high-concurrency processing capabilities: The present invention configures tools and environments suitable for high-concurrency processing, allowing multiple users to synchronize data at the same time and improve collaboration efficiency.

[0065] (4) Improved user experience: The present invention ensures that users can operate more naturally and smoothly during the synchronization process through the identification of key nodes and the smooth transition interface display, reducing the troubles in use. At the same time, the proximity algorithm is used to analyze historical data to accurately identify the key nodes in the synchronization process, enhancing the intelligent level of key node identification during CG data synchronization, and making quick and accurate judgments when facing complex real-time data.

[0066] (5) Identification of key nodes: By identifying key nodes during the CG data synchronization process, the present invention can prioritize the allocation of computing and network resources during the data synchronization process, thereby improving synchronization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 This is a flowchart of a method for automatic synchronization of CG data assets according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. By referring to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0070] According to an embodiment of the present invention, a method for automatic synchronization of CG data assets is provided.

[0071] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to an embodiment of the present invention, the automatic synchronization method of CG data assets includes:

[0072] S1. Acquire and classify the CG data assets to be synchronized and complete the preliminary preparations for data synchronization; preliminary preparations include obtaining the synchronization requirements of CG data, planning the display process of CG data, and configuring the synchronization architecture (tools and environment) of CG data.

[0073] In one embodiment, the preliminary preparations for completing data synchronization include:

[0074] S11. Collect and analyze CG data synchronization requirements and determine synchronization goals and requirements; for example, synchronization frequency, data volume, and data type (such as models, textures, animations, etc.). Clarify the target location for synchronization.

[0075] S12. Design the display process of CG data, including the user interface and data display method; the user interface includes layout, navigation and interaction methods, etc. The data display method includes determining the display type of CG data, selecting appropriate data display technology, data loading method and the user interaction method with CG data, etc.

[0076] S13. Select and configure synchronization tools and environments suitable for large file transfer and high-concurrency processing; synchronization tools use commonly used data synchronization and transmission tools, and the environment includes network environment, storage environment, computing resources and security configuration.

[0077] S2. Based on the preliminary preparation results of data synchronization, perform the initial full synchronization of the CG data assets to be synchronized, and dynamically adjust the compression ratio during the full synchronization process.

[0078] In one embodiment, dynamically adjusting the compression ratio during full synchronization includes:

[0079] S21. Collect historical performance indicators of CG data synchronization, including network bandwidth usage to understand bandwidth utilization; central processing unit usage to understand CPU load; system memory usage to understand memory occupancy; and data transmission rate to understand data transmission efficiency.

[0080] S22. Determine a reasonable compression ratio range and filter out candidate compression ratios.

[0081] S23. Define an objective function to measure the quality of the compression ratio; for example, the objective function can be a weighted sum of data transmission efficiency and system resource utilization. Define constraints, such as upper limits on network bandwidth utilization, CPU utilization, and system memory utilization.

[0082] S24, using heuristic rules to explore different compression ratios and perform local search to improve the current solution;

[0083] Heuristic rules:

[0084] An initial compression ratio is selected using heuristic rules based on historical data and current performance indicators. For example, if the current network bandwidth usage is high, a lower compression ratio may be selected.

[0085] Local Search:

[0086] Based on the initial compression ratio, a local search is performed, trying different compression ratios to find the solution with the optimal objective function value. The objective function values ​​under different compression ratios are compared and the optimal compression ratio is selected.

[0087] S25. Precalculate the optimal compression ratio under different conditions based on the results of the heuristic search and the local search, and construct a compression ratio table, wherein the optimal compression ratio under each condition and the corresponding objective function value are recorded. The precalculated optimal compression ratio and the corresponding condition are recorded in the compression ratio table.

[0088] S26. Acquire the performance indicators of CG data synchronization in real time and calculate the value of the objective function; match the corresponding optimal compression ratio and apply the corresponding optimal compression ratio.

[0089] It should be noted that during the synchronization process, current performance indicators, including network bandwidth utilization, system CPU utilization, system memory utilization, and data transfer rate, are acquired in real time. Based on these real-time performance indicators, the objective function value under the current conditions is calculated. Based on this calculated objective function value, the corresponding optimal compression ratio is searched in the compression ratio table. If no exact match exists, interpolation or approximate matching methods can be used to find the closest compression ratio. The optimal compression ratio found is applied to the current synchronization task, dynamically adjusting the data compression ratio to improve data transmission efficiency and system resource utilization.

[0090] In one embodiment, determining the compression ratio range and filtering out candidate compression ratios includes:

[0091] S221. Determine a lower limit and an upper limit of the compression ratio according to the application scenario;

[0092] Lower limit: The lower limit of the compression ratio is determined based on the minimum data transmission requirements and the characteristics of the data compression algorithm. For example, the minimum compression ratio can be set to 0, which means no compression.

[0093] Upper limit: The upper limit of the compression ratio is determined based on the maximum compression capability of the data compression algorithm and the limitations of system resources. For example, the maximum compression ratio can be set to 100, which is the maximum compression.

[0094] S222. Within the determined compression ratio range, select several compression ratios at equal intervals as candidate compression ratios. For example, if the compression ratio range is 0 to 100, 0, 20, 40, 60, 80, and 100 may be selected as candidate compression ratios. Ensure that the candidate compression ratios cover the entire compression ratio range so that the optimal compression ratio can be selected under different conditions.

[0095] S3. Configure and execute incremental synchronization, and during the incremental synchronization process, synchronize the changed parts of the CG data based on the differential algorithm.

[0096] In one embodiment, configuring and executing incremental synchronization, and synchronizing the changed data in the CG data based on the differential algorithm during the incremental synchronization process includes:

[0097] S31. Calculate the file checksum (e.g., MD5, SHA-1) of the source data and the target data, and compare the file checksum of the source data and the target data to obtain the changed data during the CG data incremental synchronization process.

[0098] S32. For the changed data during the incremental synchronization of CG data, use a differential algorithm (e.g., rsync algorithm, Delta encoding, etc.) to generate a differential data file. The rsync algorithm is a highly efficient algorithm for file synchronization and transmission, and Delta encoding is a data compression technology that reduces the amount of data stored and transmitted by recording the differences between data.

[0099] S33, synchronizing the generated differential data file to the target location, and updating the target file at the target location using the differential data file to make it consistent with the source file. The differential data can be applied using the reverse operation of the differential algorithm to merge the differential data into the target file to make it consistent with the source file.

[0100] S4. Identify key nodes in the synchronization process based on a proximity algorithm and display the transition interface between key nodes in the pre-configured synchronization process; the synchronization process includes full synchronization and incremental synchronization.

[0101] Identifies key nodes in the synchronization process based on a proximity algorithm and displays pre-configured transition interfaces between key nodes in the synchronization process, including:

[0102] S41. Collect historical CG data synchronization task data, and use the K-nearest neighbor algorithm to identify key nodes of the current CG data synchronization task to obtain a key node list.

[0103] S42. According to the key node list, a pre-configured transition interface between key nodes in the synchronization process is displayed to the user.

[0104] In one embodiment, historical CG data synchronization task data is collected, and the K nearest neighbor algorithm is used to identify key nodes of the current CG data synchronization task. The obtained key node list includes:

[0105] S411. Obtain criteria for determining key nodes, where the criteria include CPU usage continuously exceeding a first threshold, memory usage continuously exceeding a second threshold, disk I / O usage continuously exceeding a third threshold, and network bandwidth usage continuously exceeding a fourth threshold, where the duration of the duration exceeds a preset time period. For example, CPU usage continuously exceeds 80%, memory usage continuously exceeds 75%, disk I / O usage continuously exceeds 70%, and network bandwidth usage continuously exceeds 90%. The duration exceeds 5 minutes.

[0106] S412. According to the criteria for determining key nodes, collect historical CG data synchronization task data, and perform labeling on the historical CG data synchronization task data to obtain a labeled data set; that is, label which moments are key nodes.

[0107] S413, selecting a distance measurement algorithm and the number of nearest neighbors, and determining the number of selected nearest neighbors through cross-validation;

[0108] Common distance metrics include Euclidean distance, Manhattan distance, and cosine similarity. Choose an appropriate distance metric based on the characteristics of your data. Use cross-validation to determine the optimal number of nearest neighbors (K). Experiment with different K values ​​and select the one that yields the best prediction results.

[0109] S414: Construct an index structure for the labeled data set and store the labeled data set; for example, use a KD tree, a ball tree, or other efficient data structures to construct an index structure for the labeled data set to speed up the nearest neighbor search.

[0110] S415. Collect the current CG data synchronization task data, use the K nearest neighbor algorithm to analyze the current CG data synchronization task data, and identify key nodes.

[0111] S416. Generate a key node list based on the prediction result of the K-nearest neighbor algorithm. The key node list includes information and location of each key node.

[0112] In one embodiment, collecting current CG data synchronization task data, analyzing the current CG data synchronization task data using a K nearest neighbor algorithm, and identifying key nodes include the following steps:

[0113] S4151. Using the selected distance measurement algorithm, calculate the distance between the nodes in the current CG data synchronization task data and the nodes in the labeled data set to obtain a distance list;

[0114] S4152. Select several neighbors with the closest distance based on the distance list; assign a weight to each neighbor based on the distance, and perform weighted voting;

[0115] S4153. Determine whether the current data point is a key node based on the voting result.

[0116] In one embodiment, when assigning a weight to each neighbor based on the distance and performing weighted voting, the weight assignment formula is:

[0117] ;

[0118] Where, w i Indicates the i The weight of each neighbor. During the CG data asset synchronization process, this weight determines the importance of the neighbor node in identifying whether the current node is a key node. The higher the weight, the greater the influence of the neighbor node on whether the current node is a key node.

[0119] d ( x , y i ) represents the current node x With neighbors y i The distance between nodes is the distance between them. The choice of distance metric depends on the characteristics of the data and the specific requirements of the synchronization task. The smaller the distance, the more similar the two nodes are in the feature space, and the greater the influence of the neighboring node on the current node.

[0120] α Represents the parameter used to adjust the weight sensitivity. α When the value of is large, the influence of distance on weight is more significant, that is, a slight increase in distance will significantly reduce the weight, which makes the model more strict in selecting neighbor nodes. On the contrary, when α When the value of is small, the weight is less sensitive to the change of distance, and the model is more relaxed in selecting neighbor nodes. During the CG data asset synchronization process, you can adjust the value of α to optimize the recognition of key nodes.

[0121] Key nodes and non-key nodes are considered as categories, and a counter is initialized to accumulate the weighted votes of each category.

[0122] Traverse the neighbor nodes and calculate the weight of each neighbor node based on the distance; according to the category and weight of each neighbor node, update the counter of the corresponding category, and select the category with the largest counter value as the prediction result.

[0123] Collect the current CG data synchronization task data and analyze it using the K-nearest neighbor algorithm to identify key nodes. This process includes selecting a distance metric algorithm, calculating distances, selecting nearest neighbors, assigning weights, performing weighted voting, and determining key nodes based on the voting results, thereby improving the accuracy and efficiency of the data synchronization process.

[0124] In one embodiment, presenting a pre-configured transition interface between key nodes in the synchronization process to the user according to the key node list includes:

[0125] S421 , pre-configuring a transition effect (including animation, fade-in, fade-out, sliding, etc.) from any node to the next node to obtain a user interface with a smooth transition effect.

[0126] S422. Deploy the user interface to the production environment, monitor user feedback on the user interface, and optimize and adjust the user interface based on the feedback.

[0127] S5. Monitor the full synchronization and incremental synchronization process, and record the monitoring information of each full synchronization and incremental synchronization. The monitoring information includes the execution status of key nodes and the display status of the transition interface.

[0128] In one embodiment, monitoring the full synchronization and incremental synchronization process and recording monitoring information for each full synchronization and incremental synchronization includes:

[0129] S51. Configure monitoring tools to record the execution status of key nodes during full and incremental synchronization.

[0130] During each full and incremental synchronization process, record the execution status of key nodes, including success, failure, and execution time; record this information in a log file or database for subsequent analysis and optimization.

[0131] S52: Obtain the display effect of the transition interface, ensure that the user interface meets expectations, and record user feedback. For example, user feedback can be collected through questionnaires, user interviews, etc., and recorded in a database or log file.

[0132] In order to facilitate understanding of the above technical solutions of the present invention, the working principle or operation mode of the present invention in actual process is described in detail below.

[0133] 1. Preliminary preparation

[0134] Target data asset types: Includes models (A), textures (B), and animations (C). Synchronization frequency: Set to daily (D) to ensure timely updates of the latest data. Data volume: The total data volume to be synchronized is estimated to be approximately 100GB (E), including various file formats.

[0135] Synchronization Target: Choose a cloud storage solution (F) for cross-regional access and data security. User Interface Layout: Use a responsive grid layout (G) to ensure optimal display across different devices. Data Presentation: Use a 3D view (H) combined with a list view (I) to facilitate intuitive data understanding. Interaction: Support drag-and-drop (J) and single-click selection (K) to enhance the user experience.

[0136] Tool Selection: Use rsync (L) for efficient data synchronization, supporting incremental synchronization and reducing data transfer. Network Environment: Ensure a high-speed internet connection (M) with a bandwidth of at least 100Mbps to accommodate large file transfers. Storage Environment: Choose a reliable cloud storage service (N) to ensure high data availability and security.

[0137] Computing resources: Configure at least a 4-core CPU and 16GB of memory (O) to ensure sufficient computing power during the synchronization process.

[0138] 2. Initial full synchronization

[0139] 2.1. Collecting Historical Performance Metrics

[0140] Network bandwidth utilization: 70% (P) is monitored, ensuring bandwidth is not saturated. CPU utilization: 50% (Q) indicates a reasonable load. Memory utilization: 60% (R) indicates support for more data processing. Data transfer rate: 5MB / s (S) is monitored, providing a foundation for subsequent optimization.

[0141] 2.2. Determine a reasonable compression ratio range

[0142] Lower limit: Based on the minimum requirement for data transmission, set it to 0% (T), indicating no compression.

[0143] Upper limit: Based on the maximum compression capability of the data compression algorithm, set to 100% (U) to ensure maximum utilization of storage space.

[0144] 2.3. Define the objective function

[0145] Example of objective function: Efficiency (V) = Data transfer rate (S) / [CPU usage (Q) + Memory usage (R)], which is used to measure the effectiveness of the current compression ratio.

[0146] 2.4. Dynamically adjust the compression ratio

[0147] Initial compression ratio: Set to 50% (W) to balance compression effect and transmission efficiency.

[0148] Process adjustment: During the synchronization process, performance indicators are monitored in real time. If the network bandwidth usage is high, the compression ratio is adjusted to 30% (X) to improve data transmission efficiency.

[0149] 2.5. Constructing a Compression Ratio Table

[0150] Recording conditions and corresponding compression ratios:

[0151] Condition A (P, Q, R): The best compression ratio is found to be 40% (Y), which is suitable for situations with high network bandwidth.

[0152] Condition B (M, Q, R): The best compression ratio is found to be 30% (Z), which is suitable for situations where system resources are limited.

[0153] 2.6. Obtaining performance indicators in real time

[0154] During the synchronization process, network bandwidth usage, CPU usage, memory usage, and data transfer rate are continuously monitored, and the objective function value is calculated to match and apply the optimal compression ratio.

[0155] 3. Incremental synchronization

[0156] 3.1. Calculate file checksum

[0157] Source data file checksum: Use the MD5 algorithm to generate a checksum (A1) for source data file A. Target data file checksum: Generate a checksum (B1) for target data file B. Comparison result: File C (e.g., a texture file) has changed and needs to be synchronized.

[0158] 3.2. Using the differential algorithm

[0159] The rsync algorithm is used to generate the difference of file D, extract the changed part, and create the differential data file E, which is about 50MB in size. Compared with large files, the transfer efficiency is improved.

[0160] 3.3 Synchronous differential data

[0161] Synchronize the differential data file E to the target location F, and use the reverse operation of rsync to apply the differential data to the target file to ensure that the target data is consistent with the source data.

[0162] 4. Identify key nodes

[0163] 4.1. Using K-nearest neighbor algorithm

[0164] Collect historical synchronization task data and generate a key node list (G1), including the execution status of success and failure.

[0165] 4.2. Display transition interface

[0166] Pre-configure transition effects to enhance the user experience and ensure the user interface is smooth and natural when switching between key nodes.

[0167] 5. Monitor the synchronization process

[0168] 5.1. Configure monitoring tools

[0169] Select and configure monitoring tools to record the execution status of key nodes during full and incremental synchronization. Status information, including success, failure, and execution time, is recorded in a log file (I1) for subsequent analysis and optimization.

[0170] 5.2. Obtaining User Feedback

[0171] Through questionnaire surveys and user interviews, we collect user feedback on the transition interface and data synchronization process, and record the feedback results in the database (J1). We can then optimize and adjust the interface and process based on user feedback.

[0172] In summary, the present invention can optimize data transmission according to network conditions and significantly improve the synchronization speed of large files by dynamically adjusting the compression ratio. The dynamic adjustment method can flexibly cope with different network and system environments, which improves the versatility and adaptability of the method. The present invention uses incremental synchronization and differential algorithms to only transmit the changed parts, reducing bandwidth usage and storage requirements, and reducing the occupation of system resources. The present invention configures tools and environments suitable for high-concurrency processing, so that multiple users can synchronize data at the same time, improving collaboration efficiency. The present invention ensures that users' operations during the synchronization process are more natural and smooth through the identification of key nodes and the smooth transition interface display, reducing the troubles in use. At the same time, the proximity algorithm is used to analyze historical data, which can accurately identify the key nodes in the synchronization process, enhance the intelligence of key node identification during CG data synchronization, and make fast and accurate judgments when faced with complex real-time data. By identifying key nodes during the CG data synchronization process, the present invention can prioritize the allocation of computing and network resources during the data synchronization process, thereby improving synchronization efficiency.

[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatic synchronization of CG data assets, It is characterized in that The CG data asset automatic synchronization method includes: S1. Acquire and classify the CG data assets to be synchronized and complete the preliminary preparation for data synchronization; the preliminary preparation includes acquiring the synchronization requirements of CG data, planning the display process of CG data and configuring the synchronization architecture of CG data; S2. Based on the preliminary preparation results of data synchronization, the CG data assets to be synchronized are synchronized for the first time, and the compression ratio is dynamically adjusted during the full synchronization process; The dynamic adjustment of the compression ratio during the full synchronization process includes: S21. Collecting historical performance indicators of CG data synchronization, where the performance indicators include network bandwidth utilization, CPU utilization, system memory utilization, and data transmission rate; S22, determining a compression ratio range, and filtering out candidate compression ratios; S23, defining an objective function to measure the quality of the compression ratio; defining constraints; S24, using heuristic rules to explore different compression ratios and perform local search to improve the current solution; S25. Pre-calculate the best compression ratio under different conditions according to the results of the heuristic search and the local search, and construct a compression ratio table; S26, obtaining the performance index of CG data synchronization in real time, and calculating the value of the objective function; matching the corresponding optimal compression ratio; S3. Configure and execute incremental synchronization, and during the incremental synchronization process, synchronize the changed data in the CG data based on the differential algorithm; S4, identifies key nodes in the synchronization process based on the proximity algorithm, and displays the transition interface between key nodes in the pre-configured synchronization process; the synchronization process includes full synchronization and incremental synchronization; S5. Monitor the full synchronization and incremental synchronization process, and record the monitoring information of each full synchronization and incremental synchronization, and the monitoring information includes the execution status of key nodes and the display status of the transition interface; The method of identifying key nodes in the synchronization process based on a proximity algorithm and displaying a pre-configured transition interface between key nodes in the synchronization process includes: S41, collecting historical CG data synchronization task data, and using the K nearest neighbor algorithm to identify key nodes of the current CG data synchronization task to obtain a key node list; S42, according to the key node list, displaying the transition interface between the key nodes in the pre-configured synchronization process to the user; The historical CG data synchronization task data is collected, and the key nodes of the current CG data synchronization task are identified using the K nearest neighbor algorithm, and the key node list obtained includes: S411, obtaining a determination standard for a key node, wherein the determination standard for a key node includes that a CPU usage rate is continuously higher than a first threshold, a memory usage rate is continuously higher than a second threshold, a disk I / O rate is continuously higher than a third threshold, and a network bandwidth utilization rate is continuously higher than a fourth threshold, and the duration of the duration exceeds a preset duration; S412, according to the determination standard of the key nodes, collecting the historical CG data synchronization task data, and annotating the historical CG data synchronization task data to obtain an annotated data set; S413, selecting a distance measurement algorithm and the number of nearest neighbors, and determining the number of selected nearest neighbors through cross-validation; S414, constructing an index structure of the labeled data set, and storing the labeled data set; S415, collecting current CG data synchronization task data, using K nearest neighbor algorithm to analyze the current CG data synchronization task data, and identifying key nodes; S416, generating a key node list according to the prediction result of the K nearest neighbor algorithm, the key node list including the information and position of each key node; The collecting of current CG data synchronization task data, analyzing the current CG data synchronization task data using the K nearest neighbor algorithm, and identifying key nodes comprises the following steps: S4151. Using the selected distance measurement algorithm, calculate the distance between the nodes in the current CG data synchronization task data and the nodes in the labeled data set to obtain a distance list; S4152. According to the distance list, select several neighbors with the closest distance; assign a weight to each neighbor according to the distance, and perform weighted voting; S4153. Determine whether the current data point is a key node based on the voting result.

2. A method for automatic synchronization of CG data assets according to claim 1, It is characterized in that The preliminary preparations for completing data synchronization include: S11. Collect and analyze the synchronization requirements of CG data and determine the synchronization goals and requirements; S12. Design the display process of CG data, including user interface and data display method; S13. Select and configure synchronization tools and environments suitable for large file transfer and high-concurrency processing.

3. A method for automatic synchronization of CG data assets according to claim 1, It is characterized in that Determining the compression ratio range and filtering out candidate compression ratios includes: S221, determining a lower limit and an upper limit of a compression ratio according to an application scenario; S222. Within the determined compression ratio range, select a number of compression ratios at equal intervals as candidate compression ratios.

4. A method for automatic synchronization of CG data assets according to claim 3, It is characterized in that The configuration and execution of incremental synchronization, and in the incremental synchronization process, synchronizing the changed data in the CG data based on the differential algorithm includes: S31, calculate the file checksum of the source data and the target data, and compare the file checksum of the source data and the target data to obtain the changed data in the process of CG data incremental synchronization; S32, using a differential algorithm to generate a differential data file for the changed data in the CG data incremental synchronization process; S33, synchronizing the generated differential data file to the target location, and updating the target file at the target location using the differential data file.

5. A method for automatic synchronization of CG data assets according to claim 1, It is characterized in that When assigning weights to each neighbor based on the distance and performing weighted voting, the weight assignment formula is: ； In the formula, w i Indicates i The weight of the neighbors, d ( x , y i ) indicates the current node x With Neighbors y i The distance between α represents the parameter used to adjust the weight sensitivity; Take key nodes and non-key nodes as categories, and initialize a counter to accumulate weighted votes for each category; Traverse the neighbor nodes and calculate the weight of each neighbor node based on the distance; According to the category and weight of each neighbor node, the counter of the corresponding category is updated, and the category with the largest counter value is selected as the prediction result.

6. A method for automatic synchronization of CG data assets according to claim 1, It is characterized in that The step of displaying the transition interface between the key nodes in the pre-configured synchronization process to the user according to the key node list includes: S421, pre-configure the transition effect from any node to the next node to obtain a user interface; S422. Deploy the user interface to the production environment, monitor user feedback on the user interface, and optimize and adjust the user interface based on the feedback.

7. A method for automatic synchronization of CG data assets according to claim 1, It is characterized in that The monitoring of the full synchronization and incremental synchronization process and recording of the monitoring information of each full synchronization and incremental synchronization include: S51. Configure monitoring tools and record the execution status of key nodes during each full synchronization and incremental synchronization process, and the execution status includes success, failure and execution time; S52: Obtain the display effect of the transition interface and record user feedback.

Images