Cloud synchronization method, device and storage medium for model data

Abstract

The application relates to the field of electric digital data processing, and provides a cloud synchronization method and device of model data and a storage medium. The method comprises the following steps: in response to a change event of model data or a session context in a local computing environment, capturing corresponding incremental data packets or context data packets in real time through a pre-established bidirectional communication channel; identifying the captured data packets; if the captured data packets are context data packets, transmitting the context data packets to a cloud database asynchronously, and storing the context data packets by using a hybrid storage model; if the captured data packets are incremental data packets, transmitting the incremental data packets to an object storage service, and storing the incremental data packets by using a log-type storage model; when a model loading instruction is received, obtaining corresponding context data from the cloud database and / or obtaining an incremental data sequence from the object storage service, reconstructing data in the local computing environment, and delivering the data to a kernel layer of the local computing environment.

Description

Technical Field

[0001] This application relates to the field of electronic digital data processing, and in particular to a cloud synchronization method, apparatus and storage medium for model data. Background Technology

[0002] With the deep integration of cloud computing and collaborative design technologies, storing, synchronizing, and managing complex 3D model data in the cloud has become a key requirement in fields such as industrial design, Building Information Modeling (BIM), and digital twins. In these applications, model data includes not only incremental changes to the geometric entities themselves but also dynamic contextual information related to the design session, such as session configuration, permission partitions, and workflow markers. Achieving efficient, reliable, and fine-grained synchronization of model data between the local design environment and cloud storage is fundamental to supporting real-time collaboration, version rollback, and data persistence.

[0003] Currently, common cloud synchronization solutions mainly fall into two categories. One is database-based storage, which serializes model data and stores it in database records. While this approach facilitates metadata querying and management, it can lead to excessive database write pressure for frequent, large-volume incremental model operations and struggles to efficiently support multi-version management and rapid rollback of model data. The other is file-object-based storage, such as storing the entire model file or large data blocks in object storage. While suitable for storing large amounts of data, this approach typically requires uploading the entire modified file when handling fine-grained modifications to the model, failing to achieve true incremental storage and wasting network and storage resources. Furthermore, both of these solutions often mix or store model entity data with dynamic session context information, but lack a unified mechanism to capture these two different types of data in real time and select the optimal storage strategy based on their characteristics. This results in high synchronization latency, low storage efficiency, and performance bottlenecks due to full data transfer or lock contention when dealing with high-concurrency, fine-grained cloud collaborative editing scenarios.

[0004] In summary, the aforementioned existing technologies have at least the following drawbacks: First, they are difficult to achieve low-latency, fine-grained data synchronization, and cannot keep the amount of synchronized data to a minimum while ensuring data consistency; second, they lack a flexible storage structure for dynamically changing session context information, making it difficult to adapt to rapid changes in business needs; and finally, for incremental data of the model ontology, they lack an efficient storage structure to support high-frequency local modifications and version tracking, and typical solutions will incur significant I / O overhead due to the rewriting of large files. Summary of the Invention

[0005] This application provides a cloud synchronization method, apparatus, and storage medium for model data, which achieves efficient, low-latency cloud synchronization and flexible management of model data through real-time capture, off-stream storage, and on-demand reconstruction.

[0006] On the one hand, this application provides a cloud synchronization method for model data, the method comprising: In response to changes in model data or session context in the local computing environment, the corresponding incremental data packets or context data packets are captured in real time through a pre-established bidirectional communication channel. Identify the captured data packets; If the captured data packet is a context data packet, it is asynchronously transmitted to the cloud database and stored using a hybrid storage model, which stores core identification information and dynamic configuration information separately. If the captured data packet is an incremental data packet, it is transmitted to the object storage service and stored using a log-based storage model, which physically separates the data index information from the data content ontology and maintains them independently. When a model loading instruction is received, the corresponding context data is obtained from the cloud database and / or the incremental data sequence is obtained from the object storage service. The data is then reconstructed in the local computing environment and passed to the kernel layer of the local computing environment.

[0007] On the other hand, this application provides a cloud synchronization device for model data, the device comprising: The capture module is used to respond to changes in model data or session context in the local computing environment by capturing corresponding incremental data packets or context data packets in real time through a pre-established bidirectional communication channel. The identification module is used to identify the captured data packets. The first storage module is used to asynchronously transmit captured data packets to the cloud database if they are context data packets, and store them using a hybrid storage model, which separates the core identification information from the dynamic configuration information. The second storage module is used to transmit the captured data packets to the object storage service if they are incremental data packets, and store them using a log-based storage model. The log-based storage model physically separates the data index information from the data content itself and maintains them independently. The reconstruction module is used to obtain corresponding context data from the cloud database and / or incremental data sequence from the object storage service when a model loading instruction is received, reconstruct the data in the local computing environment, and pass it to the kernel layer of the local computing environment.

[0008] Thirdly, this application provides an apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the technical solution of the cloud synchronization method for model data described above.

[0009] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the cloud synchronization method for model data described above.

[0010] As can be seen from the technical solution provided in this application, on the one hand, by establishing an efficient and real-time data path from the local environment to the cloud, the technical solution of this application can respond to design operations in real time, encapsulate changes into independent data packets, and achieve millisecond-level response from event triggering to data capture, overcoming the high latency problem caused by traditional polling or timed batch synchronization, and providing underlying data flow guarantee for real-time collaboration; on the other hand, considering the characteristics of structured, frequently changing but small-sized context data, a hybrid storage model is adopted to separate core identifiers from dynamic configurations, so that the addition or removal of dynamic fields does not require modification of the database table structure, enhancing the scalability of the system. For flexibility, and considering the characteristics of incremental data—unstructured, highly sequential, and requiring strict version tracking—a log-based storage model that physically separates the index from the data ontology is adopted. This transforms any modification operation into updating a small index file and sequentially appending data to the data file, avoiding the huge I / O overhead of directly rewriting the entire large file, thereby improving the storage and modification efficiency of massive incremental model data. Thirdly, based on model loading instructions, the required data is retrieved in parallel from different but optimal storage media and reconstructed according to its inherent logic. This ensures the integrity of data loading while optimizing loading performance by leveraging the characteristics of different storage services. In summary, the technical solution of this application achieves efficient, low-latency cloud synchronization and flexible management of model data through real-time capture, off-stream storage, and on-demand reconstruction. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of the cloud synchronization method for model data provided in the embodiments of this application; Figure 2 This is a schematic diagram of the context data persistence architecture provided in the embodiments of this application; Figure 3 This is a schematic diagram illustrating the operation of incremental data in object storage provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the cloud synchronization device for model data provided in the embodiments of this application; Figure 5 This is a schematic diagram of the device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In this specification, adjectives such as "first" and "second" are used only to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0015] For ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale.

[0016] Currently, common cloud synchronization solutions mainly fall into two categories. One is database-based storage, which serializes model data and stores it in database records. While this approach facilitates metadata querying and management, it can lead to excessive database write pressure for frequent, large-volume incremental model operations and struggles to efficiently support multi-version management and rapid rollback of model data. The other is file-object-based storage, such as storing the entire model file or large data blocks in object storage. While suitable for storing large amounts of data, this approach typically requires uploading the entire modified file when handling fine-grained modifications to the model, failing to achieve true incremental storage and wasting network and storage resources. Furthermore, both of these solutions often mix or store model entity data with dynamic session context information, but lack a unified mechanism to capture these two different types of data in real time and select the optimal storage strategy based on their characteristics. This results in high synchronization latency, low storage efficiency, and performance bottlenecks due to full data transfer or lock contention when dealing with high-concurrency, fine-grained cloud collaborative editing scenarios. In summary, existing technologies have at least the following drawbacks: First, they are difficult to achieve low-latency, fine-grained data synchronization, and cannot keep the amount of synchronized data to a minimum while ensuring data consistency; second, they lack a flexible storage structure for dynamically changing session context information, making it difficult to adapt to rapid changes in business needs; and finally, for incremental data of the model ontology, they lack an efficient storage structure to support high-frequency local modifications and version tracking, and typical solutions will incur significant I / O overhead due to the rewriting of large files.

[0017] To address the aforementioned problems in the existing technology, this application proposes a cloud-based method for synchronizing model data, the flowchart of which is attached. Figure 1 As shown, the main steps include S101 to S105, which are detailed below: Step S101: In response to changes in model data or session context in the local computing environment, capture the corresponding incremental data packets or context data packets in real time through a pre-established bidirectional communication channel.

[0018] If the client periodically queries the server for data updates, network requests will be generated regardless of whether there are actual changes, resulting in significant waste of network bandwidth and computing resources. Data synchronization suffers from poor real-time performance, with latency depending on the polling interval. While timed batch synchronization (synchronizing all changes at fixed points in time or after a certain number of operations) is superior to the polling mechanism, this method still results in users not seeing real-time changes from collaborators, leading to a poor collaborative experience. Furthermore, with large data packets, network transmission and processing times are long, and there is a risk of synchronization failure and the need to retransmit large amounts of data, indicating low reliability. To achieve low-latency, fine-grained data synchronization and avoid data delays and resource waste caused by batch synchronization or polling mechanisms, this application employs a technical solution that responds to changes in model data or session context in the local computing environment by capturing corresponding incremental data packets or context data packets in real-time through a pre-established bidirectional communication channel. Here, the bidirectional communication channel can be a two-way communication channel between the kernel layer and the cloud service layer established at the kernel layer of the geometric modeling engine. The technical solution described above solves the latency and efficiency problems through event-driven and pre-established bidirectional channels. This means that once a change event occurs, the data is immediately encapsulated into small-granular data packets and pushed to the cloud service layer through a long connection, achieving millisecond-level synchronization latency. At the same time, the pre-established bidirectional communication channel also avoids the handshake overhead of each communication, ensuring high efficiency under high-frequency synchronization.

[0019] As one embodiment of this application, in response to changes in model data or session context in the local computing environment, real-time capture of corresponding incremental data packets or context data packets via a pre-established bidirectional communication channel can be achieved by: registering at least one callback interface in the kernel layer of the local computing environment; when a model entity change occurs in the kernel layer, calling the first callback interface to generate an incremental data packet containing the changed content; and when session configuration information changes in the kernel layer, calling the second callback interface to generate a context data packet containing the changed content. In the above embodiment, the local computing environment can be a geometric modeling engine, the first callback interface can be the DeltaCallback interface of the geometric modeling engine, and the second callback interface can be the SessionCtxNotifyCallback interface of the geometric modeling engine. Figure 2 The diagram shown is a schematic of the session context data persistence architecture provided in this application embodiment. The PS-KERNEL kernel layer and arrows in the diagram intuitively illustrate the process of registering a callback interface and calling a second callback interface to generate a context data packet.

[0020] Suppose a user performs a chamfering operation on a face of a part, extrudes a feature, or deletes a screw in design software. These operations directly change the geometry and structure of the 3D model, meaning that the object representing the model's geometry has changed within the geometry modeling engine kernel. The model entity change event then triggers the first callback interface (e.g., the DeltaCallback interface of the geometry modeling engine) to generate a data unit encapsulating the specific geometric change content—an incremental data packet. For example, the incremental data packet contains all the information needed for the operation "create a fillet with a radius of 5mm on edge B of entity A". Suppose a project manager in a collaborative design platform changes the status of a design task from "in progress" to "under review," adds a new "approval marker" to the model, or adjusts the model's display color scheme. These operations related to session configuration, partition lists, and marker lists do not change the model's geometry, but they alter its working environment—that is, changes to settings related to the current design session but independent of the model's geometry itself. This can trigger a second callback interface (e.g., the SessionCtxNotifyCallback interface of a geometry modeling engine) to generate a data unit encapsulating the context setting changes, i.e., a context data packet. For example, the context data packet might contain the message "Update the status field of session S to 'Completed'". These generated incremental data packets or context data packets can be encapsulated into messages of a predefined format and then pushed to the cloud service layer in real time through a pre-established bidirectional communication channel, such as a long connection channel based on the TCP protocol.

[0021] Step S102: Identify the captured data packets.

[0022] In this embodiment, two schemes can be used to identify the captured data packets. One is to start from the callback interface. As mentioned earlier, when a model entity changes in the kernel layer, the first callback interface is called to generate an incremental data packet containing the changed content. When session configuration information changes in the kernel layer, the second callback interface is called to generate a context data packet containing the changed content. Therefore, if the captured data packet is generated by calling the first callback interface, it is an incremental data packet; conversely, if the captured data packet is generated by calling the second callback interface, it is a context data packet. The other identification scheme is based on the type identifier inside the data packet. As a more robust supplementary mechanism, a clear type identifier (e.g., type: "delta" or type: "session_context") can be embedded in the message header or metadata when the data packet is generated. The cloud service layer can parse this identifier to identify whether it is an incremental data packet or a context data packet.

[0023] Step S103: If the captured data packet is a context data packet, it is asynchronously transmitted to the cloud database and stored using a hybrid storage model, wherein the hybrid storage model stores the core identification information and dynamic configuration information separately.

[0024] Step S104: If the captured data packet is an incremental data packet, it is transmitted to the object storage service and stored using the log storage model. The log storage model physically separates the data index information from the data content ontology and maintains them independently.

[0025] If all captured data packets (including large incremental data) are serialized and stored in a relational database without differentiation, the performance of writing and reading large binary incremental data will become a bottleneck. This is because relational databases excel at handling structured, small-volume data. Frequent model modifications will generate a large number of database records, leading to rapid database expansion, complex management, and high costs. Conversely, if all data (including structured context configuration) is stored as files in object storage, the lack of transaction support and efficient query capabilities of object storage will result in extremely inefficient operations. Each time a small portion of context information is read or updated (such as modifying a configuration parameter), the entire file may need to be downloaded, parsed, modified, and uploaded, which cannot meet the needs of real-time querying and updating. To address the characteristics of different data types and adopt optimal storage strategies to simultaneously achieve flexible schema management and high-performance versioned storage, in this embodiment, if the captured data packet is a context data packet, it is asynchronously transmitted to the cloud database and stored using a hybrid storage model, wherein the hybrid storage model stores core identification information and dynamic configuration information separately; if the captured data packet is an incremental data packet, it is transmitted to the object storage service and stored using a log-based storage model, wherein the log-based storage model physically separates and independently maintains the data's index information and data content ontology.

[0026] In the above embodiments, the reason for using a hybrid storage model to store context data packets is that the characteristics of context data are highly compatible with the advantages of relational databases, which are manifested in: 1) Structured data requiring complex queries: Context data (e.g., "Session configuration, Partition list, PMark list", etc.) is typical structured or semi-structured data, and business operations require frequent conditional queries (e.g., "find all sessions with a status of 'under review'"), and the powerful SQL query capabilities of relational databases perfectly meet this requirement; 2) Small data volume but frequent updates: The volume of a single record of context data is usually small, but it may change frequently. Databases are good at handling small-volume transactional updates (e.g., Update operations) and can guarantee ACID properties to ensure data consistency; 3) Schema may change dynamically: This is the core reason for adopting a hybrid storage model (core fixed columns + dynamic JSON columns). Session configuration items may be added or removed according to business needs. If a traditional database design is used, the table structure must be modified every time a field is added (ALTER). Adding fields to traditional database tables is cumbersome and can cause downtime. However, storing dynamic parts in JSON columns keeps the database table structure stable while allowing for flexible expansion of content, perfectly solving the industry problem of adding fields to traditional databases.

[0027] Similarly, the reason for using a log-based storage model to store incremental data packets is that the characteristics of incremental data packets are highly compatible with the advantages of object storage and log-based structures, namely: 1) Unstructured data, potentially large volume: Incremental data (e.g., changes to BodyEntity) is usually serialized binary large objects (BLOBs). Relational databases are not suitable for efficiently storing and reading / writing large BLOB data, which can lead to a sharp drop in performance. Object storage (e.g., OBS) is designed specifically for massive amounts of unstructured data, with near-infinite scalability and high throughput; 2) Write-intensive, primarily sequential append: In collaborative editing scenarios, model change operations are very frequent, generating a large amount of incremental data. The "append-only" characteristic of the log-based storage model transforms random write operations into sequential write operations, greatly reducing the impact of write operations. The model significantly improves write performance, meeting the needs of high-concurrency writes; 3) It requires efficient version management and backtracking: This is a key advantage of the log-based storage model. For example, each modification involves appending a new data block and updating the index, which means that all historical versions are naturally preserved in the data file. To view a historical version, you only need to find the data block address of the corresponding version through the index, achieving low-cost and efficient version backtracking; 4) It avoids rewriting large files: In traditional file storage, modifying a part of a file may require rewriting or locking the entire file, which is extremely inefficient. This model completely avoids reading and writing operations on the entire large file through the "index update + data append" method, solving the problem that "the body-level update does not achieve the design purpose of incremental storage and does not control the amount of updated data to the minimum".

[0028] As one embodiment of this application, asynchronously transmitting the context data packet to the cloud database can be as follows: after receiving the context data packet, the cloud service layer places it into a task queue and immediately returns an acknowledgment response; it starts an asynchronous thread independent of the request response, retrieves the data packet from the task queue, and performs a persistence operation with the cloud database. The persistence operation with the cloud database can be as follows: comparing the received context data with the current version stored in the cloud database to identify changes; based on the changes, generating a data manipulation language statement (e.g., an SQL statement) to update the changes; and executing the data manipulation language statement to complete incremental data persistence. Correspondingly, storing the context data packet using a hybrid storage model can be as follows: setting up a data table in the cloud database containing fixed columns and dynamic attribute columns; storing the static identification information from the context data packet in the fixed columns of the data table; and serializing the dynamic configuration information from the context data packet and storing it in the dynamic attribute columns. In the above embodiments, an index can be created for the dynamic attribute column. Specifically, the JSON string stored in the dynamic attribute column is parsed, and the frequently queried key-value pairs are extracted. An auxiliary index is created in the cloud database for the frequently queried key-value pairs to accelerate the query operation based on the dynamic configuration information.

[0029] It should be noted that in the above embodiments, static identification information is the core information that uniquely and stably identifies a session, such as the session ID, whose value is fixed once created. Static identification information is used for quick retrieval in the database and association with other data. In contrast to static identification information, dynamic configuration information describes the specific configuration, environment, and state of the session, and is flexibly variable. This includes session configuration (e.g., whether the display unit is millimeters or inches, default view mode), partition list (e.g., which users have editing permissions), and PMark list (e.g., review marks on the current model), etc., which typically change with user operations or business processes. Dynamic configuration information determines the specific functions, permissions, and current status of the session. Taking a collaborative design session named "Engine Design Project-V2" as an example, the static identification information (stored in a fixed column) is as follows: Session ID: SESS_2024_ENGINE_V2 (this is the unique identifier for this session and never changes); Creation time: 2024-05-27 10:00:00 (fixed after creation); Creator ID: USER_001 (fixed after creation). Dynamic configuration information (stored in a JSON column after serialization): its content will be a JSON object with a flexible structure, for example: { "displayUnit": "millimeter", "activeUsers": ["USER_001", "USER_005", "USER_012"], "partitions": { "editors": ["USER_001", "USER_005"], "viewers": ["USER_012", "USER_020"] }, "reviewMarks": [ {"id": "MARK_001", "type": "comment", "status": "open"},{"id": "MARK_002", "type": "approval", "status": "approved"}], "currentPhase": "detailed_design"} When a new user joins, the `activeUsers` array changes; when review markers are turned off, the status of the corresponding entry in `reviewMarks` changes. These changes only require updating a portion of the JSON columns, without altering the database table structure.

[0030] As one embodiment of this application, storing incremental data packets using a log-based storage model can be achieved by: creating an index file and a data file for each model instance in the object storage service; sequentially appending incremental data packets as data blocks to the end of the data file; and establishing and maintaining a mapping relationship between the logical address of the incremental data packet and its physical storage address in the data file in the index file. The incremental data packet corresponds to a add operation, a modify operation, or a delete operation. The processing includes: for a add operation, appending the data block to the data file and creating a new mapping record in the index file; for a modify operation, appending the modified data block as a new version to the data file and updating the mapping record of the corresponding logical address in the index file to point to the new version; for a delete operation, invalidating the mapping record of the corresponding logical address in the index file. In the above embodiment, during a modify operation, the data block in the data file before the modification is retained to achieve data version rollback. Figure 3 The diagram shown is a schematic diagram of the operation of incremental data in object storage provided in the embodiments of this application. It clearly demonstrates the physical separation of the index file and a data file, the sequential appending to the data file, and the establishment and maintenance of mapping relationships in the index file, etc.

[0031] The core advantages of a log-based storage model are avoiding the rewriting of large files and supporting version rollback. However, this advantage comes at the cost of leaving old data blocks (which can be reclaimed) in the data files after each modification or deletion operation. As the system runs, a large amount of invalid historical data accumulates in the data files, leading to problems such as wasted storage resources, degraded read performance, and increased management complexity. To resolve the contradiction between the unlimited expansion of storage space and performance degradation, Figure 1 The example method could also include compressing the storage structure, namely: periodically scanning the index file to identify all data blocks still referenced by valid mapping records as valid data blocks, and identifying data blocks not referenced by any valid mapping records as reclaimable data blocks; reading the valid data blocks from the old data file and integrating them into a new data file; generating a new index file based on the physical addresses of the valid data blocks in the new data file; and deleting the old data file containing reclaimable data blocks and the old index file. This approach, on the one hand, proactively releases storage space occupied by invalid data through a periodic garbage collection mechanism, controlling storage growth within a reasonable range related only to the current amount of valid data, significantly reducing long-term storage costs; on the other hand, after the storage structure is compressed, the data file size becomes smaller and the structure more compact, which not only reduces the metadata pressure on the file system but also makes subsequent read / write operations (especially sequential reads) and full backups more efficient.

[0032] Furthermore, in order to achieve accurate version snapshots and pinpoint backtracking at low cost and high efficiency, Figure 1 The example method may also include creating a version snapshot, that is: freezing the current index file in response to a user instruction or at a predetermined time point; recording the state of the frozen index file and its corresponding data file to form a version snapshot for subsequent fixed-point data backtracking. It should be noted that traditional snapshots usually require copying a large amount of data, while the above embodiment of this application utilizes the feature of "index file as metadata snapshot". The snapshot operation only needs to "freeze" and record the state of a small index file and the data file it points to, without copying the entire data file. This makes the snapshot creation speed extremely fast, occupies almost no additional storage space (until new data is written), and has extremely low cost. In addition, to restore to this snapshot version, it is only necessary to replace the current index file with the index file saved by the snapshot. This makes the data backtracking operation very simple, fast and reliable, avoiding the complexity of finding and calculating the state at a specific point in time from massive operation logs.

[0033] Step S105: When a model loading instruction is received, obtain the corresponding context data from the cloud database and / or obtain the incremental data sequence from the object storage service, reconstruct the data in the local computing environment, and pass it to the kernel layer of the local computing environment.

[0034] If a complete copy of the entire model is periodically saved, and the latest snapshot is downloaded directly during loading, storage space is wasted significantly because most data is duplicated between snapshots; it also cannot support fine-grained version rollback to any historical operation point. Even if all incremental operations are stored, all operations are executed sequentially from the beginning until the latest state during loading. If the model history is long, the number of incremental operations that still need to be applied during loading is enormous, resulting in extremely slow loading times. Therefore, in order to quickly and accurately restore the complete model state when needed by the client, ensuring data consistency and integrity, the technical solution adopted in this application is to obtain the corresponding context data from the cloud database and / or the incremental data sequence from the object storage service when a model loading instruction is received, reconstruct the data in the local computing environment, and then pass it to the kernel layer of the local computing environment. Here, the local computing environment can be a geometric modeling engine, and its kernel layer can act as a client in a C / S architecture. The above embodiment is based on the "latest version mapping relationship" recorded in the index file in the log-style storage model. It quickly locates and loads only the data blocks required to constitute the current model state. This makes model reconstruction not require traversing the entire operation history, achieving efficient on-demand loading. At the same time, parallel acquisition of data from two sources (database and object storage) also optimizes the loading speed.

[0035] Corresponding to the aforementioned embodiment of storing incremental data packets using a log-based storage model, as an embodiment of this application, obtaining the incremental data sequence from the object storage service may involve: parsing the index file to obtain all valid logical addresses and their corresponding latest data block physical addresses; reading the corresponding data blocks from the data file based on the physical addresses; and deserializing the read data blocks according to the logical order of model changes to reconstruct the complete model incremental state. Specifically, when reconstructing the model incremental state, if the incremental data is out of order due to the distributed environment, the incremental data is sorted according to the logical timestamp or sequence number carried in the data blocks to ensure data consistency.

[0036] Figure 1 The example method may also include consistency verification after data reconstruction, namely: calculating the first verification value of the reconstructed model state; obtaining the baseline verification value corresponding to the model state stored in the cloud; comparing the first verification value with the baseline verification value, and triggering an error handling process if they are inconsistent.

[0037] On the one hand, network transmission itself does not guarantee the arrival order of data packets. This means that during concurrent requests or changes in network routing, later-sent incremental data packets may arrive at the cloud before earlier-sent packets. If the cloud directly applies these incremental packets in the order they are received, it will lead to errors in the model state. On the other hand, network connections may be unexpectedly interrupted (e.g., WiFi switching, client crashes). After the connection is restored, determining which data was lost during the disconnection and how to efficiently and accurately resynchronize the connection becomes problematic. Simply requiring the retransmission of all data or data from the most recent period would result in a large amount of redundant transmission, inefficiency, and the potential introduction of duplicate data. To address these issues, Figure 1 The example method can also include incremental marking and order guarantee, that is: before sending each incremental data packet through the bidirectional communication channel, a globally monotonically increasing sequence number is assigned to each incremental data packet; when resynchronization is required after communication interruption is restored, the cloud service layer requests all incremental data packets after the specified sequence number from the local computing environment to ensure that the application order of incremental data packets is consistent with the generation order. In the above embodiment, since each incremental data packet is assigned a globally unique, monotonically increasing sequence number when it is generated, this sequence number becomes the logical timestamp of the data packet. Therefore, regardless of the physical order in which the data packets arrive at the cloud, the cloud can strictly sort them according to the sequence number and strictly apply changes in this order, thereby ensuring that the model state reconstructed by all terminals in the distributed environment is absolutely consistent. On the other hand, after the connection is restored, the cloud service layer can check the sequence number (e.g., N) of the last data packet that has been successfully applied, and then accurately request "all data packets with a sequence number greater than N" from the client. This avoids duplicate requests and transmission of successfully processed data, resulting in extremely high transmission efficiency, and can accurately obtain all increments generated during the interruption, ensuring the final integrity of the data.

[0038] From the above appendix Figure 1As can be seen from the cloud synchronization method of the example model data, on the one hand, by establishing an efficient and real-time data path from the local environment to the cloud, the technical solution of this application can respond to design operations instantly, encapsulate changes into independent data packets, and achieve millisecond-level response from event triggering to data capture. This overcomes the high latency problem caused by traditional polling or timed batch synchronization, and provides underlying data flow guarantee for real-time collaboration. On the other hand, considering the characteristics of structured, frequently changing but small-sized context data, a hybrid storage model is adopted to separate the core identifier from the dynamic configuration. This allows the addition or removal of dynamic fields to be done without modifying the database table structure, enhancing the scalability of the system. In terms of flexibility, and considering the characteristics of incremental data—unstructured, highly sequential, and requiring strict version tracking—a log-based storage model with physical separation of index and data ontology is adopted. This transforms any modification operation into updating a small index file and sequentially appending data to the data file, avoiding the huge I / O overhead of directly rewriting the entire large file, thereby improving the storage and modification efficiency of massive incremental model data. Thirdly, based on model loading instructions, the required data is retrieved in parallel from different but optimal storage media and reconstructed according to its inherent logic. This ensures the integrity of data loading while optimizing loading performance by leveraging the characteristics of different storage services. In summary, the technical solution of this application achieves efficient, low-latency cloud synchronization and flexible management of model data through real-time capture, distributed storage, and on-demand reconstruction.

[0039] Please see the appendix Figure 4 This application provides a cloud-based model data synchronization device, which may include a capture module 401, an identification module 402, a first storage module 403, a second storage module 404, and a reconstruction module 405, as detailed below: The capture module 401 is used to capture the corresponding incremental data packets or context data packets in real time through a pre-established bidirectional communication channel in response to changes in model data or session context in the local computing environment. The identification module 402 is used to identify the captured data packets; The first storage module 403 is used to asynchronously transmit the captured data packet to the cloud database if it is a context data packet, and store the context data packet using a hybrid storage model, wherein the hybrid storage model stores the core identification information and dynamic configuration information separately. The second storage module 404 is used to transmit the captured data packet to the object storage service if it is an incremental data packet, and to store the incremental data packet using a log-based storage model. The log-based storage model physically separates the data index information from the data content ontology and maintains them independently. The reconstruction module 405 is used to obtain the corresponding context data from the cloud database and / or the incremental data sequence from the object storage service when a model loading instruction is received, to reconstruct the data in the local computing environment and pass it to the kernel layer of the local computing environment.

[0040] From the above appendix Figure 4 As can be seen from the example of the cloud synchronization device for model data, on the one hand, by establishing an efficient and real-time data path from the local environment to the cloud, the technical solution of this application can respond to design operations instantly, encapsulate changes into independent data packets, and achieve millisecond-level response from event triggering to data capture. This overcomes the high latency problem caused by traditional polling or timed batch synchronization, providing underlying data flow assurance for real-time collaboration. On the other hand, considering the characteristics of structured, frequently changing, but small-sized context data, a hybrid storage model is adopted to separate the core identifier from dynamic configuration. This allows the addition or removal of dynamic fields to proceed without modifying the database table structure, enhancing the scalability of the system. In terms of flexibility, and considering the characteristics of incremental data—unstructured, highly sequential, and requiring strict version tracking—a log-based storage model with physical separation of index and data ontology is adopted. This transforms any modification operation into updating a small index file and sequentially appending data to the data file, avoiding the huge I / O overhead of directly rewriting the entire large file, thereby improving the storage and modification efficiency of massive incremental model data. Thirdly, based on model loading instructions, the required data is retrieved in parallel from different but optimal storage media and reconstructed according to its inherent logic. This ensures the integrity of data loading while optimizing loading performance by leveraging the characteristics of different storage services. In summary, the technical solution of this application achieves efficient, low-latency cloud synchronization and flexible management of model data through real-time capture, distributed storage, and on-demand reconstruction.

[0041] Figure 5 This is a schematic diagram of the structure of a device provided in one embodiment of this application. For example... Figure 5 As shown, the device 5 in this embodiment mainly includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as a program for a cloud synchronization method of model data. When the processor 50 executes the computer program 52, it implements the steps in the above-described embodiment of the cloud synchronization method for model data, for example... Figure 1 The steps S101 to S105 are shown. Alternatively, when the processor 50 executes the computer program 52, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 4 The functions of the capture module 401, the identification module 402, the first storage module 403, the second storage module 404, and the reconstruction module 405 are shown.

[0042] For example, the computer program 52 of the model data cloud synchronization method mainly includes: responding to changes in model data or session context in the local computing environment, capturing corresponding incremental data packets or context data packets in real time through a pre-established bidirectional communication channel; identifying the captured data packets; if the captured data packet is a context data packet, asynchronously transmitting it to the cloud database and storing the context data packet using a hybrid storage model, wherein the hybrid storage model stores core identification information and dynamic configuration information separately; if the captured data packet is an incremental data packet, transmitting it to the object storage service and storing the incremental data packet using a log-based storage model, wherein the log-based storage model physically separates and independently maintains the data index information and data content ontology; when a model loading instruction is received, obtaining the corresponding context data from the cloud database and / or obtaining the incremental data sequence from the object storage service, reconstructing the data in the local computing environment, and passing it to the kernel layer of the local computing environment. The computer program 52 can be divided into one or more modules / units, which are stored in memory 51 and executed by processor 50 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 52 in device 5. For example, computer program 52 can be divided into the functions of capture module 401, identification module 402, first storage module 403, second storage module 404, and reconstruction module 405 (a module in the virtual device). The specific functions of each module are as follows: capture module 401 is used to capture corresponding incremental data packets or context data packets in real time through a pre-established bidirectional communication channel in response to changes in model data or session context in the local computing environment; identification module 402 is used to identify the captured data packets; first storage module 403 is used to asynchronously transmit the captured data packets to the cloud database and utilize hybrid storage if the captured data packets are context data packets. The model stores context data packets, where the hybrid storage model stores core identification information and dynamic configuration information separately; the second storage module 404 is used to transmit incremental data packets to the object storage service if the captured data packets are incremental data packets, and to store the incremental data packets using a log-style storage model, where the log-style storage model physically separates the data index information from the data content ontology and maintains them independently; the reconstruction module 405 is used to obtain the corresponding context data from the cloud database and / or the incremental data sequence from the object storage service when a model loading instruction is received, to reconstruct the data in the local computing environment and pass it to the kernel layer of the local computing environment.

[0043] Device 5 may include, but is not limited to, processor 50 and memory 51. Those skilled in the art will understand that... Figure 5This is merely an example of device 5 and does not constitute a limitation on device 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, the device may also include input / output devices, network access devices, buses, etc.

[0044] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0045] The memory 51 can be an internal storage unit of the device 5, such as a hard disk or RAM of the device 5. The memory 51 can also be an external storage device of the device 5, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the device 5. Furthermore, the memory 51 can include both internal and external storage units of the device 5. The memory 51 is used to store computer programs and other programs and data required by the device. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0046] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed. That is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0047] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0048] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0049] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0050] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0051] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0052] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the implementation of all or part of the processes in the above-described embodiments can also be accomplished by a computer program instructing related hardware. The computer program for the cloud synchronization method of model data can be stored in a storage medium. When the computer program is executed by a processor, it can implement the steps of the above-described method embodiments, namely, responding to changes in model data or session context in the local computing environment, capturing corresponding incremental data packets or context data packets in real time through a pre-established bidirectional communication channel; identifying the captured data packets; if the captured data packet is a context data packet, asynchronously transmitting it to the cloud database and storing the context data packet using a hybrid storage model, wherein the hybrid storage model stores core identification information and dynamic configuration information separately; if the captured data packet is an incremental data packet, transmitting it to the object storage service and storing the incremental data packet using a log-based storage model, wherein the log-based storage model physically separates and independently maintains the data index information and data content ontology; when a model loading instruction is received, obtaining the corresponding context data from the cloud database and / or obtaining the incremental data sequence from the object storage service, reconstructing the data in the local computing environment, and transmitting it to the kernel layer of the local computing environment. Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of storage media can be appropriately added or removed according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, storage media do not include electrical carrier signals and telecommunication signals.

[0053] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the protection scope of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for cloud synchronization of model data, characterized in that, The method includes: In response to changes in model data or session context in the local computing environment, the corresponding incremental data packets or context data packets are captured in real time through a pre-established bidirectional communication channel. Identify the captured data packets; If the captured data packet is a context data packet, it is asynchronously transmitted to the cloud database and stored using a hybrid storage model, which stores the core identification information and dynamic configuration information separately. If the captured data packet is an incremental data packet, it is transmitted to the object storage service and stored using a log-based storage model. The log-based storage model physically separates the data index information from the data content ontology and maintains them independently. When a model loading instruction is received, the corresponding context data is obtained from the cloud database and / or the incremental data sequence is obtained from the object storage service. The data is then reconstructed in the local computing environment and passed to the kernel layer of the local computing environment.

2. The cloud synchronization method for model data according to claim 1, characterized in that, The storage of the context data packet using a hybrid storage model includes: Set up a data table in the cloud database that includes fixed columns and dynamic attribute columns; The static identification information in the context data packet is stored in a fixed column of the data table; The dynamic configuration information in the context data packet is serialized and stored in the dynamic attribute column.

3. The cloud synchronization method for model data according to claim 1, characterized in that, The storage of the incremental data packets using a log-based storage model includes: In the object storage service, an index file and a data file are created for each model instance; The incremental data packets are appended sequentially to the end of the data file as data blocks. In the index file, a mapping relationship is established and maintained between the logical address of the incremental data packet and its physical storage address in the data file.

4. The cloud synchronization method for model data according to claim 3, characterized in that, The incremental data packet corresponds to a new operation, a modification operation, or a deletion operation, and its processing includes: For the newly added operation, the data block is appended to the data file, and a new mapping record is created in the index file; For the modification operation, the modified data block is appended to the data file as a new version, and the mapping record of the corresponding logical address in the index file is updated to point to the new version; For the deletion operation, the mapping record of the corresponding logical address is invalidated in the index file.

5. The cloud synchronization method for model data according to claim 3, characterized in that, The method further includes: The index file is periodically scanned to identify all data blocks that are still referenced by valid mapping records as valid data blocks, and to identify data blocks that are not referenced by any valid mapping records as reclaimable data blocks; The valid data blocks are read from the old data file and integrated and written into the new data file; A new index file is generated based on the physical addresses of the valid data blocks in the new data file; Delete the old data files and old index files that contain the reclaimable data blocks.

6. The cloud synchronization method for model data according to claim 1, characterized in that, The method further includes: Before sending each incremental data packet through the bidirectional communication channel, a globally monotonically increasing sequence number is assigned to it; When resynchronization is required after a communication interruption is restored, the cloud service layer requests all incremental data packets after the specified sequence number from the local computing environment to ensure that the application order of the incremental data packets is consistent with the generation order.

7. The cloud synchronization method for model data according to claim 3, characterized in that, The method further includes: In response to user commands or at a predetermined time point, freeze the current index file; Record the status of frozen index files and their corresponding data files to form a version snapshot for subsequent fixed-point data backtracking.

8. A cloud synchronization device for model data, characterized in that, The device includes: The capture module is used to respond to changes in model data or session context in the local computing environment by capturing corresponding incremental data packets or context data packets in real time through a pre-established bidirectional communication channel. The identification module is used to identify the captured data packets; The first storage module is used to asynchronously transmit the captured data packet to the cloud database if it is a context data packet, and store the context data packet using a hybrid storage model, wherein the hybrid storage model stores the core identification information and dynamic configuration information separately. The second storage module is used to transmit the captured data packet to the object storage service if it is an incremental data packet, and to store the incremental data packet using a log-based storage model. The log-based storage model physically separates the data index information from the data content itself and maintains them independently. The reconstruction module is used to obtain corresponding context data from the cloud database and / or incremental data sequence from the object storage service when a model loading instruction is received, reconstruct the data in the local computing environment, and pass it to the kernel layer of the local computing environment.

9. An apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images