A data incremental synchronization method, device, storage medium and electronic equipment
By performing differentiated analysis and object-level processing on enterprise data assets, the problem of low synchronization efficiency of knowledge graphs in existing technologies has been solved, achieving efficient incremental data synchronization and automatic fault repair.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING SHUYU TECH CO LTD
- Filing Date
- 2026-02-05
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies in enterprise data asset management, especially in the incremental data synchronization process of knowledge graphs, suffer from time-consuming full-scale reconstruction synchronization methods, resulting in low efficiency, inability to accurately identify and process newly added, updated, and deleted objects, and lack of automatic repair mechanisms in the event of failure.
By performing differential analysis on the business resource information to be processed and the source data knowledge graph, the data type of each type of resource object is determined, the corresponding data set is constructed, and the source data knowledge graph is updated through these sets, thus achieving fine-grained processing and automatic repair at the object level.
It improves the efficiency of incremental data synchronization, reduces time costs, enables accurate updates of the knowledge graph and automatic fault repair, and avoids manual intervention.
Smart Images
Figure CN122086899A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, storage medium, and electronic device for incremental data synchronization. Background Technology
[0002] As companies grow in size, the scale of their data assets also increases. Enterprise data assets refer to data resources that an enterprise legally holds or controls, that are measurable, and that can bring future economic benefits to the enterprise during its production, operation, management, research and development activities.
[0003] Currently, knowledge graphs are commonly used to manage enterprise data assets. When these assets change, a full synchronization or incremental synchronization based on timestamps or version numbers is typically employed, followed by simple task scheduling and manual handling of synchronization failures to update the knowledge graph. However, this method of full-scale reconstruction and synchronization of the knowledge graph undoubtedly consumes a significant amount of time and reduces the efficiency of incremental data synchronization.
[0004] Therefore, how to provide a technical solution for an efficient method of incremental data synchronization has become an urgent technical problem to be solved. Summary of the Invention
[0005] The purpose of some embodiments of this application is to provide a method, apparatus, storage medium, and electronic device for incremental data synchronization. The technical solutions of the embodiments of this application can improve the efficiency of incremental data synchronization and reduce time costs.
[0006] In a first aspect, some embodiments of this application provide a method for incremental data synchronization, comprising: performing differential analysis on business resource information to be processed and source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed; wherein the data type includes newly added data, updated data, or deleted data; constructing a data set matching the data type of each type of resource object; wherein the data set includes a newly added set, an updated set, or a deleted set; and updating the source data knowledge graph through the data set to obtain an updated knowledge graph.
[0007] Some embodiments of this application obtain the data type of each type of resource object through differential analysis of the business resource information to be processed and the source data knowledge graph, and then construct the corresponding data set; finally, the source data knowledge graph is batch modified according to the data set to obtain the updated knowledge graph. By classifying the resource objects and data types of the business resource information to be processed, this application allows for incremental data synchronization only of the relevant resource objects, achieving a shift from full reconstruction to fine-grained changes, reducing time costs, and improving the efficiency of incremental data synchronization.
[0008] In some embodiments, before performing differential analysis between the business resource information to be processed and the source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed, the method further includes: reading the business scenario, data attributes, and data identifier of the business resource information to be processed; and obtaining a set of associated objects from the source data knowledge graph based on the data attributes.
[0009] Some embodiments of this application can obtain a set of related objects from the source data knowledge graph by reading relevant content from the business resource information to be processed, thus providing a basis for subsequent targeted incremental synchronization of data.
[0010] In some embodiments, the step of performing differential analysis between the business resource information to be processed and the source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed includes: preprocessing the data in the associated object set to obtain processed data; and comparing the processed data with the resource data of each type of resource object to obtain the data type.
[0011] Some embodiments of this application compare the preprocessed data from the data in the associated object set with the resource data to obtain the data type, thereby achieving accurate modification of the source data knowledge graph.
[0012] In some embodiments, before updating the source data knowledge graph through the data set, the method further includes: calling the health check interface of the source data knowledge graph; and confirming that the health check interface returns a healthy value.
[0013] Some embodiments of this application perform health checks by calling a health check interface to ensure that the change service can proceed normally and avoid failure.
[0014] In some embodiments, updating the source data knowledge graph using the data set to obtain an updated knowledge graph includes: when the data set is the update set, querying the old graph node corresponding to the resource object corresponding to the update set from the source data knowledge graph; deleting the old graph node; and writing the data in the update set to the source data knowledge graph to obtain the updated knowledge graph.
[0015] Some embodiments of this application find and delete the old knowledge graph nodes of the resource objects corresponding to the update set, and then write the data in the update set into the source data knowledge graph to obtain the updated knowledge graph. Embodiments of this application can achieve effective data updating.
[0016] In some embodiments, updating the source data knowledge graph using the data set to obtain an updated knowledge graph includes: when the data set is the deletion set, querying the process data of the resource objects to be deleted in the deletion set from the source data knowledge graph; deleting the process data from the source data knowledge graph to obtain the updated knowledge graph.
[0017] Some embodiments of this application achieve effective data deletion by finding and deleting the data corresponding to the resource objects to be deleted in the deletion set, and then obtaining an updated knowledge graph.
[0018] In some embodiments, updating the source data knowledge graph using the data set to obtain an updated knowledge graph includes: when the data set is the newly added set, constructing a data write request; adding data from the newly added set to the source data knowledge graph based on the data write request to obtain the updated knowledge graph.
[0019] Some embodiments of this application add data from the newly added set to the source data knowledge graph after constructing a data write request, thereby obtaining an updated knowledge graph and achieving incremental synchronization of new data.
[0020] Secondly, some embodiments of this application provide a data incremental synchronization apparatus, comprising: a difference analysis module, used to perform difference analysis on the business resource information to be processed and the source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed; wherein the data type includes newly added data, updated data, or deleted data; a construction module, used to construct a data set matching the data type of each type of resource object; wherein the data set includes a newly added set, an updated set, or a deleted set; and an incremental synchronization module, used to update the source data knowledge graph through the data set to obtain an updated knowledge graph.
[0021] Thirdly, some embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.
[0022] Fourthly, some embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can implement the method as described in any embodiment of the first aspect.
[0023] Fifthly, some embodiments of this application provide a computer program product, the computer program product including a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. Attached Figure Description
[0024] To more clearly illustrate the technical solutions of some embodiments of this application, the accompanying drawings used in some embodiments of this application will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 System diagrams for incremental data synchronization provided for some embodiments of this application; Figure 2 One of the flowcharts for a method of incremental data synchronization provided for some embodiments of this application; Figure 3 A second flowchart illustrating a method for incremental data synchronization provided for some embodiments of this application; Figure 4 Block diagrams of apparatus for incremental data synchronization provided for some embodiments of this application; Figure 5 A schematic diagram of an electronic device provided for some embodiments of this application. Detailed Implementation
[0026] The technical solutions of some embodiments of this application will now be described with reference to the accompanying drawings.
[0027] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0028] In related technologies, the common practice for synchronizing enterprise data assets (such as datasets, knowledge bases, and document libraries) to knowledge graph / GraphRAG systems is "full synchronization or incremental synchronization based on timestamps / version numbers + simple task scheduling + manual handling of failures". Typical technical solutions include: Solution A: Full reconstruction synchronization: Periodically read all data from the source system and push it uniformly to the graph service to overwrite / reconstruct the graph data; Solution B: Timestamp / version number incremental: Compare the update time or version number of the source data to decide whether to push; Solution C: General state machine / error recovery: Record task states through a finite state machine, but these are mostly task-level (resource-level) states, making object-level tracking and replay difficult; Solution D: General synchronization location management: Determine whether to sleep or continue synchronization by recording the synchronization location / offset, but does not recognize object-level differences such as "added / updated / deleted". That is, the business source system stores a large number of data objects (such as dataset records, knowledge base paragraphs, etc.), and synchronization tasks are triggered at fixed intervals. Synchronization tasks either read all objects directly and push them to the graph service, or filter out objects that "may have changed" based on update time / version number. Synchronization tasks typically only record resource-level / task-level success and failure, lacking status records for each individual object, making it impossible to accurately pinpoint "which object failed, the reason for failure, or whether it has been written to the graph." Furthermore, when an object is deleted from the source system, locating and deleting the corresponding node / relationship on the graph side often relies on manual scripts or a full rebuild. In cases of network failures, service unavailability, or abnormal data formats, manual intervention is often required, or simple retries may fail, ultimately necessitating a full rerun.
[0029] As can be seen from the aforementioned technologies, when the data scale reaches hundreds of thousands or millions of records, full retrieval and full writing result in huge overhead for network transmission and graph writing, long synchronization time, and may affect the availability of the graph service. The root cause lies in the lack of a precise difference identification mechanism, which cannot handle only sets of "added / updated / deleted" objects. Existing tasks often only record task-level success / failure, failing to locate "which object failed and what the reason for the failure," leading to fault recovery requiring rerunning the entire batch or even the entire dataset. Timestamp / version number-based incremental updates can often only identify "updates," making it difficult to identify "source deletions." Even if deletion is identified, there is a lack of graph-side positioning markers, making precise deletion impossible, resulting in residual old nodes or relationships in the graph, affecting retrieval and inference results. Furthermore, the current technology involves various anomalies in the synchronization link, including network, data format, and graph service issues. The current technology lacks an automatic repair loop for error type classification, retry limits, and exponential backoff, resulting in a high MTTR (Mean Time To Repair / Recover / Resolve).
[0030] In view of this, some embodiments of this application provide a method for incremental data synchronization. This method determines the data type of each type of resource object involved in the business resource information to be processed by performing differential analysis on the business resource information to be processed and the source data knowledge graph. Then, a corresponding data set is constructed using this data type. Finally, the source data knowledge graph is updated using this data set to obtain the updated knowledge graph. Embodiments of this application can achieve fine-grained progress tracking and updates of the knowledge graph at the object level. By accurately distinguishing data types, batch processing of the source data knowledge graph is achieved, improving processing efficiency. Throughout the process, this application can achieve automatic closed-loop repair without manual intervention.
[0031] The following is in conjunction with the appendix Figure 1 The overall structure of a data incremental synchronization system provided by some embodiments of this application is illustrated by way of example.
[0032] like Figure 1 As shown, some embodiments of this application provide a system diagram for incremental data synchronization. This incremental data synchronization system may include: a business source system 100, a resource association layer 120, an object progress layer 130, a scheduled task 140, and a graph service 150. The business source system 100 includes resource information such as datasets, knowledge bases, or document libraries related to the business system. The resource association layer 120 stores resource binding relationships and resource-level statistical information; it expands resource information into object-level records and writes them to the object progress layer 130. The object progress layer 130 stores the object-level status, content fingerprint, graph mapping, and error information of the resource objects. The scheduled task 140 can periodically read the pending records of pending business resource information in the object progress layer 130 and process the pending records by calling the graph service 150 to update the source data knowledge graph.
[0033] The scheduled task 140 includes full update processing, full deletion processing, batch cyclic processing of new additions, and error scanning and automatic repair. The error scanning and automatic repair in the scheduled task 140 can perform categorized repair and retry control on resource objects corresponding to error information in the object progress layer 130. The graph service 150 includes functions such as data writing, data deletion, and health checks.
[0034] In some embodiments of this application, the data incremental synchronization system can be deployed on a terminal device, which can be a mobile terminal or a non-portable computer terminal. This application does not specifically limit the implementation of the system.
[0035] The following is in conjunction with the appendix Figure 2 The present application provides an exemplary embodiment of the implementation process of incremental data synchronization performed by a system for incremental data synchronization.
[0036] Please see the appendix Figure 2 , Figure 2 A flowchart of a data incremental synchronization method is provided for some embodiments of this application. The data incremental synchronization method may include: S210, perform differential analysis between the business resource information to be processed and the source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed; wherein, the data type includes newly added data, updated data, or deleted data.
[0037] For example, in a specific embodiment of this application, by performing differential analysis on the data progress in the business resource information to be processed and the source data knowledge graph, it can be determined whether the data of each type of resource object is newly added, updated, or needs to be deleted. In this way, targeted preliminary data statistics can be achieved, providing a foundation for subsequent incremental data synchronization.
[0038] In some embodiments of this application, before executing S210, the method for incremental data synchronization may include: reading the business scenario, data attributes, and data identifier of the business resource information to be processed; and obtaining a set of associated objects from the source data knowledge graph based on the data attributes.
[0039] For example, in a specific embodiment of this application, the resource association layer 120 reads the resource binding information of the business resource information to be processed, such as graphRagId (i.e., business scenario), feature (i.e., data attribute), objectId (i.e., data identifier), etc. Then, it obtains an object set from the source system (as a specific example of the source data knowledge graph) through the feature type and constructs a mapping. Next, it loads the progress records of all related data under this object set from the object progress layer 130 into the memory mapping to obtain the associated object set. This associated object set stores the source data related to the resource objects in the business resource information to be processed, and subsequent operations can be used to add, update, or delete this source data.
[0040] In some embodiments of this application, S210 may include: preprocessing the data in the associated object set to obtain processed data; comparing the processed data with the resource data of each type of resource object to obtain the data type.
[0041] For example, in a specific embodiment of this application, the source data undergoes normalization processing (i.e., preprocessing), such as removing whitespace, sorting keys, uniform encoding, and extracting only the semantic fields data / content. After preprocessing, SHA-256 is calculated on the processed data to obtain the current data hash value; the current data hash value is compared with the stored dataHash to determine the data type of each type of resource object.
[0042] S220, construct a data set that matches the data type of each type of resource object; wherein the data set includes an addition set, an update set, or a deletion set.
[0043] For example, in a specific embodiment of this application, the data types obtained above are used to generate a new set, an update set, and a deletion set. The new set represents new data to be written into the source data knowledge graph, the update set represents data content that needs to be updated for related nodes in the source data knowledge graph, and the deletion set represents data that needs to be deleted from the source data knowledge graph. Simultaneously, the object progress layer 130 performs batch record updates for the new, updated, and deleted data, setting corresponding status and actions. Here, status represents the processing result (new / done / error), and action represents the type of operation to be performed (new / update / delete, i.e., new, update, and delete), and the two are decoupled. Update does not directly write overwrite; instead, deleteData is executed first to delete the old graph node. Only if the deletion is successful is it allowed to switch to action=new and enter pushData for writing. If the deletion fails, action=update is maintained and status=error is set for retrying, avoiding the inconsistency of "old data not being cleaned but new data being written." After a successful delete, the progress record is physically deleted to save storage; error can be automatically repaired and reset back to new before entering the scheduling process.
[0044] For example, for newly added data, its status is set to new and action to new. After a successful push, status is set to done and action to new; after a failed push, status is set to error and action to new, allowing for automatic or manual retries. After the first successful push, if the corresponding resource or data is deleted, status is set to new and action to delete; if deletion fails, status is set to error and action to delete; if deletion succeeds, the record is physically deleted. If a data update operation is detected after the first successful push, status is set to new and action to update; if the update fails, status is set to error and action to update, and subsequent automatic or manual retries are performed. Status is then set to new after confirming the deletion of the old graph node.
[0045] In some embodiments of this application, before executing S230, the method for incremental data synchronization may further include: calling the health check interface of the source data knowledge graph; and confirming that the return status of the health check interface is a healthy value.
[0046] For example, in a specific embodiment of this application, a health check is performed on the knowledge graph service 150 to avoid generating a large number of failure records when the service is unavailable. The specific method is to use a "liveness detection interface + timeout control + result judgment + degradation handling + recording diagnostic information". At the beginning of each round, the scheduled task module calls the healthCheck interface of the knowledge graph service 150 (as a specific example of a health check interface). The health check determines whether the conditions are met to confirm whether the service is healthy. For example, if the interface returns a "success" status (such as HTTP 200 or return code success=true, as a specific example of a health value), the service is healthy; otherwise, it is unhealthy, and an abnormal service status is returned. If the health check fails, i.e., the service is unhealthy, all update / delete / new stages in this round are skipped, and the current scheduling is terminated directly. No status changes are made to the Progress records in the source data knowledge graph, thus avoiding marking a large number of records as errors. After the service's health status is restored (such as HTTP 200 or return code success=true), the subsequent stages proceed in a predetermined order: full update → full delete → new in batches. To facilitate automatic repair and maintenance location, the following are recorded during health checks: call time, timeout / exception type, return code / error message; number of consecutive failures (which can be used for alarm thresholds), etc.
[0047] If the health check passes, perform the following update operation; otherwise, end this scheduling.
[0048] S230, the source data knowledge graph is updated using the data set to obtain the updated knowledge graph.
[0049] For example, in a specific embodiment of this application, the source data knowledge graph is modified using the data set obtained above to obtain an updated knowledge graph. The resource-level statistics and last synchronization time in the updated knowledge graph are then written back in the resource association layer 120. During the update process, update and delete operations are performed on a full scale first to ensure that old or invalid data is cleaned up first, reducing conflicts with new writes; new operations are performed in batches to reduce peak memory and network usage per operation, and failure in one batch does not affect subsequent batches; failed objects enter an error state, are categorized and processed by the automatic repair module, and are retried.
[0050] In some embodiments of this application, S230 may include: when the data set is the update set, querying the old graph node corresponding to the resource object corresponding to the update set from the source data knowledge graph; deleting the old graph node, and writing the data in the update set to the source data knowledge graph to obtain the updated knowledge graph.
[0051] For example, in a specific embodiment of this application, during the Update phase, the query conditions for an old graph node are the data status="new" AND action="update" of a certain resource object. By calling the data deletion function deleteData, the graph unique identifier graphDataId and graph parameter configuration graphDataParameters of the involved old graph nodes are deleted. After successful deletion, the action and status of the data are changed to "new". Finally, the data is updated to the corresponding graph node to obtain the updated knowledge graph. If deletion fails, status is changed to "error" and error message is written. For example, for resource object A, object A (objectId=101): content change → status=new, action=update, graphDataId="g-aaa", graphDataParameters={p1...}, rawData="new content A".
[0052] Specifically, query all nodes with status=new AND action=update to obtain object A; call deleteData(graphDataId="g-aaa", graphDataParameters={p1...}) to delete the old graph node; if deleteData succeeds: update the Progress of object A to: action=new, status=new (indicating "old has been deleted, waiting to be written as new"); if deleteData fails: set the status of object A to error and record errorMessage, waiting for subsequent automatic repair / retry.
[0053] In some embodiments of this application, S230 may include: when the data set is the deletion set, querying the process data of the resource objects to be deleted in the deletion set from the source data knowledge graph; deleting the process data from the source data knowledge graph to obtain the updated knowledge graph.
[0054] For example, in a specific embodiment of this application, the query conditions for the Delete stage are: status="new" AND action="delete". By calling the data deletion function deleteData, the unique graph identifier graphDataId and graph parameter configuration graphDataParameters of the resource object to be deleted are deleted. If the deletion is successful, the content of the physically deleted Progress record is recorded; otherwise, status="error" is set and an error is recorded. For example, for resource object B to be deleted: Object B (objectId=102): Deleted at the source → status=new, action=delete, graphDataId="g-bbb", graphDataParameters={p2...}.
[0055] Specifically, query all instances of status=new AND action=delete to obtain object B; call deleteData(graphDataId="g-bbb", graphDataParameters={p2...}); if the deletion is successful, physically delete the Progress record of object B (the progress table no longer retains this object); if the deletion fails, set status=error and record errorMessage, and retry in the next round or handle manually.
[0056] In some embodiments of this application, S230 may include: when the data set is the newly added set, constructing a data write request; adding the data in the newly added set to the source data knowledge graph based on the data write request, thereby obtaining the updated knowledge graph.
[0057] For example, in a specific embodiment of this application, for the new stage, the query conditions for this stage are status="new" AND action="new", and data records in the newly added set are retrieved according to the batch size; by constructing the pushData request data, the input parameters are written: rawData, nodeSets (the nodeSets for constructing the request), and if necessary, other context such as the dataset name involved in the data to be written in the newly added set are also written; after successful write-back, the new graph node graphDataId and graph configuration parameters graphDataParameters are returned, and status="done" is set; if the write-back fails, status="error" is set and the error message is recorded. For example, for the newly added resource object C, object C (objectId=103): add → status=new, action=new, rawData="content C", nodeSets=["dataset-10"].
[0058] Specifically, queries are performed in batches based on `status=new AND action=new`. This includes object C (which was originally `new`) and object A (which was converted to `new` after a successful `delete` in the `Update` phase). A batch `pushData` request is constructed: Object A requests `nodeSets=...` (inherited from `progress` or `resource`), `data=rawData("New content A")`; Object C requests `nodeSets=["dataset-10"]`, `data=rawData("Content C")`. After calling `pushData([A,C])`, each record is written back: If object A succeeds, `graphDataId="g-aaa2", graphDataParameters={p1new...}`, `status=done` is written back; if object C succeeds, `graphDataId="g-ccc", graphDataParameters={p3...}`, `status=done` is written back. If a record fails, its `status=error` is set and an `errorMessage` is written, without affecting other successful records in the same batch. The batch is looped until no more records with `status=new AND action=new` can be found.
[0059] The following is in conjunction with the appendix Figure 3 The present application provides an exemplary description of the specific process of incremental data synchronization provided by some embodiments.
[0060] Please see the appendix Figure 3 , Figure 3A flowchart of a data incremental synchronization method provided for some embodiments of this application.
[0061] The above process is illustrated below by example.
[0062] S310: Obtain the business resource information to be processed and obtain the source data set.
[0063] The source data set is the source data knowledge graph.
[0064] S320: Obtain the set of associated objects corresponding to the resource objects in the business resource information to be processed from the source data set.
[0065] S330: Preprocess the data in the associated object collection to obtain the processed data.
[0066] S340 compares the processed data with the resource data of each type of resource object to obtain the data type.
[0067] S350, construct a data set that matches the data type of each type of resource object.
[0068] S360 updates the source dataset using the dataset to obtain the updated knowledge graph.
[0069] It is understood that the specific implementation process of S310~S360 can be referred to the method implementation examples provided above. To avoid repetition, detailed descriptions are omitted here.
[0070] Please refer to Figure 4 , Figure 4 The diagram illustrates a block diagram of a data incremental synchronization apparatus provided in some embodiments of this application. It should be understood that this data incremental synchronization apparatus corresponds to the method embodiments described above and is capable of performing the various steps involved in the method embodiments. The specific functions of this data incremental synchronization apparatus can be found in the description above; detailed descriptions are omitted here to avoid repetition.
[0071] Figure 4The data incremental synchronization device includes at least one software functional module that can be stored in a memory or embedded in the data incremental synchronization device in the form of software or firmware. The data incremental synchronization device includes: a difference analysis module 410, used to perform difference analysis on the business resource information to be processed and the source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed; wherein, the data type includes newly added data, updated data, or deleted data; a construction module 420, used to construct a data set that matches the data type of each type of resource object; wherein, the data set includes a newly added set, an updated set, or a deleted set; and an incremental synchronization module 430, used to update the source data knowledge graph through the data set to obtain the updated knowledge graph.
[0072] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.
[0073] Some embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can perform the operation of any of the methods corresponding to the methods provided in the above embodiments.
[0074] Some embodiments of this application also provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operation of any of the methods corresponding to the above embodiments provided in the above embodiments.
[0075] like Figure 5 As shown, some embodiments of this application provide an electronic device 500, which includes a memory 510, a processor 520, and a computer program stored in the memory 510 and executable on the processor 520. When the processor 520 reads the program from the memory 510 via a bus 530 and executes the program, it can implement the methods of any of the above embodiments.
[0076] Processor 520 can process digital signals and can include various computing architectures. For example, it can be a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements multiple instruction set combinations. In some examples, processor 520 can be a microprocessor.
[0077] The memory 510 can be used to store instructions executed by the processor 520 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all of the functions of one or more modules described in the embodiments of this application. The processor 520 of this disclosure embodiment can be used to execute the instructions in the memory 510 to implement the methods shown above. The memory 510 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.
[0078] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0079] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0080] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. A method for incremental data synchronization, characterized in that, include: The pending business resource information is compared with the source data knowledge graph to obtain the data type of each type of resource object in the pending business resource information; wherein, the data type includes newly added data, updated data, or deleted data; Construct a data set that matches the data type of each type of resource object; wherein the data set includes an addition set, an update set, or a deletion set; The source data knowledge graph is updated using the data set to obtain the updated knowledge graph.
2. The method as described in claim 1, characterized in that, Before performing differential analysis between the business resource information to be processed and the source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed, the method further includes: Read the business scenario, data attributes, and data identifier of the business resource information to be processed; Based on the data attributes, a set of associated objects is obtained from the source data knowledge graph.
3. The method as described in claim 2, characterized in that, The step of performing differential analysis between the business resource information to be processed and the source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed includes: The data in the set of associated objects is preprocessed to obtain the processed data; The processed data is compared with the resource data of each type of resource object to obtain the data type.
4. The method according to any one of claims 1-3, characterized in that, Before updating the source data knowledge graph using the dataset, the method further includes: Call the health check interface of the source data knowledge graph; Confirm that the health check interface returns a healthy value.
5. The method according to any one of claims 1-3, characterized in that, The step of updating the source data knowledge graph using the data set to obtain the updated knowledge graph includes: When the data set is the updated set, query the old graph node corresponding to the resource object corresponding to the updated set from the source data knowledge graph; The old knowledge graph nodes are deleted, and the data in the updated set is written to the source data knowledge graph to obtain the updated knowledge graph.
6. The method according to any one of claims 1-3, characterized in that, The step of updating the source data knowledge graph using the data set to obtain the updated knowledge graph includes: When the data set is the deletion set, the process data of the resource objects to be deleted in the deletion set are queried from the source data knowledge graph. The updated knowledge graph is obtained by deleting the process data from the source data knowledge graph.
7. The method according to any one of claims 1-3, characterized in that, The step of updating the source data knowledge graph using the data set to obtain the updated knowledge graph includes: When the data set is the newly added set, a data write request is constructed; Based on the data write request, the data in the newly added set is added to the source data knowledge graph to obtain the updated knowledge graph.
8. A device for incremental data synchronization, characterized in that, include: The difference analysis module is used to perform difference analysis between the business resource information to be processed and the source data knowledge graph to obtain the data type of each type of resource object in the business resource information to be processed; wherein, the data type includes newly added data, updated data, or deleted data; A construction module is used to construct a data set that matches the data type of each type of resource object; wherein the data set includes an addition set, an update set, or a deletion set; The incremental synchronization module is used to update the source data knowledge graph using the data set to obtain the updated knowledge graph.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is executed by a processor to perform the method as described in any one of claims 1-7.
10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the computer program is executed by the processor to perform the method as described in any one of claims 1-7.