Distributed data sharing method, apparatus and electronic device

Abstract

This application discloses a distributed data sharing method, apparatus, and electronic device, belonging to the field of data sharing technology. The method includes: upon receiving a data change event, obtaining a target synchronization strategy for the data change event; the target synchronization strategy includes a target operator chain; if the target synchronization strategy passes a first strategy verification, performing change conflict detection on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, performing conflict processing based on the target operator chain to generate a data change result; if the data change result indicates successful data change, sending the data change result to the next node. The distributed data sharing method disclosed in this application solves the problem that it is difficult to guarantee data consistency when sharing data based on message queues in distributed scenarios.

Description

Technical Field

[0001] This application belongs to the field of data sharing technology, and in particular relates to a distributed data sharing method, apparatus and electronic device. Background Technology

[0002] To accommodate unidirectional, asynchronous data sharing, highly available and scalable data replication is typically achieved using message queues (such as Pulsar or Kafka). However, in distributed data sharing platforms, business requirements often involve bidirectional data synchronization and state collaboration.

[0003] In distributed scenarios, when multiple parties concurrently modify the same data, message queue-based approaches struggle to guarantee data consistency. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a distributed data sharing method, apparatus, and electronic device to solve the problem that it is difficult to guarantee data consistency when sharing data based on message queues in distributed scenarios.

[0005] Firstly, this application provides a distributed data sharing method, including: Upon receiving a data change event, obtain the target synchronization strategy for the data change event; the target synchronization strategy includes the target operator chain. If the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy. In the event of a data change conflict, conflict handling is performed based on the target operator chain to generate the data change result; If the data change result indicates that the data change was successful, the data change result is sent to the next node.

[0006] According to the distributed data sharing method of this application, upon receiving a data change event, the target synchronization strategy for the data change event is obtained; if the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, conflict handling is performed based on the target operator chain in the target synchronization strategy corresponding to the data change event. This method ensures data consistency when multiple parties concurrently modify the same data, resulting in a change conflict in the data change event.

[0007] According to one embodiment of this application, upon receiving a data change event, obtaining the target synchronization strategy for the data change event includes: Upon receiving a data change event, the target synchronization policy corresponding to the data change event is searched in the local storage synchronization policy. If the search fails, a policy pull event is generated, and the target synchronization policy is pulled from the centralized policy center based on the policy pull event.

[0008] According to one embodiment of this application, before obtaining the target synchronization strategy for the data change event upon receiving a data change event, the method further includes: Obtain the target text input by the user; the target text represents the data synchronization requirement. Using a large language model, intent is parsed from the target text to generate a strategy draft; Based on the strategy draft, a synchronization strategy is generated.

[0009] According to one embodiment of this application, a synchronization strategy is generated based on a strategy draft, including: Perform a second policy check on the policy draft; the second policy check includes at least one of the following: syntax check, reference check, dependency check, constraint check, and deduction check; If the second strategy verification passes, semantic parsing is performed on the strategy draft; The semantic parsing results are compiled to generate the target operator chain, and the target operator chain and private key are encapsulated into the target synchronization strategy.

[0010] According to one embodiment of this application, after obtaining the target synchronization strategy for the data change event upon receiving a data change event, the method further includes: Perform a first policy verification on the target synchronization policy; the first policy verification includes at least one of policy version verification, key verification, and effective time verification.

[0011] According to one embodiment of this application, if the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy, including: If the target synchronization strategy passes the first strategy verification, obtain the target data change record; the target data change record is the local uncommitted data change record or the last historical data change record; Based on the target data change records, perform change conflict detection on data change events.

[0012] According to one embodiment of this application, in the event of a data change conflict, conflict processing is performed based on the target operator chain to generate a data change result, including: In the event of a data change event conflict, determine the type of change conflict. Based on the target operator chain, decision conflict handling is performed according to the change conflict type to generate data change results; decision conflict handling includes at least one of data overwrite, data addition, data deletion and ignoring.

[0013] According to one embodiment of this application, in the event of a data change conflict, after performing conflict handling based on a target synchronization strategy to generate a data change result, the method further includes: If the data change result indicates that the data change was unsuccessful, an arbitration event is generated based on the data change event and the data change result, and the arbitration event is written to the task processing queue.

[0014] Secondly, this application provides a distributed data sharing device, comprising: The first acquisition module is used to acquire the target synchronization strategy of the data change event upon receiving the data change event; the target synchronization strategy includes a target operator chain. The detection module is used to perform change conflict detection on data change events based on the target synchronization strategy, provided that the target synchronization strategy passes the first strategy verification. The processing module is used to perform conflict processing based on the target operator chain in the event of a data change conflict, so as to generate a data change result; The sending module is used to send the data change result to the next node when the data change result indicates that the data change was successful.

[0015] According to the distributed data sharing device of this application, upon receiving a data change event, the device obtains the target synchronization strategy for the data change event; if the target synchronization strategy passes the first strategy verification, it performs change conflict detection on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, it performs conflict handling based on the target operator chain in the target synchronization strategy corresponding to the data change event. This is to ensure data consistency when multiple parties concurrently modify the same data, resulting in a change conflict in the data change event.

[0016] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the distributed data sharing method of the first aspect described above.

[0017] Fourthly, this application provides a non-transitory computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the distributed data sharing method of the first aspect described above.

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating the distributed data sharing method provided in the embodiments of this application; Figure 2 This is a second flowchart illustrating the distributed data sharing method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the distributed data sharing device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0022] The distributed data sharing method, electronic device, and storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0023] The distributed data sharing method can be applied to the terminal, specifically executed by the hardware or software within the terminal.

[0024] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0025] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0026] The distributed data sharing method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the distributed data sharing method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The distributed data sharing method provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0027] In some embodiments, this distributed data sharing method can be applied to the client.

[0028] like Figure 1 As shown, the distributed data sharing method includes steps 110, 120, 130 and 140.

[0029] Step 110: Upon receiving a data change event, obtain the target synchronization strategy for the data change event; the target synchronization strategy includes the target operator chain.

[0030] In some embodiments, in a distributed data sharing scenario, multiple business systems can share the same database. Each of these systems can perform modification, deletion, addition, or any theoretically feasible change operation on data entities in any table within the database. For example, the health commission system and the public security system share a population census data table.

[0031] In some embodiments, when a data change operation is detected by the first business system based on Change Data Capture (CDC), a data change event is generated and sent to the agent device of the second business system based on the communication message bus; the first business system and the second business system can both be any business system among multiple business systems in a distributed data sharing scenario.

[0032] In some embodiments, the database increment can be captured based on the CaptureAgent algorithm of Debezium, GoldenGate or any theoretically feasible technology, and the field desensitization and filtering can be completed based on the Transformation function, and the data change event can be output based on the Serializer function.

[0033] In actual execution, data change events can include the event number eventId, the entity primary key entityId, the entity type entityType, the operation type op (e.g., DELETE, INSERT, or UPDATE), the before record, the after record, the changed field set changedFields, the business submission timestamp ts, the policy version policyRef, the source system sourceOrg, and the target system targetOrg.

[0034] In some embodiments, multiple pre-generated synchronization strategies can be stored. When a data change event is received, the target synchronization strategy corresponding to the data change event is matched among the multiple synchronization strategies.

[0035] In some embodiments, the user's input data synchronization requirements can be obtained in advance, and a synchronization strategy can be generated based on the data synchronization requirements.

[0036] In some embodiments, after obtaining the target synchronization policy for the data change event, a first policy verification can be performed on the target synchronization policy.

[0037] Step 120: If the target synchronization strategy passes the first strategy verification, perform change conflict detection on the data change event based on the target synchronization strategy.

[0038] In related technologies, in scenarios where the first business system and the second business system share data, and when the first business system and the second business system concurrently perform change operations on the data table, taking the first business system as an example, the first business system itself generates a processing change event for the data table, and at the same time, it also receives a data change event for the data table from the second business system. It is impossible to determine whether the change operation on the data table is based on the processing change event or the data change event, resulting in a change conflict.

[0039] In some embodiments, if the target synchronization policy passes the first policy verification, change conflict detection can be performed on local data change records and data change events based on the target synchronization policy.

[0040] In some embodiments, if there is no change conflict in the data change event, the data change event is processed normally to generate a data change result, and the data change result is written to the local system through the PersistenceBuffer, and the AuditPublisher records the automatic decision audit event.

[0041] Step 130: In the event of a data change conflict, perform conflict handling based on the target operator chain to generate the data change result.

[0042] In some embodiments, when there is a change conflict in a data change event, the unique change result of the data targeted by the data change event is determined based on the target operator chain in the target synchronization strategy.

[0043] In some embodiments, in the event of a failure to resolve a conflict, an arbitration event may be generated, and the conflicting data change event may be handled manually.

[0044] Step 140: If the data change result indicates that the data change was successful, send the data change result to the next node.

[0045] In some embodiments, if a unique change result is determined for the data targeted by the data change event based on the target operator chain, the data change is considered successful, and the data change result is sent to the next node. The next node can be the next business system sharing the data.

[0046] According to the distributed data sharing method of this application embodiment, upon receiving a data change event, a target synchronization strategy for the data change event is obtained; if the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, conflict processing is performed based on the target operator chain in the target synchronization strategy corresponding to the data change event. This is to generate a data change result in the case where multiple parties concurrently modify the same data, resulting in a change conflict in the data change event, thereby ensuring data consistency.

[0047] In some embodiments, upon receiving a data change event, the target synchronization policy corresponding to the data change event is searched in the locally stored synchronization policies; if the search fails, a policy retrieval event is generated, and the target synchronization policy is retrieved from the centralized policy center based on the policy retrieval event.

[0048] In some embodiments, upon receiving a data change event, the target synchronization strategy corresponding to the data change event can be found in the local storage synchronization strategy based on information such as the entity primary key (entityId), entity type (entityType), and operation type (op) in the data change event.

[0049] In some embodiments, the synchronization strategy that matches the information in the data change event, such as the entity primary key (entityId), entity type (entityType), and operation type (op), in the local storage synchronization strategy can be determined as the target synchronization strategy.

[0050] In some embodiments, if the target synchronization policy cannot be found in the locally stored synchronization policy, a policy retrieval event can be generated based on the data change event to obtain the target synchronization policy corresponding to the data change event.

[0051] According to the distributed data sharing method of this application embodiment, upon receiving a data change event, a target synchronization strategy for the data change event is obtained; if the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, conflict handling is performed based on the target synchronization strategy corresponding to the data change event. This method aims to generate a data change result and ensure data consistency when multiple parties concurrently modify the same data, resulting in a change conflict in the data change event.

[0052] In some embodiments, upon receiving a data change event, before obtaining the target synchronization strategy for the data change event, the target text input by the user is obtained; the target text represents the data synchronization requirement; the intent of the target text is parsed using a large language model to generate a strategy draft; and a synchronization strategy is generated based on the strategy draft.

[0053] In some embodiments, the target text input by the user can be obtained based on an input component. The input component can be a text input box, a voice acquisition component, or any theoretically feasible input component, and this application does not specifically limit it. For example, the user inputs the following target text: "Synchronize the basic patient information of business system A to business systems B and C; where the mobile phone number is based on business system B, the home address is prioritized based on the last update time, the diagnosis opinion field can only be synchronized unidirectionally from business system A to business system B, if business system B and business system C modify the contact field simultaneously within a 10-minute window, then manual arbitration will be initiated; the policy is valid for 30 days, and will be released to tenant T1 in a gray-scale manner first."

[0054] In some embodiments, after obtaining the target text, the Prompt template library of the large language model can be used to perform operations such as intent extraction, parameter completion, and sensitive word filtering on the target text, and output a strategy draft.

[0055] In some embodiments, after obtaining the target text, intent parsing can be performed on the target text using a large language model, and a strategy draft can be generated based on the parsing results.

[0056] In some embodiments, after obtaining the strategy draft, the StrategyMetadata function of the large language model can be used to automatically identify missing fields and provide revision suggestions based on contextual constraints such as synchronization direction.

[0057] In some embodiments, after obtaining the policy draft, a verification operation can be performed on the policy draft, and if the verification passes, a synchronization policy can be generated based on the policy draft.

[0058] In some embodiments, the synchronization strategy includes a policy ID, a business object entity (including entity name, primary key format, tenant, source business system set, target business system set, and field identifiers), conflict handling rules (including synchronization direction, triggering conditions, field filtering rules, desensitization rules, priority rules, conflict detection conditions, conflict resolution strategies, retry thresholds, and compensation actions), and policy lifecycle metadata.

[0059] According to the distributed data sharing method of this application embodiment, by obtaining the target text representing the data synchronization requirement input by the user, using a large language model to perform intent parsing on the target text, a strategy draft is generated; based on the strategy draft, a synchronization strategy is generated to generate a synchronization strategy for resolving data change conflicts through natural language generation, thereby improving the understandability of the data change conflict resolution process, enhancing the semantics of the change conflict resolution rules, and significantly reducing the participation threshold for business personnel.

[0060] In some embodiments, a second policy verification is performed on the policy draft; the second policy verification includes at least one of syntax verification, reference verification, dependency verification, constraint verification, and deduction verification; if the second policy verification passes, semantic parsing is performed on the policy draft; a compilation operation is performed on the parsing result of the semantic parsing to generate a target operator chain, and the target operator chain and the private key are encapsulated into a target synchronization policy.

[0061] In some embodiments, syntax validation is used to check whether the policy draft meets the predefined syntax template, avoiding basic errors such as missing fields, unclosed parentheses, incomplete conditional expressions, incorrect boundary values, and insecurity.

[0062] In some embodiments, reference verification is used to perform schema-level verification, comparing the entities, fields, enumeration values, and business system identifiers referenced in the policy draft with the formal models in the metadata center item by item to determine that the referenced objects actually exist.

[0063] In some embodiments, dependency validation is used to check whether an illegal circular dependency is formed between field desensitization, field mapping, conflict conditions, and compensation actions.

[0064] In some embodiments, constraint validation is used to detect whether conflict handling rules in the policy draft are prohibited, contradictory, or have expired.

[0065] In some embodiments, the deductive verification is used to detect whether the draft strategy can generate deterministic convergence results in scenarios of addition, update, deletion and concurrent modification, based on data change sample events.

[0066] In some embodiments, if all policy drafts pass syntax validation, citation validation, dependency validation, constraint validation, and deduction validation, the policy draft is determined to have passed the second policy validation. Conversely, if the policy draft fails any one or more of the following validations: syntax validation, citation validation, dependency validation, constraint validation, and deduction validation, the policy draft is determined to have failed the second policy validation.

[0067] In some embodiments, if the second policy verification passes, the ExecutionPlanner function can be used to generate a target operator chain based on the policy type (LWW, PRIORITY, FIELD_MERGE, MANUAL_ARBITRATION, etc.), and perform RFC8785 standardization processing on the target operator chain. The processed target operator chain and the private key (e.g., the RSA private key or ECDSA private key provided by HSM) are then encapsulated into a target synchronization policy.

[0068] In some embodiments, the target operator chain is a device-executable command transformed from a policy draft.

[0069] According to the distributed data sharing method of this application embodiment, by obtaining the target text representing the data synchronization requirement input by the user, a large language model is used to generate a strategy draft based on the target text; based on the strategy draft, a synchronization strategy is generated to generate a synchronization strategy to resolve data change conflicts through natural language generation, thereby improving the understandability of the data change conflict resolution process, enhancing the semantics of the change conflict resolution rules, and significantly reducing the participation threshold for business personnel.

[0070] In some embodiments, upon receiving a data change event, after obtaining the target synchronization policy of the data change event, a first policy verification can be performed on the target synchronization policy; the first policy verification includes at least one of policy version verification, key verification, and effective time verification.

[0071] In some embodiments, the PolicySignatureVerifier function can be used to verify whether the private key in the target synchronization policy is the same as the private key in the data change event, so as to perform key verification on the target synchronization policy.

[0072] In some embodiments, it can be verified whether the policy version in the target synchronization policy is consistent with the policy version in the data change event, so as to perform policy version verification on the target synchronization policy.

[0073] In some embodiments, it can be verified whether the current time is within the effective time interval of the target synchronization policy, so as to perform an effective time verification on the target synchronization policy.

[0074] In some embodiments, if the target synchronization policy passes the policy version verification, key verification, and effective time verification, it is determined that the target synchronization policy passes the first policy verification.

[0075] In some embodiments, if the target synchronization strategy fails the first strategy verification, the data change event is marked as an event to be evaluated, and the event to be evaluated is stored in the replay queue.

[0076] In some embodiments, the event to be evaluated can be replayed based on an exponential backoff strategy (e.g., 1 second → 5 seconds → 15 seconds → 30 seconds → 60 seconds) for reprocessing. If the number of replays of the event to be evaluated is greater than or equal to a threshold, an arbitration event is generated based on the event to be evaluated.

[0077] According to the distributed data sharing method of this application embodiment, upon receiving a data change event, a target synchronization strategy for the data change event is obtained; if the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, conflict handling is performed based on the target synchronization strategy corresponding to the data change event. This method aims to generate a data change result and ensure data consistency when multiple parties concurrently modify the same data, resulting in a change conflict in the data change event.

[0078] In some embodiments, if the target synchronization strategy passes the first strategy verification, the target data change record is obtained; the target data change record is a local uncommitted data change record or the last historical data change record; based on the target data change record, change conflict detection is performed on the data change event.

[0079] In some embodiments, after obtaining the target data change record, standardized processing can be performed on the target data change record and data change event to unify the time format, field name, null value representation and business system identifier.

[0080] In some embodiments, the target data change record can be compared with the source business system and data entity in the data change event to perform change conflict detection on the data change event.

[0081] In some embodiments, if the source business system in the target data change record and the data change event are different, and at least one field in the data entity is the same, it is determined that the data entity is concurrently modified by different business systems, that is, the data change event has a change conflict.

[0082] According to the distributed data sharing method of this application embodiment, upon receiving a data change event, a target synchronization strategy for the data change event is obtained; if the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, conflict handling is performed based on the target synchronization strategy corresponding to the data change event. This method aims to generate a data change result and ensure data consistency when multiple parties concurrently modify the same data, resulting in a change conflict in the data change event.

[0083] In some embodiments, when a data change event has a change conflict, the change conflict type of the data change event is determined; based on the target operator chain, decision conflict processing is performed according to the change conflict type to generate a data change result; the decision conflict processing includes at least one of data overwrite, data addition, data deletion, and ignoring.

[0084] In some embodiments, change conflict types include time sequence conflicts, priority conflicts, field overwrite conflicts, loopback write conflicts, and semantic contradiction conflicts.

[0085] In some embodiments, if the data change result indicates that the data change was successful, an audit work order can be generated based on the data change result.

[0086] In some embodiments, based on the target operator chain, decision conflict handling such as data overwriting, data addition, data deletion, and ignoring can be performed according to the change conflict type. For example, if the change conflict type is a time sequence conflict, and the execution logic of the target operator chain is to determine the data value with a later time as the unique data result, then a data overwriting operation is performed on the original data value based on the data value with a later time.

[0087] According to the distributed data sharing method of this application embodiment, upon receiving a data change event, a target synchronization strategy for the data change event is obtained; if the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, conflict handling is performed based on the target synchronization strategy corresponding to the data change event. This method aims to generate a data change result and ensure data consistency when multiple parties concurrently modify the same data, resulting in a change conflict in the data change event.

[0088] In some embodiments, when there is a change conflict in the data change event, conflict handling is performed based on the target synchronization strategy to generate the data change result. If the data change result indicates that the data change was unsuccessful, an arbitration event is generated based on the data change event and the data change result, and the arbitration event is written to the task processing queue.

[0089] In some embodiments, when the data change result indicates that the data change was unsuccessful, a conflict context can be generated based on the conflict cause, data change record, target synchronization strategy, historical audit, and risk tags in the data change result. Semantic analysis of the conflict context is performed using a large language model, outputting a natural language report. The natural language report includes natural language explanations, field-level suggested actions, and confidence scores. The GuardRailService function performs fact verification, sensitive word filtering, and compliance checks on the natural language report, and generates an arbitration event based on the results.

[0090] In some embodiments, when the number of arbitration events is greater than or equal to a threshold, an alarm work order can be generated and pushed based on the Monitor function to meet regulatory requirements.

[0091] To better understand the distributed data sharing method provided in the embodiments of this application, further explanation is provided below. It should be understood that the following discussion is merely exemplary.

[0092] This application provides a distributed data sharing method, the specific steps of which are as follows: Figure 2 As shown: Step 210: Obtain the target text input by the user; the target text represents the data synchronization requirement; use a large language model to perform intent parsing on the target text and generate a strategy draft; based on the strategy draft, generate a synchronization strategy.

[0093] In some embodiments, the target text input by the user can be obtained based on an input component. The input component can be a text input box, a voice acquisition component, or any theoretically feasible input component, and this application does not specifically limit it. For example, the user inputs the following target text: "Synchronize the basic patient information of business system A to business systems B and C; where the mobile phone number is based on business system B, the home address is prioritized based on the last update time, the diagnosis opinion field can only be synchronized unidirectionally from business system A to business system B, if business system B and business system C modify the contact field simultaneously within a 10-minute window, then manual arbitration will be initiated; the policy is valid for 30 days, and will be released to tenant T1 in a gray-scale manner first."

[0094] In some embodiments, after obtaining the target text, the Prompt template library of the large language model can be used to perform operations such as intent extraction, parameter completion, and sensitive word filtering on the target text, and output a strategy draft.

[0095] In some embodiments, after obtaining the target text, intent parsing can be performed on the target text using a large language model, and a strategy draft can be generated based on the parsing results.

[0096] In some embodiments, after obtaining the strategy draft, the StrategyMetadata function of the large language model can be used to automatically identify missing fields and provide revision suggestions based on contextual constraints such as synchronization direction.

[0097] In some embodiments, after obtaining the policy draft, a verification operation can be performed on the policy draft, and if the verification passes, a synchronization policy can be generated based on the policy draft.

[0098] In some embodiments, the synchronization strategy includes a policy ID, a business object entity (including entity name, primary key format, tenant, source business system set, target business system set, and field identifiers), conflict handling rules (including synchronization direction, triggering conditions, field filtering rules, desensitization rules, priority rules, conflict detection conditions, conflict resolution strategies, retry thresholds, and compensation actions), and policy lifecycle metadata.

[0099] In some embodiments, a second policy verification is performed on the policy draft; the second policy verification includes at least one of syntax verification, reference verification, dependency verification, constraint verification, and deduction verification; if the second policy verification passes, semantic parsing is performed on the policy draft; a compilation operation is performed on the parsing result of the semantic parsing to generate a target operator chain, and the target operator chain and the private key are encapsulated into a target synchronization policy.

[0100] In some embodiments, syntax validation is used to check whether the policy draft meets the predefined syntax template, avoiding basic errors such as missing fields, unclosed parentheses, incomplete conditional expressions, incorrect boundary values, and insecurity.

[0101] In some embodiments, reference verification is used to perform schema-level verification, comparing the entities, fields, enumeration values, and business system identifiers referenced in the policy draft with the formal models in the metadata center item by item to determine that the referenced objects actually exist.

[0102] In some embodiments, dependency validation is used to check whether an illegal circular dependency is formed between field desensitization, field mapping, conflict conditions, and compensation actions.

[0103] In some embodiments, constraint validation is used to detect whether conflict handling rules in the policy draft are prohibited, contradictory, or have expired.

[0104] In some embodiments, the deductive verification is used to detect whether the draft strategy can generate deterministic convergence results in scenarios of addition, update, deletion and concurrent modification, based on data change sample events.

[0105] In some embodiments, if all policy drafts pass syntax validation, citation validation, dependency validation, constraint validation, and deduction validation, the policy draft is determined to have passed the second policy validation. Conversely, if the policy draft fails any one or more of the following validations: syntax validation, citation validation, dependency validation, constraint validation, and deduction validation, the policy draft is determined to have failed the second policy validation.

[0106] In some embodiments, if the second policy verification passes, the ExecutionPlanner function can be used to generate a target operator chain based on the policy type (LWW, PRIORITY, FIELD_MERGE, MANUAL_ARBITRATION, etc.), and perform RFC8785 standardization processing on the target operator chain. The processed target operator chain and the private key (e.g., the RSA private key or ECDSA private key provided by HSM) are then encapsulated into a target synchronization policy.

[0107] Step 220: Upon receiving a data change event, search for the target synchronization strategy corresponding to the data change event in the local storage synchronization strategy; the target synchronization strategy includes a target operator chain.

[0108] In some embodiments, in a distributed data sharing scenario, multiple business systems can share the same database. Each of these systems can perform modification, deletion, addition, or any theoretically feasible change operation on data entities in any table within the database. For example, the health commission system and the public security system share a population census data table.

[0109] In some embodiments, when a data change operation is detected by the first business system based on Change Data Capture (CDC), a data change event is generated and sent to the agent device of the second business system based on the communication message bus; the first business system and the second business system can both be any business system among multiple business systems in a distributed data sharing scenario.

[0110] In some embodiments, the database increment can be captured based on the CaptureAgent algorithm of Debezium, GoldenGate or any theoretically feasible technology, and the field desensitization and filtering can be completed based on the Transformation function, and the data change event can be output based on the Serializer function.

[0111] In actual execution, data change events can include the event number eventId, the entity primary key entityId, the entity type entityType, the operation type op (e.g., DELETE, INSERT, or UPDATE), the before record, the after record, the changed field set changedFields, the business submission timestamp ts, the policy version policyRef, the source system sourceOrg, and the target system targetOrg.

[0112] In some embodiments, upon receiving a data change event, the target synchronization strategy corresponding to the data change event can be found in the local storage synchronization strategy based on information such as the entity primary key (entityId), entity type (entityType), and operation type (op) in the data change event.

[0113] In some embodiments, the synchronization strategy that matches the information in the data change event, such as the entity primary key (entityId), entity type (entityType), and operation type (op), in the local storage synchronization strategy can be determined as the target synchronization strategy.

[0114] In some embodiments, if the target synchronization policy cannot be found in the locally stored synchronization policy, a policy retrieval event can be generated based on the data change event to obtain the target synchronization policy corresponding to the data change event.

[0115] Step 230: Perform a first policy verification on the target synchronization policy; the first policy verification includes at least one of policy version verification, key verification, and effective time verification.

[0116] In some embodiments, the PolicySignatureVerifier function can be used to verify whether the private key in the target synchronization policy is the same as the private key in the data change event, so as to perform key verification on the target synchronization policy.

[0117] In some embodiments, it can be verified whether the policy version in the target synchronization policy is consistent with the policy version in the data change event, so as to perform policy version verification on the target synchronization policy.

[0118] In some embodiments, it can be verified whether the current time is within the effective time interval of the target synchronization policy, so as to perform an effective time verification on the target synchronization policy.

[0119] In some embodiments, if the target synchronization policy passes the policy version verification, key verification, and effective time verification, it is determined that the target synchronization policy passes the first policy verification.

[0120] In some embodiments, if the target synchronization strategy fails the first strategy verification, the data change event is marked as an event to be evaluated, and the event to be evaluated is stored in the replay queue.

[0121] In some embodiments, the event to be evaluated can be replayed based on an exponential backoff strategy (e.g., 1 second → 5 seconds → 15 seconds → 30 seconds → 60 seconds) for reprocessing. If the number of replays of the event to be evaluated is greater than or equal to a threshold, an arbitration event is generated based on the event to be evaluated.

[0122] Step 240: If the target synchronization strategy passes the first strategy verification, obtain the target data change record; the target data change record is the local uncommitted data change record or the last historical data change record; based on the target data change record, perform change conflict detection on the data change event.

[0123] In some embodiments, after obtaining the target data change record, standardized processing can be performed on the target data change record and data change event to unify the time format, field name, null value representation and business system identifier.

[0124] In some embodiments, the target data change record can be compared with the source business system and data entity in the data change event to perform change conflict detection on the data change event.

[0125] In some embodiments, if the source business system in the target data change record and the data change event are different, and at least one field in the data entity is the same, it is determined that the data entity is concurrently modified by different business systems, that is, the data change event has a change conflict.

[0126] Step 250: In the event of a data change event with a change conflict, determine the type of change conflict; based on the target synchronization strategy, perform decision conflict handling according to the type of change conflict to generate the data change result.

[0127] In some embodiments, change conflict types include time sequence conflicts, priority conflicts, field overwrite conflicts, loopback write conflicts, and semantic contradiction conflicts.

[0128] In some embodiments, if the data change result indicates that the data change was successful, an audit work order can be generated based on the data change result.

[0129] Step 260: If the data change result indicates that the data change was unsuccessful, generate an arbitration event based on the data change event and the data change result, and write the arbitration event into the task processing queue.

[0130] In some embodiments, when the data change result indicates that the data change was unsuccessful, a conflict context can be generated based on the conflict cause, data change record, target synchronization strategy, historical audit, and risk tags in the data change result. Semantic analysis of the conflict context is performed using a large language model, outputting a natural language report. The natural language report includes natural language explanations, field-level suggested actions, and confidence scores. The GuardRailService function performs fact verification, sensitive word filtering, and compliance checks on the natural language report, and generates an arbitration event based on the results.

[0131] In some embodiments, when the number of arbitration events is greater than or equal to a threshold, an alarm work order can be generated and pushed based on the Monitor function to meet regulatory requirements.

[0132] Step 270: If the data change result indicates that the data change was successful, send the data change result to the next node.

[0133] In some embodiments, if a unique change result is determined for the data targeted by the data change event based on the target operator chain, the data change is considered successful, and the data change result is sent to the next node. The next node can be the next business system sharing the data.

[0134] This application also provides a distributed data sharing device.

[0135] like Figure 3 As shown, the distributed data sharing device 300 includes a first acquisition module 310, a detection module 320, a processing module 330, and a sending module 340. The first acquisition module 310 is used to acquire the target synchronization strategy of the data change event upon receiving a data change event; the target synchronization strategy includes a target operator chain. The detection module 320 is used to perform change conflict detection on data change events based on the target synchronization strategy, provided that the target synchronization strategy passes the first strategy verification. The processing module 330 is used to perform conflict processing based on the target operator chain to generate data change results when there is a change conflict in the data change event; The sending module 340 is used to send the data change result to the next node when the data change result indicates that the data change was successful.

[0136] According to the distributed data sharing device of this application embodiment, upon receiving a data change event, the device obtains the target synchronization strategy for the data change event; if the target synchronization strategy passes the first strategy verification, it performs change conflict detection on the data change event based on the target synchronization strategy; if a change conflict exists in the data change event, it performs conflict handling based on the target synchronization strategy corresponding to the data change event. This is to generate a data change result in the case where multiple parties concurrently modify the same data, resulting in a change conflict in the data change event, thereby ensuring data consistency.

[0137] In some embodiments, the first acquisition module 310 includes: The first acquisition unit is used to find the target synchronization strategy corresponding to the data change event in the synchronization strategy stored locally when a data change event is received. The first generation unit is used to generate a policy retrieval event in the event of a search failure, and to retrieve the target synchronization policy from the centralized policy center based on the policy retrieval event.

[0138] In some embodiments, the distributed data sharing device 400 further includes: The second acquisition module is used to acquire the target text input by the user; the target text represents the data synchronization requirement. The first generation module is used to perform intent parsing on the target text using a large language model and generate a strategy draft. The second generation module is used to generate a synchronization strategy based on the strategy draft.

[0139] In some embodiments, the second generation module includes: The verification unit is used to perform a second policy verification on the policy draft; the second policy verification includes at least one of syntax verification, reference verification, dependency verification, constraint verification and deduction verification. The parsing unit is used to perform semantic parsing on the policy draft if the second policy verification passes. The second generation unit is used to perform a compilation operation on the semantic parsing results, generate the target operator chain, and encapsulate the target operator chain and the private key into a target synchronization strategy.

[0140] In some embodiments, the distributed data sharing device 400 further includes: The verification module is used to perform a first policy verification on the target synchronization policy; the first policy verification includes at least one of policy version verification, key verification and effective time verification.

[0141] In some embodiments, the detection module 320 includes: The second acquisition unit is used to acquire the target data change record when the target synchronization strategy passes the first strategy verification; the target data change record is the local uncommitted data change record or the last historical data change record; The detection unit is used to perform change conflict detection on data change events based on the target data change record.

[0142] In some embodiments, the processing module 330 includes: The classification unit is used to determine the type of change conflict in a data change event when there is a change conflict. The processing unit is used to perform decision conflict processing based on the target operator chain and according to the change conflict type to generate data change results; the decision conflict processing includes at least one of data overwrite, data addition, data deletion and ignoring.

[0143] In some embodiments, the distributed data sharing device 400 further includes: The third generation module is used to generate an arbitration event based on the data change event and the data change result when the data change result indicates that the data change was unsuccessful, and write the arbitration event into the task processing queue.

[0144] The distributed data sharing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific implementation.

[0145] The distributed data sharing device in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0146] The distributed data sharing device provided in this application embodiment can achieve... Figure 1 and Figure 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0147] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored on the memory 402 and executable on the processor 401. When the program is executed by the processor 401, it implements the various processes of the above-described distributed data sharing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0148] It should be noted that the computer equipment in this application embodiment includes the mobile electronic equipment and non-mobile electronic equipment described above.

[0149] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described distributed data sharing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0150] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0151] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described distributed data sharing method.

[0152] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0153] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described distributed data sharing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0154] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0155] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0157] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0158] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0159] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method of distributed data sharing, characterized by, include: Upon receiving a data change event, obtain the target synchronization strategy for the data change event; The target synchronization strategy includes a target operator chain; If the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy. In the event of a data change conflict, conflict handling is performed based on the target operator chain to generate a data change result; If the data change result indicates that the data change was successful, the data change result is sent to the next node.

2. The distributed data sharing method of claim 1, wherein, Upon receiving a data change event, the step of obtaining the target synchronization strategy for the data change event includes: Upon receiving a data change event, the target synchronization strategy corresponding to the data change event is searched in the synchronization strategy stored locally. If the search fails, a policy pull event is generated, and the target synchronization policy is pulled from the centralized policy center based on the policy pull event.

3. The distributed data sharing method of claim 1, wherein, Upon receiving a data change event, before obtaining the target synchronization strategy for the data change event, the method further includes: Obtain the target text input by the user; the target text represents the data synchronization requirement. Using a large language model, intent parsing is performed on the target text to generate a strategy draft; Based on the aforementioned strategy draft, a synchronization strategy is generated.

4. The distributed data sharing method of claim 3, wherein, The generation of a synchronization strategy based on the draft strategy includes: A second strategy verification is performed on the strategy draft; the second strategy verification includes at least one of syntax verification, reference verification, dependency verification, constraint verification, and deduction verification. If the second strategy passes the verification, semantic parsing is performed on the strategy draft; The semantic parsing result is compiled to generate a target operator chain, and the target operator chain and private key are encapsulated into a target synchronization strategy.

5. The distributed data sharing method of claim 1, wherein, Upon receiving a data change event, after obtaining the target synchronization strategy for the data change event, the method further includes: Perform a first policy verification on the target synchronization policy; the first policy verification includes at least one of policy version verification, key verification, and effective time verification.

6. The distributed data sharing method according to claim 1, characterized in that, If the target synchronization strategy passes the first strategy verification, change conflict detection is performed on the data change event based on the target synchronization strategy, including: If the target synchronization strategy passes the first strategy verification, the target data change record is obtained; the target data change record is a local uncommitted data change record or the last historical data change record; Based on the target data change record, perform change conflict detection on the data change event.

7. The distributed data sharing method according to claim 1, characterized in that, In the event of a data change conflict, conflict handling is performed based on the target operator chain to generate a data change result, including: If a data change event has a change conflict, determine the type of change conflict for the data change event; Based on the target operator chain, decision conflict processing is performed according to the change conflict type to generate data change results; the decision conflict processing includes at least one of data overwrite, data addition, data deletion and ignoring.

8. The distributed data sharing method according to claim 1, characterized in that, In the event of a data change conflict, after performing conflict handling based on the target synchronization strategy to generate the data change result, the method further includes: If the data change result indicates that the data change was unsuccessful, an arbitration event is generated based on the data change event and the data change result, and the arbitration event is written into the task processing queue.

9. A distributed data sharing device, characterized in that, include: The first acquisition module is used to acquire the target synchronization strategy of the data change event upon receiving a data change event. The target synchronization strategy includes a target operator chain; The detection module is used to perform change conflict detection on the data change event based on the target synchronization strategy if the target synchronization strategy passes the first strategy verification. The processing module is used to perform conflict processing based on the target operator chain in the event of a data change conflict, so as to generate a data change result; The sending module is used to send the data change result to the next node when the data change result indicates that the data change was successful.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the distributed data sharing method according to any one of claims 1-8.

Images