Strong-consistency data component reading method and device and storage medium
By using a central node in the SCADA system to add write value requests to broadcast frames and using KCP protocol for communication, the problem of difficult data consistency in redundant deployment is solved, and high consistency between data components and system stability is achieved. sex.
Patent Information
- Application Number
- CN202411134104.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-12-10
AI Technical Summary
In SCADA systems, under redundant deployment, the consistency of the independent data of memory variables and the data of write value data is difficult to ensure. Traditional methods rely on active periodic synchronization. Improper synchronization frequency setting will lead to inconsistent data or occupy system resources.
By using the central node to add the write value request as synchronization information in a redundant system or a cluster system, the data component updates the local data after receiving the broadcast frame and then processes the read value request, ensuring that all data components receive the latest data changes. Use KCP protocol to communicate, update the location of the central node in real time, and achieve data consistency.
The high consistency between independent nodes and independent data due to change is achieved, the system stability and reliability are improved, and the gap data is difficult to ensure and bandwidth occupancy in traditional methods is avoided.
Smart Images
Figure CN119105884A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data consistency, and in particular to a method, device and storage medium for reading a strongly consistent data component. Background Art
[0002] In SCADA systems, memory variables play a key role. They can change autonomously and accept write operations from the monitoring end to support secondary calculations and status records of data. In order to improve the stability and reliability of the system, redundant deployment has become a common strategy. However, under this redundant deployment mode, the data consistency of memory variables has become a challenge, especially the data consistency of self-changing data and write data. For write data, the traditional method is to ensure data consistency by sending write requests to two redundant data components at the same time. For self-changing data, periodic data synchronization is usually adopted between redundant systems, but the consistency of data cannot be guaranteed during the synchronization interval. When the data volume is large, this data synchronization consumes a lot of bandwidth. In traditional redundant deployment environments, data consistency is usually guaranteed by active periodic synchronization mechanism. Specifically, this mechanism requires the master node to perform data proofreading and synchronization operations with the slave node regularly. However, if the synchronization frequency is set too low, the risk of data inconsistency will increase significantly, because data may change multiple times in a long synchronization interval, resulting in data differences between the master and slave nodes; while increasing the synchronization frequency to reduce the risk of data inconsistency will excessively occupy system resources.
[0003] For example, a method and device for implementing strong consistency read operations in a distributed system disclosed in a Chinese patent document, with publication number: CN107844491B, discloses that based on a read request for a target data object, it is queried whether an operation log corresponding to the target data object exists; if the operation log exists, the operation log is obtained; and a response result to the read request is fed back, wherein the response result is determined by the operation log. However, this solution is difficult to ensure the consistency of the changed data in each independent node. Summary of the invention
[0004] In order to solve the problem in the prior art that it is difficult to maintain consistency in data of each independent node, the present invention provides a method, device and storage medium for reading strongly consistent data components, which can ensure that self-changing or changed data maintains a high degree of consistency among each independent node in a redundant system or cluster system.
[0005] In order to achieve the above object, the present invention provides the following technical solutions: A method for reading a strongly consistent data component comprises: the data component stores an upper layer application request in a message queue and processes the request when a broadcast frame is received; The data component sends the write value request as synchronization information to the central node. When the central node receives the synchronization information, it adds the synchronization information to the next broadcast frame. When a data component receives a broadcast frame, it first updates the local data according to the broadcast frame, and then processes the read value request of the message queue. This ensures that all data components can receive the latest data changes. When a data component receives a broadcast frame, it first updates the local data and then processes the read value request. This order ensures that the read operation is always based on the latest data status, thereby achieving strong consistency; even if a data component is temporarily unable to receive or process a broadcast frame, the central node will continue to retain the synchronization information and send it again during the next broadcast. Avoid the problem of intermittent data being difficult to guarantee and occupying bandwidth in traditional methods.
[0006] Preferably, when the central node receives multiple write value requests, it sorts the write value requests in the order in which they are received and adds them to the next broadcast frame in sequence, ensuring the order of data updates and simplifying the logic of data processing, ensuring that all data components can receive these updates in the same order, thereby maintaining data consistency.
[0007] Preferably, when any data component receives a write value request and a read value request, the data component directly processes the read value request and synchronizes the write value request with any other data component through the central node. Although the read operation is performed locally, the write operation is synchronized through the central node, ensuring that all data components can eventually receive the data change. When a data component fails, the central node can still continue to accept and process the write value request and synchronize the data through other healthy data components, ensuring the stability and reliability of the system.
[0008] Preferably, when the data component receives only a read value request, the data component directly processes the read value request and processes the next request after receiving the next broadcast frame, so that the read value request can be processed and returned immediately, so that the data component will not block the processing of other requests due to waiting for the broadcast frame, and the data synchronization is completed before the request is processed after the next broadcast frame arrives, so that each request will obtain updated data.
[0009] Preferably, when the data component receives the broadcast frame and synchronization information at the same time, the data component first updates the local data according to the synchronization information, and then executes the read and write requests in its own message queue according to the broadcast frame. This ensures that the local data of the data component is up to date before processing the read and write requests.
[0010] Preferably, the data components communicate with each other through the KCP protocol, and several data components are grouped together, and the central node position in each group of data components is updated in real time. This can ensure data consistency during operation to the greatest extent, so that the entire system can remain efficient and stable when processing large amounts of data and high concurrent requests.
[0011] As a preferred method, the method includes comparing the IDs of all data components, arranging a central node at the data component with the largest ID; the central node broadcasts the operation command and independent variable seeds to other data components, and realizes adaptive adjustment of the position of the central node according to the operation status, and realizes adaptive data synchronization.
[0012] A device, applicable to a strongly consistent data component reading method as described in any one of the embodiments, comprising: a plurality of interconnected data components, any data component being connected to respective upper-layer business applications; A central node connected to a data component, the central node is located in one of the data components; the central node sends a broadcast frame to other data components. Data synchronization and request processing are achieved through broadcast frames, and complete synchronization of running frames is achieved, thereby ensuring data consistency.
[0013] Preferably, the data component updates the independent variable after receiving the broadcast frame, and processes the read and write requests of the upper-layer business application after updating the independent variable; the central node receives the instruction synchronization of any data component, and synchronizes the instruction of any data component to other arbitrary data components through the central node.
[0014] A storage medium stores a computer program, which, when executed, implements a method for reading a strongly consistent data component as described in any one of the embodiments.
[0015] The present invention has the following advantages: (1) Ensure that self-changing data and dependent-changing data maintain high consistency between independent nodes, which can improve the overall stability of the system and enhance the accuracy and reliability of data processing; (2) Ensure that all data components can receive the latest data changes. When a data component receives a broadcast frame, it first updates the local data and then processes the read value request. This order ensures that the read operation is always based on the latest data status, thereby achieving strong consistency; even if a data component is temporarily unable to receive or process a broadcast frame, the central node will continue to retain the synchronization information and send it again during the next broadcast. Avoid the problem of intermittent data being difficult to guarantee and occupying bandwidth in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The drawings described below are merely exemplary. A person skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.
[0017] Figure 1 is a logical diagram of the method in the embodiment.
[0018] Figure 2 It is a redundancy principle block diagram in the embodiment.
[0019] Figure 3 is a timing diagram of independent variable synchronization in the embodiment.
[0020] Figure 4 is a dependent variable synchronization timing diagram in the embodiment.
[0021] Figure 5 It is a logical diagram of data consistency of the first case in the embodiment.
[0022] Figure 6 It is a logical diagram of data consistency of the second case in the embodiment.
[0023] In the figure: C-central node; A-first data component; B-second data component; a-first business application; b-second business application; 6-upper-layer business application. DETAILED DESCRIPTION
[0024] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.
[0025] like Figure 1 As shown, in a preferred embodiment, the present invention discloses a method for reading a data component with strong consistency, including: the data component stores the upper layer application request in the message queue, and each data component returns the request of its own upper layer application. When a broadcast frame is received, it is processed; the return operation of each data component is synchronized according to the broadcast frame to ensure the consistency of the data.
[0026] The data component sends the write value request as synchronization information to the central node. When any data component writes data and updates the data, the data component synchronizes the write value request through the central node and performs redundant data synchronization on other data nodes. When the central node receives the synchronization information, the central node acts as a transfer station for the synchronization information and can broadcast the synchronization information of any data component with the broadcast frame as the node. The synchronization information is added to the next broadcast frame to ensure that all data components can receive the latest data changes.
[0027] When a data component receives a broadcast frame, it first updates the local data according to the broadcast frame, and then processes the read value request of the message queue. This order ensures that the read operation is always based on the latest data status, thereby achieving strong consistency; even if a data component is temporarily unable to receive or process a broadcast frame, the central node will continue to retain the synchronization information and send it again during the next broadcast. This avoids the problem of intermittent data being difficult to guarantee and occupying bandwidth in traditional methods.
[0028] In other embodiments, when the central node receives multiple write value requests in this solution, it means that multiple data components have updated data at this time, and the updated content of each data component needs to be synchronized with other data components. The write value requests are sorted according to the order in which they are received and added to the next broadcast frame in sequence. The order of data updates is ensured, and the logic of data processing is simplified to ensure that all data components can receive these updates in the same order, thereby maintaining data consistency.
[0029] When in use, read-write separation is achieved, that is, read operations are performed directly on local data components, while write operations are synchronized through central nodes. This design reduces the processing pressure of central nodes because read operations are usually more frequent than write operations, and read operations do not require global synchronization. At the same time, since read operations are performed directly locally, requests from upper-level applications can be responded to more quickly, improving the overall performance of the system. For read operations, since they are processed directly locally, there is no need to transmit data over the network, reducing network bandwidth consumption and latency.
[0030] The synchronization of write value requests through the central node facilitates flexible control of data synchronization strategies, including adjusting the timing and method of synchronization according to the update frequency and importance of the data.
[0031] In other embodiments, when the data component in this scheme receives a write value request and a read value request, it is necessary to ensure that the data returned by the read value request is consistent, and regardless of the content of the write value request, it is necessary to synchronize according to the write value request. At this time, the data component directly processes the read value request to ensure the immediate return of the read value request. The write value request is synchronized with any other data component through the central node. Although the read operation is performed locally, the write operation is synchronized through the central node to ensure that all data components can eventually receive the data change. When a data component fails, the central node can still continue to accept and process the write value request, and synchronize the data through other healthy data components, ensuring the stability and reliability of the system.
[0032] In other embodiments, the data component in this scheme does not need to wait for the broadcast frame when only receiving the read value request, and can respond immediately. The data component directly processes the read value request, and the waiting time of the request is greatly shortened, so that the read value request can be processed and returned immediately. The next request is processed only after the next broadcast frame is received. The data component does not block the processing of other requests due to waiting for the broadcast frame, and completes data synchronization before request processing after the next broadcast frame arrives, so that each request will obtain updated data.
[0033] In other embodiments, when a data component receives a broadcast frame and synchronization information at the same time, the data component needs to continue to execute the request in the message queue and update it according to the synchronization information. The data component first updates the local data according to the synchronization information, which ensures that the local data of the data component is the latest before processing the read and write request. This ensures data consistency when multiple data components need to share and synchronize data in a distributed system. Then, the read and write requests in its own message queue are executed according to the broadcast frame. Since the local data is already the latest at this time, the read and write requests can be processed based on accurate data, reducing errors and retries caused by inconsistent data. Although the read and write requests are not processed until the broadcast frame arrives, the system can still support a certain degree of real-time performance due to the immediate update of the synchronization information. Since the read and write requests are processed after the local data is updated, their sensitivity to external network delays is reduced. Even if there is a delay in the transmission of the broadcast frame, it will not directly affect the processing results of the read and write requests. In other embodiments, the data components described in this solution communicate with each other through the KCP protocol, which can guarantee the data consistency during operation to the greatest extent, so that the entire system can remain efficient and stable when processing large amounts of data and high concurrent requests. Several data components are formed into a group, and several data components form a distributed system. The position of the central node in each group of data components is updated in real time. The position of the central node is adaptively adjusted according to the operating status of several data components. Ensure that each broadcast frame can be sent at the current optimal data component.
[0034] In other embodiments, the scheme includes comparing the IDs of all data components, and whenever different data components are enabled, several new data components can be screened. The central node is arranged at the data component with the largest ID; so that the central node is always at the largest data node, ensuring the best synchronization efficiency among the current several data components. The central node broadcasts the operation command and the independent variable seed to other data components. The position of the central node is adaptively adjusted according to the operation status, and adaptive data synchronization is achieved.
[0035] In another embodiment, the present solution discloses a device, which is located in a distributed system and is applicable to a strongly consistent data component reading method as described in any embodiment, including: a plurality of interconnected data components, and different data components independently process read value requests and write value requests. Any data component is connected to its own upper-layer business application; each upper-layer business application sends its own read value request and write value request to its own connected data component. The data components and upper-layer businesses are arranged in a distributed manner.
[0036] A central node connected to a data component, the central node broadcasts in several data components. The central node is located in one of the data components; when a different data component is enabled, the central node is re-determined in several data components. The central node sends a broadcast frame to other data components. Data synchronization and request processing are achieved through broadcast frames, and complete synchronization of running frames is achieved, thereby ensuring data consistency.
[0037] In other embodiments, the data component described in this solution updates the independent variable after receiving the broadcast frame to achieve unified update of local data. After updating the independent variable, the read and write request of the upper-layer business application is processed; so that each upper-layer business application can obtain the request return according to the latest local data after synchronization. The central node receives the instruction synchronization of any data component. It is realized that the instruction of any data component is synchronized to other arbitrary data components through the central node.
[0038] In other embodiments, the present solution discloses a device suitable for a strongly consistent data component reading method as described in any one of the embodiments, including the following structure: several data components, each of which is connected to an upper-layer application; a central node is provided in the data component, and the central node obtains a write value request from at least one other data component, and the central node sends a broadcast frame to all other data components.
[0039] In another embodiment, the present solution discloses a strongly consistent data service component, wherein several of the data service components form a group, each group of data service components is provided with a central node, and the central node is automatically switched by the Bully algorithm. Several of the data service components communicate with each other through the KCP protocol, which effectively reduces the occupancy of the network bandwidth and improves the efficiency and reliability of data transmission. It not only solves the problem of data asynchrony in the traditional master-slave data synchronization method, but also ensures data consistency in cluster data synchronization. By automatically switching the central node and efficient inter-node communication, it can ensure that the data of the memory variables in the SCADA system remain highly consistent under redundant deployment, thereby improving the stability and reliability of the entire system.
[0040] like Figure 2 As shown, in another embodiment, the present solution discloses a redundancy principle, including the following process. After the upper-layer business application 6 connects to any data component, it sends instructions including at least one instruction of writing and reading values to it. The data component receiving the instruction is synchronized according to the central node C, and synchronized with other data components through frame instructions. Thus, the consistency of redundant data is achieved.
[0041] like Figure 3 As shown, in another embodiment, the present solution discloses a method for synchronizing an independent variable. When synchronizing an independent variable, the method includes the following steps: first, a frame sequence and an independent variable seed are generated at a central node C, and broadcasted through a broadcast frame. After receiving the broadcast frame, other data components first update the independent variable, and then process the read and write requests of their own upper layer applications. At this time, the other data components include any other at least one or more data components.
[0042] like Figure 4 As shown, in another embodiment, the present scheme discloses a method for synchronization of dependent variables, and when performing synchronization of dependent variables, the method includes the following steps. When any data component receives a broadcast frame, it first updates the independent variable according to the broadcast frame, then processes the read and write requests of its own upper-layer application, and synchronizes the write value instructions therein to the central node C. The central node generates a frame sequence and an independent variable seed according to the synchronized write value instruction, and generates a packing instruction, which is broadcast through a broadcast frame, and the packing instruction is broadcast together with the broadcast frame. After receiving the broadcast frame, other data components update local data according to the packing instruction in the broadcast frame, update the independent variable again according to the broadcast frame, and then process the read and write requests of its own upper-layer application. At this time, after the broadcast frame and the packing instruction are received by several other data components, the other several data components include data components synchronized with the write value instruction and data components not synchronized with the write value instruction.
[0043] In other embodiments, in a traditional redundant architecture, each node runs independently and has a different processing frequency. When two redundant components receive write value instructions from an upper-layer application at the same time, the write value data cannot be kept completely consistent within one end time due to the difference in processing speed. In order to solve this problem, the redundant system needs to synchronize data regularly. During the data synchronization process, in addition to comparing all independent and dependent variable data, the time needs to be calibrated. Although increasing the synchronization frequency can reduce the proportion of inconsistent data time, this method will also consume a large amount of traditional resources, making it difficult to completely solve the data consistency problem no matter how the synchronization strategy is adjusted.
[0044] In this solution, a redundant structure is disclosed, which includes a central node and multiple data nodes. The central node is elected from the data nodes through the Bully algorithm, and the central node is adaptively adjusted according to the running state. The central node promotes the running frame, broadcasts the running command and the independent variable seed. When the other data nodes receive the running frame promotion instruction, they will give priority to the update of the dependent variable and the independent variable, and then execute other various instructions from the business application. The running frame can be fully synchronized to ensure the consistency of the data. The data nodes are connected through the KCP protocol, which further improves the speed and efficiency of the frame synchronization. The KCP protocol can guarantee the data consistency during the operation process to the greatest extent, so that the entire system can still maintain high efficiency and stability when processing large amounts of data and high concurrent requests. Through the special redundant structure, the data consistency problem in the traditional redundant architecture is solved, and the stable operation of the business application is guaranteed.
[0045] like Figure 5 As shown, in another specific embodiment, the scheme includes the following steps. At this time, the initial data is off, the first business application a initiates a write on value request to the first data component A, and then initiates a read value request, and the second business application b initiates a read value request to the second data component B. In this case, the first data component and the second data component will first store the requests from the upper-layer business application in the message queue after receiving them respectively, and wait until the central node C sends a broadcast frame, and the first data component and the second data component both receive the broadcast frame sent by the central node before processing the corresponding request. For the write value request received by the first data component, the first data component will first synchronize to the central node. After receiving the synchronized write value request, the central node broadcasts the write value request through the central node in the next broadcast frame, and for the read value request received by the data component A, the current actual value off is directly read. For the read value request received by the second data component, the second data component directly reads the current actual value off for return.
[0046] At this time, the data read by the upper-layer business application of the first data component and the second data component are consistent. When the next broadcast frame of the central node arrives, the first data component and the second data component first change the data in the memory from off to on according to the write value request in the broadcast frame, and then process the upper-layer business application request in the message queue respectively. At this time, when the first data component and the second data component perform data processing again, data synchronization must have been completed to ensure data consistency.
[0047] like Figure 6As shown, in the second specific embodiment, this scheme includes the following steps. At this time, the initial data is off, and the first business application a initiates a write on value request to the first data component A, and then initiates a read value request. The second business application b initiates a write off value request to the second data component B, and then initiates a read value request. At this time, the first data component and the second data component can still ensure the consistency of data reading, but it is difficult to ensure whether the final read data is on or off. Therefore, in this scheme, for the write value request received by the first data component, it will be synchronized to the central node C first, and the central node will broadcast the write value request received by the first data component in the next broadcast frame, and the first data component will directly read the current actual value off as a return for the received read value request. For the write value request received by the second data component, the second data component will also synchronize its write value request to the central node. For the read value request of the second data component, the second data component directly reads the current actual value off as a return after receiving the read value request. The central node accepts the write value instructions of the first data component and the second data component respectively. When the central node receives the write value instructions from the two data components, it attaches a write value request in the next broadcast packet in the order in which the write value packets are received. After the first data component or the second data component receives the broadcast frame from the central node, it updates the local data according to the write value requests sorted in the broadcast packet, and then processes the read value of the upper layer application in the message queue, so that the redundant data still maintains consistency.
[0048] When the present invention is in use, the write value requests of multiple data components are sorted and broadcasted through the central node, so that the redundant data of other data nodes can be kept consistent at any time.
[0049] In other embodiments, the present solution discloses a storage medium having a computer program stored thereon, and when the computer program is executed, a method for reading a strongly consistent data component as described in any one of the embodiments is implemented.
[0050] These modifications or improvements made on the basis of the present invention without departing from the spirit of the present invention all fall within the scope of protection required by the present invention.
Claims
1. A method for reading a strongly consistent data component, characterized in that: include: The data component stores the upper-layer application request in the message queue and processes it when a broadcast frame is received; The data component sends the write value request as synchronization information to the central node. When the central node receives the synchronization information, it adds the synchronization information to the next broadcast frame. When the data component receives a broadcast frame, it first updates the local data according to the broadcast frame, and then processes the read value request of the message queue.
2. A method for reading a strongly consistent data component according to claim 1, characterized in that: When the central node receives multiple write value requests, it sorts the write value requests according to the order in which they are received and adds them to the next broadcast frame in sequence.
3. A method for reading a strongly consistent data component according to claim 1 or 2, characterized in that: When any data component receives a write value request and a read value request, the data component directly processes the read value request and synchronizes the write value request with any other data component through the central node.
4. A method for reading a strongly consistent data component according to claim 1 or 2, characterized in that: When the data component receives only a read value request, the data component directly processes the read value request and processes the next request after receiving the next broadcast frame.
5. A method for reading a strongly consistent data component according to claim 4, characterized in that: When a data component receives a broadcast frame and synchronization information at the same time, the data component first updates the local data according to the synchronization information, and then executes the read and write requests in its own message queue according to the broadcast frame.
6. A method for reading a strongly consistent data component according to claim 1, characterized in that: The data components communicate with each other through the KCP protocol, and several data components are formed into a group. The position of the central node in each group of data components is updated in real time.
7. A method for reading a strongly consistent data component according to claim 6, characterized in that: The method includes comparing the IDs of all data components and placing a central node at the data component with the largest ID; the central node broadcasts the running command and independent variable seeds to other data components.
8. A device, applicable to a method for reading a strongly consistent data component as claimed in any one of claims 1 to 7, characterized in that: include: Several interconnected data components, each of which is connected to its own upper-layer business application; A central node connected to the data components, wherein the central node is located in one of the data components; the central node sends a broadcast frame to other data components.
9. A device according to claim 8, characterized in that: The data component updates the independent variable after receiving the broadcast frame, and processes the read and write request of the upper-layer business application after updating the independent variable; the central node receives the instruction synchronization of any data component.
10. A storage medium storing a computer program, characterized in that: When the computer program is executed, the method for reading a strongly consistent data component as described in any one of claims 1 to 7 is implemented.