A data partitioning and synchronization method, system, device and storage medium based on domain model

Abstract

The present invention discloses a data partitioning and synchronization method, system, device and storage medium based on a domain model, which belongs to the field of automation system / software design technology. The method comprises: creating domain units, data partitioning units and data points in the domain model, the domain units comprising a station domain and a control center domain, acquiring background data and change data and dividing them into various data points, dividing the data points into various domain units according to the data partitioning units, the data points comprising machine nodes and service nodes; establishing a communication connection between the station domain and the control center domain, performing bidirectional background data synchronization between the station domain and the control center domain based on the communication connection, performing inter-domain change data synchronization between service nodes in the station domain and the control center domain; performing intra-domain background data synchronization between the station domain and the control center domain; performing intra-domain change data synchronization between machine nodes in the station domain and the control center domain; the present invention clearly divides the data, fits the actual situation and divides accurately.

Description

Technical Field

[0001] The present invention relates to a data partitioning and synchronization method, system, device and storage medium based on a domain model, and belongs to the technical field of automation system / software design. Background Art

[0002] The Integrated Supervisory Control System for Rail Transit is a common industrial-grade distributed automation system. As a monitoring platform, it is responsible for coordinating the data of various business subsystems into an organic whole, including but not limited to fire alarm, automatic train monitoring, environment and equipment monitoring, etc.

[0003] The data division and synchronization method of the rail transit integrated monitoring system essentially includes the definition and division of data sets, the division of communication IPs, the division of corresponding functions and services, the design of communication processes, the design of data flows, and the mechanism to ensure data consistency of distributed nodes. With the continuous expansion of various business subsystems and the continuous upgrading of subsystem data scale, how to effectively divide data and ensure the consistency of real-time data has become a difficult problem and focus of the rail transit integrated monitoring system.

[0004] At present, some existing technologies use domains as units to divide the distributed real-time system architecture of the rail transit comprehensive monitoring system, and use a real-time data bus based on message middleware to complete the communication services between domains and nodes within the system, but do not involve the description of the specific network architecture, and do not involve methods, processes, mechanisms and means to ensure the reliability and consistency of distributed node data transmission; some existing technologies divide the system into different data domains according to the center and the station, but do not explain the data division method and process. When the application host of the station group is connected to the center group, it is still unknown how to ensure the integration of new background data and the consistency of distributed domain data. Summary of the invention

[0005] The purpose of the present invention is to provide a data partitioning and synchronization method, system, device and storage medium based on a domain model to solve the problems of unclear data partitioning and inability to ensure data synchronization consistency in the prior art.

[0006] To achieve the above objectives, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a data partitioning and synchronization method based on a domain model, comprising:

[0008] Create domain units, data partition units and data points in the domain model, the domain units include station domains and control center domains, obtain background data and change data and divide them into data points, divide the data points into domain units according to the data partition units, the data points include machine nodes and service nodes;

[0009] Establish a communication connection between the station domain and the control center domain, perform two-way background data synchronization between the station domain and the control center domain based on the established communication connection, and perform inter-domain change data synchronization between the service nodes of the station domain and the control center domain;

[0010] Perform intra-domain background data synchronization in the station domain and control center domain respectively;

[0011] Synchronize the intra-domain change data between the machine nodes in the station domain and the control center domain.

[0012] In combination with the first aspect, further, the domain unit, data partition unit and data point are created through a unified configuration tool.

[0013] In combination with the first aspect, further, the communication connection is a TCP connection.

[0014] In combination with the first aspect, further, the bidirectional background data synchronization between the station domain and the control center domain includes:

[0015] First, set up the background data synchronization source partitions for the station domain and the control center domain respectively, synchronize the background data of the background data synchronization source partition in the station domain to the control center domain and then cache it, synchronize the background data of the background data synchronization source partition in the control center domain to the station domain and then cache it, and the two background data synchronization steps are performed simultaneously.

[0016] In combination with the first aspect, further, the intra-domain background data synchronization is performed in the station domain and the control center domain respectively, including:

[0017] In the station domain, the service node of the station domain synchronizes the background data of the specified partition to the machine nodes in the station domain;

[0018] In the control center domain, the service node of the control center domain synchronizes the background data of the specified partition to the machine nodes in the control center domain.

[0019] In combination with the first aspect, further, synchronizing inter-domain change data between service nodes in the station domain and the control center domain includes:

[0020] The service nodes in the control center domain send the change data of the real-time library module to all the service nodes in the station domains that are connected to it for communication, and the service nodes in the station domain send the change data of the real-time library module to the service nodes in the control center domains that are connected to it for communication.

[0021] In combination with the first aspect, further, synchronizing the intra-domain change data between the machine nodes in the station domain and the control center domain respectively includes:

[0022] The change data received by the station domain during the inter-domain change data synchronization phase is synchronized to all machine nodes in the station domain, and the change data received by the control center domain during the inter-domain change data synchronization phase is synchronized to all machine nodes in the control center domain.

[0023] In a second aspect, the present invention further provides a data partitioning and synchronization system based on a domain model, comprising:

[0024] Data partitioning module: used to create domain units, data partitioning units and data points in the domain model. The domain units include station domains and control center domains. Background data and change data are obtained and divided into data points. Data points are divided into domain units according to data partitioning units. The data points include machine nodes and service nodes.

[0025] Inter-domain data synchronization module: used to establish a communication connection between the station domain and the control center domain, perform two-way background data synchronization between the station domain and the control center domain based on the established communication connection, and perform inter-domain change data synchronization between the service nodes of the station domain and the control center domain;

[0026] Intra-domain background data synchronization module: used to synchronize intra-domain background data in the station domain and control center domain respectively;

[0027] Intra-domain change data synchronization module: used to synchronize intra-domain change data between machine nodes within the station domain and the control center domain.

[0028] In a third aspect, the present invention also provides a data partitioning and synchronization device based on a domain model, including a processor and a storage medium;

[0029] The storage medium is used to store instructions;

[0030] The processor is used to operate according to the instructions to execute the steps of the method according to any one of the first aspects.

[0031] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0032] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0033] The data partitioning and synchronization method, system, device and storage medium based on domain model provided by the present invention clearly divide the data of rail transit integrated monitoring system, take data partition unit as the minimum set unit, divide the data partition unit contained in the domain, which is highly consistent with the division of rail transit system control center and station, fits the actual situation, and divides the data accurately; the station domain and the station domain communicate independently and are deployed in different IP network segments, and the control center domain contains all the information of the monitoring system, which can play the role of whole network perception and unified management;

[0034] By flexibly setting the background data synchronization source partitions in the station domain and the control center domain, the problem of background data flow when the station domain is connected to the control center domain is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is one of the flow charts of a data partitioning and synchronization method based on a domain model provided by an embodiment of the present invention;

[0036] Figure 2 is a server / workstation module deployment diagram provided by an embodiment of the present invention;

[0037] Figure 3 This is a diagram of a data synchronization framework within a domain provided by an embodiment of the present invention;

[0038] Figure 4 is a diagram of an inter-domain data synchronization framework provided by an embodiment of the present invention;

[0039] Figure 5 This is the second flowchart of a data partitioning and synchronization method based on a domain model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.

[0041] Example 1

[0042] like Figure 1 As shown, a data partitioning and synchronization method based on a domain model provided by an embodiment of the present invention includes the following steps:

[0043] S1. Create domain units, data partition units and data points in the domain model. The domain units include station domains and control center domains. Divide background data and change data into data points. Divide data points into domain units according to data partition units. The data points include machine nodes and service nodes.

[0044] Use the unified configuration tool to create domain units, data partition units, and data points in the pre-built domain model. The domain units include the station domain and the control center domain. The data points include machine nodes and service nodes. Then, divide the background data and change data into each data point. Divide the data points into each domain unit according to the data partition unit, including dividing the machine nodes into their corresponding domain units. Load the data partition unit in each domain unit.

[0045] S2. Establish a communication connection between the station domain and the control center domain, perform two-way background data synchronization between the station domain and the control center domain based on the established communication connection, and perform inter-domain change data synchronization between service nodes in the station domain and the control center domain.

[0046] The state service module elects the master node of the inter-domain synchronization module of the station domain, and the master node establishes a communication connection with the inter-domain synchronization module of the control center domain. The communication connection is a TCP connection. If the master node exits abnormally, the state service module re-elects a new master node and re-establishes a communication connection with the control center domain.

[0047] Based on the above communication connection, first set the background data synchronization source partitions of the station domain and the control center domain respectively, synchronize the background data of the background data synchronization source partition in the station domain to the control center domain and then cache it, synchronize the background data of the background data synchronization source partition in the control center domain to the station domain and then cache it, and the two background data synchronization steps are performed simultaneously.

[0048] The service nodes in the control center domain send the change data of the real-time library module to all the service nodes in the station domains that are connected to it for communication, and the service nodes in the station domain send the change data of the real-time library module to the service nodes in the control center domains that are connected to it for communication.

[0049] S3. Synchronize the intra-domain background data in the station domain and the control center domain respectively.

[0050] In the station domain, the service node of the station domain synchronizes the background data of the specified partition to the machine nodes in the station domain;

[0051] In the control center domain, the service node of the control center domain synchronizes the background data of the specified partition to the machine nodes in the control center domain.

[0052] S4. Synchronize the intra-domain change data between the machine nodes in the station domain and the control center domain.

[0053] The change data received by the station domain during the inter-domain change data synchronization phase is synchronized to all machine nodes in the station domain, and the change data received by the control center domain during the inter-domain change data synchronization phase is synchronized to all machine nodes in the control center domain.

[0054] Through the above steps S1 to S4, the data is clearly divided, with the data partition unit as the minimum set unit, and the data partition unit contained in the domain division is highly consistent with the division of the rail transit system control center and station, in line with the actual situation, and the data division is accurate; and by flexibly setting the background data synchronization source partition in the station domain and the control center domain, the problem of the flow of background data when the station domain is connected to the control center domain is solved; the station domain and the station domain communicate independently and are deployed in different IP network segments. The control center domain contains all the information of the monitoring system, which can play a role in full network perception and unified management.

[0055] Example 2

[0056] An embodiment of the present invention provides a data partitioning and synchronization system based on a domain model for implementing the method described in Example 1, including: an inter-domain synchronization module, an intra-domain background synchronization module, an intra-domain change synchronization module, a real-time library module, a unified configuration tool and a status service module.

[0057] The functional modules described in this embodiment are respectively deployed on the server and the workstation as follows: Figure 2 As shown, a real-time library module, an intra-domain background synchronization module, an intra-domain change synchronization module, a status service module, a unified configuration tool and an inter-domain synchronization module are deployed on the server, and a real-time library module, an intra-domain background synchronization module, an intra-domain change synchronization module, a status service module and a unified configuration tool are deployed on the workstation; the inter-domain synchronization module is deployed in the station domain and is connected to the inter-domain synchronization module of the control center domain for communication, the intra-domain background synchronization module is connected to the server by the workstation in the control center domain and the station domain, and the intra-domain change synchronization module is connected to the control center domain and the station domain in their respective multicast network segments.

[0058] This embodiment proposes to use the concept of domain model to divide the data of the hierarchical distributed rail transit integrated monitoring system, and divide the integrated monitoring system into domain units, which is not only the division of data sets, but also the division of communication clusters and service domains; specifically, domain units, data partition units and data points are created through a unified configuration tool. The domain units include station domains and control center domains, and the data points include machine nodes and service nodes. Then the background data and change data are divided into each data point.

[0059] The real-time library module divides the data points into domain units according to the data partition unit (with the data partition unit as the smallest basic unit), including dividing the machine nodes into the corresponding domain units and loading the data partition unit in each domain unit; it also includes dividing the multicast addresses within different domain units and dividing the data partition units within different domain units.

[0060] The multicast addresses deployed in each domain unit are not in the same network segment. The multicast data between different domain units is filtered after being divided by the switch VLAN. The intra-domain change synchronization module cannot receive the multicast data of other domain units.

[0061] The real-time library module is used to provide the real-time data set belonging to the local computer of the monitoring system; the real-time data set is provided to other processes belonging to the local computer of the monitoring system for reading, writing and checking in the form of shared memory; other processes include but are not limited to: inter-domain synchronization module, intra-domain background synchronization module, and intra-domain change synchronization module.

[0062] Through the configuration of the unified configuration tool, different station domains load different data partition units, and the control center domain loads all data partition units; different station domains are isolated from each other in communication and need to be connected through the control center domain.

[0063] The state service module elects the master node of the inter-domain synchronization module of the station domain, and the master node establishes a communication connection with the inter-domain synchronization module of the control center domain. The communication connection is a TCP connection. If the master node exits abnormally, the state service module re-elects a new master node and re-establishes a communication connection with the control center domain.

[0064] This embodiment uses an inter-domain synchronization module to achieve data synchronization between the control center domain and the station domain, and between the station domain and the station domain. The inter-domain data synchronization framework is as follows: Figure 4 As shown, the inter-domain synchronization module of the station domain is deployed on the master node, and the master node elected by the status service module establishes a communication connection with the inter-domain synchronization module of the control center domain to synchronize the changed data of the control center domain to all station domains, and synchronize the changed data of the station domain to the control center domain, which is forwarded by the control center domain to other station domains, thereby realizing data synchronization between station domains.

[0065] The inter-domain synchronization module of the control center domain synchronizes the background data of the background synchronization data source partition of the control center domain to the station domain; at the same time, the inter-domain synchronization module of the station domain synchronizes the background data of the background synchronization data source partition of the station domain to the central domain.

[0066] During the background data synchronization from the control center domain to the station domain, the change data of the real-time library module of the control center domain server is synchronized to the station domain server and cached. After the background data synchronization is completed, the cache is written to the real-time library module of the station domain server; During the background data synchronization from the station domain to the control center domain, the change data of the real-time library module of the station domain is synchronized to the control center domain server and cached. After the background data synchronization is completed, the cache is written to the real-time library module of the control center domain server.

[0067] The intra-domain data synchronization framework between the control center domain and the station domain is as follows: Figure 3As shown, the state service module elects the master node of the server's intra-domain background synchronization module to provide the end-face data source (background data and change data to be synchronized) of the real-time library module in this domain; in the station domain, the service node of the intra-domain background synchronization module in the station domain (the elected master node) synchronizes the background data of the specified partition to the machine nodes in the station domain; in the control center domain, the service node of the intra-domain background synchronization module in the control center domain (the elected master node) synchronizes the background data of the specified partition to the machine nodes in the control center domain.

[0068] Through the intra-domain background synchronization module in each domain unit, the change data received by the station domain during the inter-domain change data synchronization phase is synchronized to all machine nodes within the station domain, and the change data received by the control center domain during the inter-domain change data synchronization phase is synchronized to all machine nodes within the control center domain.

[0069] The system provided in this embodiment can also be used to implement the following Figure 5 The method shown, such as Figure 5 As shown, the method includes:

[0070] The unified configuration tool divides the data partition unit and the domain unit, and specifies the background data synchronization partition list of the domain unit ( Figure 5 sync partition list in .

[0071] The real-time library module specifies the domain unit to which the local node belongs, and loads the data partition units to which each domain unit belongs.

[0072] The inter-domain synchronization module establishes a communication connection between the station domain and the control center domain, and performs two-way background data synchronization between the station domain and the control center domain ( Figure 5 bidirectional background synchronization in ) and change data synchronization.

[0073] The change synchronization is performed by the intra-domain change synchronization module. If the change data is generated by other nodes in the domain (nodes other than the service node), the change data will be synchronized to the service node. If the change data is received from the inter-domain synchronization module, the change data will be synchronized to other nodes in the domain (here refers to the machine node).

[0074] The background synchronization module in the domain executes background synchronization, and synchronizes the background data of the designated data partition obtained from the control center domain / station domain to other nodes in the domain (here refers to the machine node).

[0075] If the process exits abnormally, the state service module will re-elect the main process node and restart data division and synchronization.

[0076] Example 3

[0077] An embodiment of the present invention provides a data partitioning and synchronization system based on a domain model, including:

[0078] Data partitioning module: used to create domain units, data partitioning units and data points in the domain model. The domain units include station domains and control center domains. Background data and change data are obtained and divided into data points. Data points are divided into domain units according to data partitioning units. The data points include machine nodes and service nodes.

[0079] Inter-domain data synchronization module: used to establish a communication connection between the station domain and the control center domain, perform two-way background data synchronization between the station domain and the control center domain based on the established communication connection, and perform inter-domain change data synchronization between the service nodes of the station domain and the control center domain;

[0080] Intra-domain background data synchronization module: used to synchronize intra-domain background data in the station domain and control center domain respectively;

[0081] Intra-domain change data synchronization module: used to synchronize intra-domain change data between machine nodes within the station domain and the control center domain.

[0082] Example 4

[0083] An embodiment of the present invention provides a domain model-based data partitioning and synchronization device, including a processor and a storage medium;

[0084] The storage medium is used to store instructions;

[0085] The processor is configured to operate according to the instructions to execute the steps of the following method:

[0086] Create domain units, data partition units and data points in the domain model, the domain units include station domains and control center domains, obtain background data and change data and divide them into data points, divide the data points into domain units according to the data partition units, the data points include machine nodes and service nodes;

[0087] Establish a communication connection between the station domain and the control center domain, perform two-way background data synchronization between the station domain and the control center domain based on the established communication connection, and perform inter-domain change data synchronization between the service nodes of the station domain and the control center domain;

[0088] Perform intra-domain background data synchronization in the station domain and control center domain respectively;

[0089] Synchronize the intra-domain change data between the machine nodes in the station domain and the control center domain.

[0090] Example 5

[0091] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the following method are implemented:

[0092] Create domain units, data partition units and data points in the domain model, the domain units include station domains and control center domains, obtain background data and change data and divide them into data points, divide the data points into domain units according to the data partition units, the data points include machine nodes and service nodes;

[0093] Establish a communication connection between the station domain and the control center domain, perform two-way background data synchronization between the station domain and the control center domain based on the established communication connection, and perform inter-domain change data synchronization between the service nodes of the station domain and the control center domain;

[0094] Perform intra-domain background data synchronization in the station domain and control center domain respectively;

[0095] Synchronize the intra-domain change data between the machine nodes in the station domain and the control center domain.

[0096] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0097] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0100] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data partitioning and synchronization method based on a domain model, characterized in that: include: Create domain units, data partition units and data points in the domain model, the domain units include station domains and control center domains, obtain background data and change data and divide them into data points, divide the data points into domain units according to the data partition units, the data points include machine nodes and service nodes; Establish a communication connection between the station domain and the control center domain. Based on the established communication connection, first set the background data synchronization source partitions of the station domain and the control center domain respectively, synchronize the background data of the background data synchronization source partition in the station domain to the control center domain and cache it, synchronize the background data of the background data synchronization source partition in the control center domain to the station domain and cache it, the two background data synchronization steps are performed simultaneously, the service node in the control center domain sends the change data of the real-time library module to all the service nodes in the station domain that are connected to it for communication, and the service node in the station domain sends the change data of the real-time library module to the service node in the control center domain that is connected to it for communication; In the station domain, the service node of the station domain synchronizes the background data of the specified partition to the machine nodes in the station domain; in the control center domain, the service node of the control center domain synchronizes the background data of the specified partition to the machine nodes in the control center domain; The change data received by the station domain during the inter-domain change data synchronization phase is synchronized to all machine nodes in the station domain, and the change data received by the control center domain during the inter-domain change data synchronization phase is synchronized to all machine nodes in the control center domain.

2. A method for data partitioning and synchronization based on a domain model according to claim 1, characterized in that: The domain unit, data partition unit and data point are created through a unified configuration tool.

3. The method for data partitioning and synchronization based on a domain model according to claim 1, characterized in that: The communication connection is a TCP connection.

4. A data partitioning and synchronization system based on a domain model, characterized in that: include: The data partitioning module is used to: create domain units, data partitioning units and data points in the domain model, the domain units include station domains and control center domains, obtain background data and change data and divide them into data points, and divide data points into domain units according to the data partitioning units, the data points include machine nodes and service nodes; The inter-domain data synchronization module is used to: establish a communication connection between the station domain and the control center domain, based on the established communication connection, first set the background data synchronization source partitions of the station domain and the control center domain respectively, synchronize the background data of the background data synchronization source partition in the station domain to the control center domain and cache it, synchronize the background data of the background data synchronization source partition in the control center domain to the station domain and cache it, the two background data synchronization steps are performed simultaneously, the service node in the control center domain sends the change data of the real-time library module to all the service nodes in the station domain connected to it, and the service node in the station domain sends the change data of the real-time library module to the service node in the control center domain connected to it; The intra-domain background data synchronization module is used to: in the station domain, the service node of the station domain synchronizes the background data of the specified partition to the machine node in the station domain; in the control center domain, the service node of the control center domain synchronizes the background data of the specified partition to the machine node in the control center domain; The intra-domain change data synchronization module is used to: synchronize the change data received by the station domain during the inter-domain change data synchronization phase to all machine nodes within the station domain, and synchronize the change data received by the control center domain during the inter-domain change data synchronization phase to all machine nodes within the control center domain.

5. A data partitioning and synchronization device based on a domain model, characterized in that: including processors and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images