Micro-service data processing method and system

Abstract

The application provides a micro-service data processing method and system, which comprises the following steps: splitting a distributed control system into a plurality of micro-services including a point item configuration service, and standardizing and decoupling interfaces of each micro-service by standardizing data exchange between the micro-services and containing at least a value, a timestamp, a quality code and a version number. When a new function needs to be expanded, only the logic of the relevant micro-service needs to be adjusted, and the interfaces of a plurality of associated modules do not need to be synchronously modified, thereby reducing interface dependence and coupling degree between the modules and avoiding cross-module faults in the maintenance and upgrading processes. When configuration data is changed and the version number is updated, the point item configuration service notifies other micro-services to obtain the new version configuration data without restarting the service processes, so that configuration updating can be completed without restarting any service process during the configuration change process, the defect of control interruption caused by restarting the service in the traditional DCS is overcome, and the continuous operation ability of the system is ensured.

Description

Technical Field

[0001] This invention relates to the field of industrial control system (ICS) software design technology, specifically to a microservice data processing method and system. Background Technology

[0002] Industrial control systems deploy a large amount of software. Although traditional distributed architecture DCS software has split some functional modules, the interfaces between modules are complex and highly coupled. When expanding new functions, the interface logic of multiple related modules needs to be modified simultaneously, and cross-module failures are prone to occur during maintenance and upgrades. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a microservice data processing method and system to solve the problem of cross-module failures that are prone to occur during maintenance and upgrades.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: The first aspect of this invention discloses a microservice data processing method, the method comprising: The distributed control system is divided into multiple microservices, including point item configuration services. The data exchange between the microservices includes at least numerical values, timestamps, quality codes, and version numbers. When a change in configuration data is detected and the version number is updated, the point item configuration service notifies other microservices to obtain the new version of configuration data without restarting the service process.

[0005] Preferably, the multiple microservices are divided into: stateless computing microservices and stateful storage microservices; The stateless computational microservices include at least: point item configuration service, historical data service, and real-time data service; The stateful storage-type microservices include at least: a point item historical data storage cluster, a point item metadata storage cluster, a sampling service cluster, an IO service cluster, and a service registration module.

[0006] Preferably, when a change in configuration data is detected and the version number is updated, the point item configuration service notifies other microservices to obtain the new version of configuration data without restarting the service process, including: After the point item configuration service detects that the configuration data has changed and the version number has been updated, it pushes a configuration change notification to all other microservices that have subscribed to the configuration through the point item configuration service. This allows the other microservices to obtain the new version configuration data and update the new version configuration data to their local memory mapping table without restarting the service process.

[0007] Preferred options also include: At least some of the microservices are deployed in a cluster and scaled horizontally.

[0008] Preferred options also include: The number of running instances of the historical data service and the real-time data service can be dynamically increased.

[0009] Preferably, dynamically increasing the number of running instances of the historical data service and the real-time data service includes: Start a new service process; The new service process retrieves the point item configuration from the point item configuration service and loads it into the local memory mapping table. Registering the address of the new instance with the service registration module enables the historical data service and the real-time data service to request the distribution of the new instance from the service registration module.

[0010] Preferred options also include: When the upstream microservice experiences a network timeout or disconnection, the downstream microservice automatically triggers a security interlock protection logic based on the quality code.

[0011] A second aspect of this invention discloses a microservice data processing system, the microservice data processing system comprising: A splitting unit is used to split the distributed control system into multiple microservices, including point item configuration services. The data exchange between the microservices includes at least numerical values, timestamps, quality codes, and version numbers. The processing unit is used to notify other microservices to obtain the new version configuration data without restarting the service process when a change in configuration data is detected and the version number is updated.

[0012] Preferably, the multiple microservices are divided into: stateless computing microservices and stateful storage microservices; The stateless computational microservices include at least: point item configuration service, historical data service, and real-time data service; The stateful storage-type microservices include at least: a point item historical data storage cluster, a point item metadata storage cluster, a sampling service cluster, an IO service cluster, and a service registration module.

[0013] Preferably, the processing unit is specifically used to: after detecting a change in configuration data and an update in the version number through the point item configuration service, push a configuration change notification to all other microservices that have subscribed to the configuration through the point item configuration service, so that the other microservices can obtain the new version configuration data and update the new version configuration data to the local memory mapping table without restarting the service process.

[0014] Based on the microservice data processing method and system provided by the above embodiments of the present invention, the distributed control system is divided into multiple microservices, including a point-item configuration service, and the data exchange between microservices is standardized to include at least numerical values, timestamps, quality codes, and version numbers, thus standardizing and decoupling the interfaces of each microservice. When new functions need to be added, only the logic of the relevant microservice itself needs to be adjusted, without the need to simultaneously modify the interfaces of multiple related modules, thereby reducing the interface dependencies and coupling between modules and avoiding cross-module failures during maintenance and upgrades. When the configuration data changes and the version number is updated, the point-item configuration service notifies other microservices to obtain the new version configuration data without restarting the service process. This allows the configuration update to be completed without restarting any service process during the configuration change process, overcoming the defect of traditional DCS where control is interrupted due to service restarts, and ensuring the continuous operation capability of the system. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a microservice data processing method provided in an embodiment of the present invention; Figure 2 This is a diagram illustrating the interaction between microservices provided in an embodiment of the present invention. Figure 3 A structural block diagram of a microservice data processing system provided in an embodiment of the present invention; Figure 4 Another structural block diagram of a microservice data processing system provided in an embodiment of the present invention; Figure 5 This is another structural block diagram of a microservice data processing system provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0019] In the field of industrial control, as system scale expands (from thousands of points to hundreds of thousands or even millions of points), traditional DCS software architecture faces the following severe challenges: High coupling: Traditional DCS software often adopts a monolithic or coarse-grained distributed architecture, with complex inter-module interface dependencies. Expanding with new features (such as adding an AI algorithm module) often requires modifying multiple related modules in a coordinated manner, resulting in a domino effect where changing one part affects the whole.

[0020] Scaling difficulties: Traditional architectures typically require computing resources and storage resources to be scaled together, making it impossible to scale independently based on actual bottlenecks (such as insufficient computing or insufficient storage).

[0021] Configuration requires restart to take effect: In existing technologies, after modifying control policies or point table configurations, it is usually necessary to restart the relevant service processes for the changes to take effect. This can lead to temporary monitoring blind spots or control interruptions in the production process, failing to meet the "uninterrupted" requirements of continuous production.

[0022] In summary, industrial control systems deploy a large amount of software. Although traditional distributed architecture DCS software has broken down some functional modules, the interfaces between modules are complex and highly coupled. When expanding new functions, the interface logic of multiple related modules needs to be modified simultaneously, and cross-module failures are prone to occur during maintenance and upgrades.

[0023] To address the aforementioned issues, this invention proposes a microservice data processing method and system. By decomposing the distributed control system into multiple microservices, including a point-item configuration service, and standardizing data exchange between microservices to include at least numerical values, timestamps, quality codes, and version numbers, the microservice interfaces are standardized and decoupled. When new functionalities need to be added, only the logic of the relevant microservice needs to be adjusted, without simultaneously modifying the interfaces of multiple related modules. This reduces interface dependencies and coupling between modules, avoiding cross-module failures during maintenance and upgrades. When configuration data changes and the version number is updated, the point-item configuration service notifies other microservices to obtain the new version configuration data without restarting service processes. This allows configuration updates to be completed without restarting any service processes during configuration changes, overcoming the control interruption caused by service restarts in traditional DCS systems and ensuring continuous system operation.

[0024] See Figure 1 The flowchart illustrates a microservice data processing method provided by an embodiment of the present invention, which includes the following steps: Step S101: The distributed control system is split into multiple microservices, including point item configuration services. Data exchange between microservices includes at least numerical values, timestamps, quality codes, and version numbers.

[0025] In the specific implementation of step S101, based on the single responsibility principle and the characteristics of DCS business, the distributed control system is divided into multiple microservices, including point item configuration services.

[0026] Specifically, the microservices are divided into stateless compute-based microservices and stateful storage-based microservices. In other words, the distributed control system is split into stateless compute-based microservices and stateful storage-based microservices.

[0027] Stateless compute-intensive microservices do not hold any persistent data and can be scaled horizontally at any time. Stateful storage-intensive microservices rely on RocksDB+Raft for persistent storage, and scaling requires a clustering mechanism.

[0028] As shown in Table 1, stateless compute-based microservices include at least: point item configuration service, historical data service, and real-time data service. Stateful storage-based microservices include at least: point item historical data storage cluster, point item metadata storage cluster, sampling service cluster, IO service cluster, and service registration module.

[0029] Table 1:

[0030] It should be noted that the ItemConfigServer is the "brain" of the system, responsible for managing the metadata, range, alarm thresholds, and topology of all measurement points (tags).

[0031] IO Server Cluster: Responsible for collecting L1 layer data from the site and performing protocol parsing.

[0032] Historical data service and real-time data service (Realtime / HistoryDB Server): Stateless computing nodes responsible for data routing, caching, and query processing.

[0033] Storage Cluster: Responsible for persistent data storage. The storage cluster includes a point-to-point historical data storage cluster and a point-to-point metadata storage cluster.

[0034] The ServiceRegister module is responsible for maintaining the address registration and service discovery of all microservice instances within the system. When each microservice instance starts, it registers its service type and network address with the ServiceRegister module. Before initiating a cross-service call, each microservice queries the ServiceRegister module to obtain a list of available instances of the target service, and then selects the specific target instance based on load balancing strategies to complete request routing.

[0035] In some embodiments, an industrial RPC communication protocol based on Quality Code is constructed. This industrial RPC communication protocol redefines the communication contract between microservices. On the basis of gRPC and Protobuf serialization, it forces the encapsulation of a "quadruple" structure, that is, the data exchange between microservices includes at least a value, a timestamp, a quality code, and a version number (config_version).

[0036] The optimized interface definition of this industrial RPC communication protocol is shown in Table 2.

[0037] Table 2:

[0038] When each microservice instance initiates an RPC call, it first queries the list of available instances of the target service through the service registration module (ServiceRegister), then selects the target instance according to the load balancing strategy, and finally completes the communication through the industrial RPC protocol carrying the "four-tuple".

[0039] Understandably, in practical applications, quality codes and configuration version numbers are encapsulated in the microservice communication interface, and downstream microservices automatically execute degradation or interlocking protection logic based on the quality code.

[0040] In some embodiments, the data exchange between upstream and downstream microservices includes a quality code. When a network timeout or disconnection occurs in the upstream microservice, the downstream microservice automatically triggers security interlock protection logic based on the quality code.

[0041] Step S102: When a change in configuration data and an update in version number are detected, other microservices are notified through the point item configuration service to obtain the new version of configuration data without restarting the service process.

[0042] It should be noted that "other microservices" refers to any microservice other than point item configuration services.

[0043] In the specific implementation step S102, after the point item configuration service detects that the configuration data has changed and the version number has been updated, the point item configuration service pushes the configuration change notification to all other microservices that have subscribed to the configuration, so that other microservices can obtain the new version configuration data and update the new version configuration data to the local memory mapping table without restarting the service process.

[0044] Specifically, an online hot-reload mechanism based on a configuration center was built. Addressing the issue that traditional DCS configuration modifications often require service restarts, leading to control interruptions, this mechanism leverages a centralized configuration management service (Config Server) within a microservice architecture to achieve online hot updates of configuration data. Detailed update content is as follows: First, it's important to clarify that configuration data is the global configuration data for the entire DCS system, including the definition of all measurement points, data sources, storage locations, sampling periods, upper and lower limits of measurement ranges, alarm thresholds, etc. The management process for configuration data is divided into offline configuration, downloading to the database, and runtime loading.

[0045] Offline configuration: Configuration is performed uniformly by the Engineer Station, generating full configuration data.

[0046] Download and store: Configuration data is downloaded and stored in the ItemConfigServer, which manages it uniformly and maintains the configuration version number (config_version).

[0047] Runtime loading: When all business-related microservices in the system (IO service cluster, real-time data service, historical data service, sampling service cluster, and other microservices) start, they pull the configuration data they need from the point item configuration service, load it into the local memory mapping table (Map), and then start working.

[0048] Therefore, the ItemConfigServer is the sole authoritative source of configuration data. Other microservices do not hold their own configuration data and all rely on the ItemConfigServer for provision.

[0049] This solution achieves online, non-intrusive configuration updates, which differs from traditional DCS, through version number comparison and asynchronous hot reloading mechanisms.

[0050] The hot-loading process of this solution is as follows: The engineer station completes the configuration of the new version of the configuration data and downloads it to the point-item configuration service, incrementing the version number; after detecting the version number update, the point-item configuration service actively pushes a configuration change notification to all other microservices that have subscribed to the configuration; after receiving the configuration change notification, other microservices pull the new version of the configuration data (i.e., the latest configuration data) from the point-item configuration service in a background asynchronous thread; after the pull is completed, the local memory mapping table (Map) is directly updated, and the new data processing logic is immediately executed based on the new configuration (new version of the configuration data). During the execution of the aforementioned hot-loading process, the data acquisition, IO connection, and control logic of the foreground main thread are uninterrupted, and the service process does not need to be restarted.

[0051] Understandably, the role of the version number during the hot reload process is that other microservices (business-related microservices) can compare the version number cached locally with the current version number of the point item configuration service to determine whether the configuration has changed at any time, ensuring configuration consistency and preventing the mixing of old and new version configuration data during the transition period.

[0052] In general, the hot reloading process mainly consists of two parts: "centralized management" and "asynchronous push and hot reloading".

[0053] Centralized management: All point item definitions, range upper and lower limits, alarm thresholds and other configuration information are uniformly managed by the point item configuration service, which also maintains the version number of the configuration file.

[0054] Asynchronous push and hot reloading: When configuration data changes, the point item configuration service sends configuration change notifications to other microservices that have subscribed to the configuration (such as IO service cluster, real-time data service, historical data service, sampling service cluster, etc.).

[0055] In practical applications, the point-to-point configuration service can send configuration change notifications to all microservices except for the service registration module.

[0056] After receiving the configuration change notification, other microservices asynchronously retrieve the new version of the configuration data in a background thread and update the in-memory mapping table. The entire process does not require restarting the service process or disconnecting the underlying I / O connection, thus ensuring that the real-time data acquisition and control logic of the DCS system are not affected during configuration adjustments.

[0057] In some embodiments, to address the difficulty of scaling up traditional DCS, this solution constructs a compute-scalability separation architecture, which comprises a compute layer (stateless) and a storage layer (stateful).

[0058] Computation Layer (Stateless): Real-time data services and historical data services are designed as stateless nodes. They do not hold any persistent data and only route requests based on configuration. This means that when the DCS faces high concurrency query pressure, new instances with extremely short execution times can be launched to share the load.

[0059] Storage layer (stateful): The underlying layer is a storage cluster based on the RocksDB+Raft consistency protocol.

[0060] Among them, the point-storage-cluster is responsible for the persistence of time-series data.

[0061] The storage-server-cluster is responsible for the persistence of configuration data.

[0062] The Raft algorithm ensures strong consistency in data reading and writing even if n nodes fail out of the 2n+1 nodes, preventing unacceptable data loss or rollback in industrial settings.

[0063] Based on the elastic scaling architecture, when the DCS system faces high concurrency query pressure, the number of running instances of historical data service and real-time data service can be dynamically increased.

[0064] In the specific implementation, the method to dynamically increase the number of running instances of historical data service and real-time data service is as follows: start a new service process; pull point item configuration from the point item configuration service through the new service process and load it into the local memory mapping table; register the address of the new instance with the service registration module, so that historical data service and real-time data service request the distribution of new instances from the service registration module.

[0065] To further understand how to increase the number of running instances, the following example illustrates the process: Step 1, Process Startup: Start a new service process on the target node. Since the service program has been pre-deployed to each node, the startup time is extremely short.

[0066] Step 2: The new service process pulls the point item configuration from the point item configuration service and loads it into the local memory-mapped table (Map). Since both the real-time data service and the historical data service are stateless nodes and do not hold any persistent data, no data migration or state synchronization is required. Once the configuration loading is complete, the service logic is ready.

[0067] Step 3: The new instance registers its address with the ServiceRegister module. When the historical data service and real-time data service initiate another call, they can detect the existence of the new instance through the ServiceRegister module. The historical data service and real-time data service then request the distribution of the new instance from the ServiceRegister module to complete the scaling and access.

[0068] Steps 1 through 3 do not require restarting any existing service processes, and the data in the storage cluster remains unaffected, achieving seamless elastic scaling of the running DCS system.

[0069] In some embodiments, this solution introduces a cluster design to deploy and horizontally scale at least a subset of microservices in a clustered manner. Specifically, microservices requiring high availability or high throughput (such as IO service clusters, point item metadata storage clusters, point item historical data storage clusters, etc.) are deployed and horizontally scaled in a clustered manner. High availability of the cluster is achieved using consistency mechanisms such as Raft, ensuring that the overall functionality is not affected by the failure of n or fewer nodes.

[0070] The above is a detailed description of this solution. To better understand the functions of each microservice decomposed in this solution and the interactions between microservices, the following content and... Figure 2 The diagram shown illustrates an example of interactions between microservices.

[0071] First, the distributed control system is divided into point item configuration service, point item historical data storage cluster, historical data service, real-time data service, sampling service cluster, point item metadata storage cluster, and IO service cluster. Each microservice can be independently expanded according to actual needs without the need to adjust other services in conjunction with it.

[0072] The business microservice layer includes business-related microservices, such as IO service clusters, sampling service machines, real-time data services, point item metadata storage clusters, and point item historical data storage clusters.

[0073] ItemConfigServer: Manages basic information for each data point (such as data point type, data source, storage location, sampling period, etc.). This service is stateless and can be freely scaled; configuration data is preloaded into memory at runtime, resulting in high performance and resolving the coupling issue between configuration modules and business modules in traditional distributed architectures.

[0074] Point-storage cluster: High availability is achieved using the Raft mechanism, with RocksDB storage at the underlying layer. A single cluster consists of 2n+1 nodes, and the system can deploy multiple clusters based on the amount of point-data. Compared to traditional distributed storage modules, this cluster supports independent expansion, and the failure of n or fewer nodes does not affect the overall data storage function, avoiding global data read / write interruptions caused by the failure of traditional distributed storage modules.

[0075] Historical Data Service (historydb-server-n): Historical data write and query operations converge to this service, which connects to the corresponding storage nodes based on the configuration of each point. This service is stateless and can be scaled in parallel into a cluster, solving the problems of coupling between the historical data processing module and other business modules in traditional distributed architectures, and the need to modify related interfaces for expansion.

[0076] Real-time Data Service (realtimedb-server-n): Real-time data write and query operations converge to this service, which connects to the corresponding storage nodes based on the configuration of each data item. This service is stateless and can be scaled in parallel into a cluster, breaking through the limitation of traditional distributed real-time data modules requiring coordinated adjustments to upstream and downstream interfaces for expansion.

[0077] The sampling service cluster (sample-server-cluster) periodically notifies the real-time database of data points based on configuration information. This module features high availability and can be horizontally scaled as the number of system collection points grows, without requiring modification of the coupling interface with the data receiving module as in traditional distributed sampling modules.

[0078] The point-database storage cluster (storage-server-cluster) employs the Raft mechanism for high availability and uses RocksDB storage at the underlying level. A single cluster consists of 2n+1 nodes, and the system can deploy multiple clusters based on the amount of point-data. Compared to traditional distributed metadata modules, this cluster supports independent fault recovery; the system continues to function normally even when n or fewer nodes fail, avoiding global service unavailability caused by metadata module failures. The system supports deploying multiple point-database historical data storage cluster instances, with the point-database storage cluster maintaining the mapping relationship between each data point and the storage cluster instance. When performing data read / write operations, the historical data service first queries the point-database storage cluster to obtain the storage cluster number of the target data point, and then routes the operation to the corresponding cluster. This supports horizontal sharding storage for millions of data points, breaking through the capacity limit of a single storage cluster.

[0079] IO server cluster: Writes L1 data to the real-time database periodically according to configuration information. This module has high availability and can be horizontally scaled as the number of system collection points increases, solving the problem that traditional distributed IO modules need to adjust data format adaptation logic synchronously when expanding.

[0080] The ServiceRegister module is responsible for managing the network address registration and discovery of each microservice instance within the system. Each microservice instance registers with the module upon startup. When a caller initiates a service request, it queries the module for the target instance's address, and dynamic request routing is achieved using load balancing strategies. Dynamic addition and removal of microservice instances is supported. During scaling, new instances are registered and immediately recognized and integrated by other services without requiring manual configuration or restarting of existing services.

[0081] In summary, this solution has the following beneficial effects: This invention ensures industrial-grade data security: Existing general-purpose microservices cannot express sensor faults. By forcibly introducing a quality code transmission mechanism through the RPC layer, this invention enables the system to transmit the BAD quality bit, triggering subsequent security protection logic, in the event of data interruption or service failure, just like a traditional DCS.

[0082] This invention achieves truly uninterrupted online configuration: Traditional DCS parameter modifications often require system shutdown or restart, posing a significant risk. The hot-loading mechanism of this invention leverages microservice characteristics to ensure uninterrupted service and no data loss during configuration changes, meeting continuous production requirements.

[0083] Extremely high system availability and elasticity: Through storage-compute separation and Raft clustering, on-demand expansion of computing resources (such as responding to sudden alarm storms) and high reliability of storage resources (tolerating single-machine failures) are achieved, solving the problems of rigid expansion and total shutdown caused by single point of failure in traditional distributed DCS.

[0084] Corresponding to the microservice data processing method provided in the above embodiments of the present invention, see also... Figure 3 The present invention also provides a structural block diagram of a microservice data processing system, which includes a splitting unit 301 and a processing unit 302.

[0085] Splitting unit 301 is used to split the distributed control system into multiple microservices, including point item configuration services. Data exchange between microservices includes at least numerical values, timestamps, quality codes, and version numbers.

[0086] In some embodiments, multiple microservices are divided into: stateless compute microservices and stateful storage microservices. Among them, stateless computational microservices include at least: point item configuration service, historical data service, and real-time data service.

[0087] Stateful storage-based microservices include at least: a point item historical data storage cluster, a point item metadata storage cluster, a sampling service cluster, an IO service cluster, and a service registration module.

[0088] The processing unit 302 is used to notify other microservices to obtain the new version of configuration data without restarting the service process when a change in configuration data and version number are detected.

[0089] In some embodiments, the processing unit 302 is specifically configured to: after detecting a change in configuration data and an update in the version number through the point item configuration service, push a configuration change notification to all other microservices that have subscribed to the configuration through the point item configuration service, so that other microservices can obtain the new version configuration data and update the new version configuration data to the local memory mapping table without restarting the service process.

[0090] Preferably, in some embodiments, the processing unit 302 is further configured to: when a network timeout or disconnection occurs in an upstream microservice, automatically trigger a security interlock protection logic based on the quality code in a downstream microservice.

[0091] Preferred, combined Figure 3 See Figure 4 This diagram illustrates another structural block diagram of a microservice data processing system provided in an embodiment of the present invention, which further includes: Cluster design unit 303 is used for clustered deployment and horizontal scaling of at least a portion of microservices.

[0092] Preferred, combined Figure 3 See Figure 5 This diagram illustrates another structural block diagram of a microservice data processing system provided in an embodiment of the present invention, which further includes: Add unit 304 to dynamically increase the number of running instances of historical data service and real-time data service.

[0093] In some embodiments, the added unit 304 is specifically used for: starting a new service process; pulling point item configuration from the point item configuration service through the new service process and loading it into the local memory mapping table; registering the address of the new instance to the service registration module, so that the historical data service and the real-time data service request the service registration module to distribute the new instance.

[0094] In summary, this invention provides a microservice data processing method and system. By decomposing a distributed control system into multiple microservices, including a point-item configuration service, and standardizing data exchange between microservices to include at least numerical values, timestamps, quality codes, and version numbers, the interfaces of each microservice are standardized and decoupled. When new functions need to be added, only the logic of the relevant microservices needs to be adjusted, without simultaneously modifying the interfaces of multiple related modules. This reduces interface dependencies and coupling between modules, avoiding cross-module failures during maintenance and upgrades. When configuration data changes and the version number is updated, the point-item configuration service notifies other microservices to obtain the new version of configuration data without restarting the service process. This allows configuration updates to be completed without restarting any service processes during configuration changes, overcoming the drawback of traditional DCS systems where control is interrupted due to service restarts, and ensuring the continuous operation of the system.

[0095] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0096] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A microservice data processing method, characterized in that, The method includes: The distributed control system is divided into multiple microservices, including point item configuration services. The data exchange between the microservices includes at least numerical values, timestamps, quality codes, and version numbers. When a change in configuration data is detected and the version number is updated, the point item configuration service notifies other microservices to obtain the new version of configuration data without restarting the service process.

2. The method according to claim 1, characterized in that, The microservices are divided into: stateless compute-based microservices and stateful storage-based microservices; The stateless computational microservices include at least: point item configuration service, historical data service, and real-time data service; The stateful storage-type microservices include at least: a point item historical data storage cluster, a point item metadata storage cluster, a sampling service cluster, an IO service cluster, and a service registration module.

3. The method according to claim 1, characterized in that, When a change in configuration data is detected and the version number is updated, the point item configuration service notifies other microservices to obtain the new version of configuration data without restarting the service process, including: After the point item configuration service detects that the configuration data has changed and the version number has been updated, it pushes a configuration change notification to all other microservices that have subscribed to the configuration through the point item configuration service. This allows the other microservices to obtain the new version configuration data and update the new version configuration data to their local memory mapping table without restarting the service process.

4. The method according to claim 1, characterized in that, Also includes: At least some of the microservices are deployed in a cluster and scaled horizontally.

5. The method according to claim 2, characterized in that, Also includes: The number of running instances of the historical data service and the real-time data service can be dynamically increased.

6. The method according to claim 5, characterized in that, Dynamically increasing the number of running instances of the historical data service and the real-time data service includes: Start a new service process; The new service process retrieves the point item configuration from the point item configuration service and loads it into the local memory mapping table. Registering the address of the new instance with the service registration module enables the historical data service and the real-time data service to request the distribution of the new instance from the service registration module.

7. The method according to claim 1, characterized in that, Also includes: When the upstream microservice experiences a network timeout or disconnection, the downstream microservice automatically triggers a security interlock protection logic based on the quality code.

8. A microservice data processing system, characterized in that, The microservice data processing system includes: A splitting unit is used to split the distributed control system into multiple microservices, including point item configuration services. The data exchange between the microservices includes at least numerical values, timestamps, quality codes, and version numbers. The processing unit is used to notify other microservices to obtain the new version configuration data without restarting the service process when a change in configuration data is detected and the version number is updated.

9. The microservice data processing system according to claim 8, characterized in that, The microservices are divided into: stateless compute-based microservices and stateful storage-based microservices; The stateless computational microservices include at least: point item configuration service, historical data service, and real-time data service; The stateful storage-type microservices include at least: a point item historical data storage cluster, a point item metadata storage cluster, a sampling service cluster, an IO service cluster, and a service registration module.

10. The microservice data processing system according to claim 8, characterized in that, The processing unit is specifically used to: after detecting a change in configuration data and an update in the version number through the point item configuration service, push a configuration change notification to all other microservices that have subscribed to the configuration through the point item configuration service, so that the other microservices can obtain the new version configuration data and update the new version configuration data to the local memory mapping table without restarting the service process.

Images