Deterministic soft bus communication method for virtualized industrial control system

Abstract

The invention discloses a deterministic soft bus communication method for a virtualized industrial control system, and relates to the field of industrial automation. According to the method, a soft bus communication mode is adopted, so that the containers are interacted through the intermediate message bus, and direct connection does not need to be directly established; all data streams are managed by a unified bus, so that the number of connections is reduced; and meanwhile, a shared memory zero copy mechanism is adopted to optimize data transmission, and the data is directly shared between a producer and a consumer by using a shared memory and a direct memory mapping mode, so that unnecessary copy operation is avoided, the overhead of data transmission is reduced, and the communication efficiency is improved. According to the invention, a unified and efficient communication channel can be established among a plurality of industrial edge applications on the edge device, real-time communication among local containers is supported, automatic routing capability of remote communication is realized, and the communication requirements of high concurrency and high real-time performance in a virtualized industrial control system are met.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of industrial automation, and in particular to a deterministic soft bus communication method for a virtualized industrial control system. BACKGROUND

[0002] With the continuous evolution of industrial automation systems, the traditional control architecture based on programmable logic controllers (PLC) exposes many limitations in terms of system scalability, flexibility, and maintenance costs. In particular, in large-scale production scenarios, the distributed deployment of thousands of PLC nodes not only increases the complexity of firmware upgrades, fault diagnosis, and parameter configuration, but also limits the unified management and dynamic adjustment capabilities of the control system.

[0003] To improve the resource utilization and maintainability of industrial systems, more and more industrial edge nodes are beginning to introduce general-purpose computing platforms with multi-core processing capabilities, and on this basis, deploy virtualization technologies such as lightweight containers (Containers) and virtual machines, to achieve the isolation of computing resources, flexible scheduling of tasks, and modular packaging of system functions. This container-based virtual PLC architecture has higher reconfigurability and deployment efficiency in terms of functionality, and has become one of the key directions for the development of industrial edge computing.

[0004] However, with the increase in the number of containers and the enhancement of concurrency of industrial tasks, the demand for local communication between containers has significantly increased. In a multi-container high-concurrency execution environment, traditional communication mechanisms (such as socket communication based on TCP or Unix Domain Socket) are difficult to meet the strict requirements of industrial real-time performance in terms of data copy times, context switching overhead, and delay predictability. In addition, some existing asynchronous communication solutions (such as message queues, publish / subscribe, etc.) have good throughput capacity, but most lack deterministic scheduling mechanisms and do not provide deterministic communication times, making it difficult to guarantee the time boundaries of event responses, and thus are not suitable for delay-sensitive industrial application scenarios.

[0005] In the prior art, there are also some communication frameworks based on shared memory that attempt to bypass the kernel state to improve transmission efficiency. However, these methods often lack unified scheduling strategies and container-level event-driven interface support, which is not conducive to the combination and reuse of industrial function blocks. At the same time, for cross-container data exchange and event routing, there is still no unified communication mechanism with low overhead, high predictability, and easy deployment.

[0006] Therefore, the person skilled in the art is committed to developing a virtual industrial control system deterministic soft bus communication method. A high-performance, deterministic communication mechanism suitable for containerized industrial edge environment can guarantee low-delay predictable communication while supporting event-driven execution mode, zero-copy data transmission and flexible routing control across containers, thereby meeting the urgent needs of high-concurrency, reconfigurable and predictable communication infrastructure in industrial field. SUMMARY

[0007] In view of the above defects of the prior art, the technical problem to be solved by the present application is the high-concurrency, reconfigurable and predictable communication requirement of the containerized industrial edge environment.

[0008] To achieve the above-mentioned purpose, the present application provides a virtual industrial control system deterministic soft bus communication method, which adopts soft bus communication and interacts between containers through an intermediate message bus.

[0009] Further, the data flow is managed by a unified bus.

[0010] Further, by using shared memory and direct memory mapping, data is directly shared between producers and consumers, realizing zero-copy transmission of data.

[0011] Further, the system architecture includes a hardware layer, an operating system layer, an intermediate support layer and a container layer.

[0012] Hardware layer: physical resources of edge computing nodes;

[0013] Operating system layer: on top of the hardware layer, a lightweight real-time operating system or a Linux system supporting real-time scheduling features is deployed;

[0014] Intermediate support layer: including a container engine and a soft bus; the container engine is responsible for the life cycle management and resource isolation of the function block container; the soft bus provides a unified and efficient communication mechanism, supports event delivery and data exchange between containers; the soft bus adopts an event-driven mechanism, provides a unified communication interface between containers and between containers and the outside world, and provides predictable communication delay;

[0015] Container layer: including a plurality of function block containers, each container encapsulating a plurality of microservices, and a built-in daemon receiving external events and triggering internal microservices.

[0016] Further, including a container and microservice layer, a registry center, a memory pool, a connection descriptor table, an event queue and an event return queue, and an event scheduler;

[0017] Container and microservice layer: responsible for industrial control tasks;

[0018] Registration center: records the microservice identifier, resource requirements, and communication configuration of the registered container, and allocates a dedicated memory area in the memory pool;

[0019] Memory pool: allocates data exchange areas for containers and microservices in the system;

[0020] Connection descriptor table: storage system communication topology information;

[0021] Event queue and event return queue: The system has an event queue and an event return queue to cache input events and subsequent trigger events;

[0022] Event scheduler: Responsible for extracting events from the event return queue, querying the connection descriptor table to determine the target container and target microservice, and distributing events to the event queue.

[0023] Furthermore, the registration center manages the identification information, resource requirements and communication configuration of the container, and the container communicates with the soft bus through an interface.

[0024] Furthermore, the communication topology information includes the connection relationship between each container and microservice, the target address and the event routing rules.

[0025] Furthermore, the event scheduler maintains the specific transmission order of the event queue through a specific scheduling strategy, thereby ensuring the real-time and deterministic nature of the entire system.

[0026] Furthermore, the memory arrangement in the memory pool adopts a hierarchical block structure, and memory blocks of different sizes are divided in advance.

[0027] Further, the following steps are included:

[0028] Step 1: Task execution of the first container: After receiving the input event, the first container starts the microservice module to process the task and generate output data;

[0029] Step 2: Data is written to the memory pool: The first container writes the output data directly to the specified location in the shared memory pool. The memory pool is responsible for managing the data exchange area.

[0030] Step 3: Return data address: The memory pool generates and returns the memory address offset value of the stored output data for subsequent transmission and access;

[0031] Step 4: Event reporting to the soft bus: The first container packages the event containing the data address information and submits it to the soft bus as an event return, indicating that the current task execution has been completed;

[0032] Step 5, event scheduling: After receiving the event, the soft bus queries the target container and scheduling strategy through the connection descriptor table to determine the subsequent event transmission path and activation timing;

[0033] Step 6, event distribution to the second container: the soft bus distributes the event to the target second container according to the scheduling result, and the metadata carried by the event contains address information of the output data in the shared memory pool;

[0034] Step 7, second container data access: after receiving the event, the second container extracts the shared memory address from the event metadata and directly accesses the input data through the zero-copy mechanism;

[0035] Step 8, second container task execution: after obtaining the input data, the second container starts the corresponding micro-service logic to process the task, and completes the analysis and further calculation of the input data.

[0036] The existing virtualization industrial control system point-to-point communication has the problems of a sharp increase in the number of connections and rising management costs. The present application is based on a soft bus communication mode. The present application uses a soft bus communication mode, which allows containers to interact through an intermediate message bus without directly establishing a direct connection. All data streams are managed by a unified bus, reducing the number of connections and improving the scalability of the system. The present application can solve the problem of a sharp increase in the number of connections under high concurrency through soft bus communication.

[0037] Multiple data copying and user-kernel state switching result in high communication delay. The present application uses a shared memory zero-copy mechanism to optimize data transmission. The zero-copy technology of the present application uses shared memory and direct memory mapping to allow data to be directly shared between producers and consumers, avoiding unnecessary copying operations and reducing data transmission overheads and improving communication efficiency. The present application reduces communication delay and simplifies the communication mechanism, thereby greatly reducing communication delay and delay jitter.

[0038] The predictability of communication delay is greatly improved. The present application improves the predictability of communication by simplifying the transmission mechanism. The mechanism of the present application is simplified due to the design of the transmission mechanism, with no large number of context switches and copying, making it easy to predict the worst-case execution time. The present application improves the predictability of communication delay.

[0039] Compared with the prior art, the present application has the following obvious substantial characteristics and significant advantages:

[0040] 1. Technical advantages

[0041] 1.1 Avoiding a sharp increase in the number of connections to support large-scale virtualization applications

[0042] The application effectively avoids the system management pressure and performance bottleneck caused by the sharp increase in the number of connections in traditional point-to-point communication by introducing a soft bus-based communication mechanism. This mechanism significantly reduces system management overhead and improves communication efficiency. More importantly, the soft bus mode gives the system excellent scalability, enabling it to easily adapt to the high concurrency requirements of large-scale virtualization and industrial automation scenarios.

[0043] 1.2 Data transmission mechanism optimization, enhancing communication certainty

[0044] The application uses shared memory and zero-copy technology to eliminate the performance loss caused by multiple data copying and user-kernel state switching in traditional communication, achieving efficient data transmission between containers. This optimization not only significantly reduces communication delay and improves data throughput, but also fully meets the stringent requirements of industrial edge applications for efficient data interaction.

[0045] In addition, the simplified communication process converts tedious data copying operations into memory read-write operations based on event-driven, facilitating accurate prediction of communication execution time. This further enhances the real-time performance and certainty of the system, laying a solid foundation for efficient operation of industrial automation applications.

[0046] 2. Performance indicators

[0047] The experiment uses Ubuntu 22.04.5 LTS operating system (kernel version 6.8.0), and the hardware platform configuration includes Intel Xeon Platinum 8252C processor (2 processor slots, 12 cores, dual threads, total 48 threads), equipped with 377GB DDR4 memory (rate 3200MT / s), supporting six-channel NUMA architecture and ECC verification.

[0048] 2.1 Connection initialization delay

[0049] The system conducted initialization delay tests under different connection quantities (50, 100, 150, 200, 300, 500, 1000). The results show that the connection initialization delay of the application using software bus (Software Bus) is significantly lower than that of traditional TCP and UDS methods:

[0050] Table 1 Connection initialization time

[0051]

[0052] 2.2 Data transmission delay

[0053] Figure 6The one-way message transmission delay range (minimum value, average value, maximum value) of the software bus (Software Bus) of the application, traditional TCP and UDS three communication mechanisms under different loads is shown in the figure. It can be seen that the average delay of the software bus under all load conditions is significantly lower than that of TCP and UDS, and the maximum delay changes less with the increase of load, showing excellent stability and real-time performance. In contrast, the delay of TCP and UDS increases significantly with the increase of data load, and the maximum delay presents a large fluctuation, and the stability is poor. This result fully illustrates the technical advantages of the software bus in the multi-task, high concurrency and low delay scene, and it is especially suitable for industrial edge systems that require high determinacy and predictability.

[0054] 3. Production implementation

[0055] The application is suitable for various industrial scenes, including local inter-process communication of traditional industrial control systems, inter-application communication of industrial edge computing systems, and inter-container communication of virtualized industrial control systems.

[0056] In some industrial edge computing systems, there is a typical demand for multi-language heterogeneous application to work together: the application program written in C++ is used for high-speed acquisition of image data, and image analysis and processing are completed by Python application. By using the soft bus mechanism proposed by the application, local zero-copy real-time communication of image transmission can be provided.

[0057] The soft bus mechanism can establish a unified and efficient communication channel between multiple industrial edge applications on the edge device, not only supporting local inter-container real-time communication, but also having automatic routing capability for remote communication, meeting the communication demand of high concurrency and high real-time performance in virtualized industrial control systems.

[0058] The concept, specific structure and technical effects of the application will be further described below in combination with the drawings, so as to fully understand the purpose, features and effects of the application. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a container-based virtualized industrial control system architecture of a preferred embodiment of the application;

[0060] Figure 2 is an event-driven soft bus architecture composition of a preferred embodiment of the application;

[0061] Figure 3 is a container registration and communication routing of a preferred embodiment of the application;

[0062] Figure 4 is a soft bus communication timing of a preferred embodiment of the application;

[0063] Figure 5 is a memory arrangement structure of a preferred embodiment of the present application;

[0064] Figure 6 is a range of one-way message transmission delay of software bus, TCP and UDS communication mechanism under different loads. DETAILED DESCRIPTION

[0065] The preferred embodiments of the present application are described below with reference to the accompanying drawings, so that the technical contents can be more clear and convenient to understand. The present application can be embodied in many different forms, and the protection scope of the present application is not limited to the embodiments mentioned herein.

[0066] In the drawings, components of the same structure are denoted by the same reference numerals, and components having similar structures or functions are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present application is not limited to the size and thickness of each component. In order to make the drawing clearer, the thickness of some components is appropriately exaggerated in some places in the drawing.

[0067] As Figure 1 is a container-based virtualization industrial control system architecture, which aims to meet the communication and computing requirements of high concurrency, real-time and high reliability in industrial field, adopts hierarchical decoupling design, and has good portability, scalability and efficient communication capability.

[0068] The system architecture is composed of a hardware layer, an operating system layer, an intermediate support layer and a container layer.

[0069] Hardware layer: The hardware layer is composed of physical resources of edge computing nodes, including but not limited to CPU, memory, input / output device (IO device), network card, etc. It provides necessary computing power, storage and communication capability support for subsequent software and container layer.

[0070] Operating system layer: On top of the hardware layer, a lightweight real-time operating system (RTOS) or a Linux system supporting real-time scheduling features (such as PREEMPT_RT) is deployed to guarantee the low delay and deterministic execution of critical control tasks in the system, and to improve the real-time response capability of the system.

[0071] Intermediate support layer: composed of container engine and software bus. The container engine (such as Docker) is responsible for the life cycle management and resource isolation of functional block containers; the software bus provides a unified and efficient communication mechanism, supports event transmission and data exchange between containers, and meets the real-time communication requirements in industrial scenarios. The software bus (Software Bus) adopts an efficient event-driven mechanism, provides a unified communication interface between containers and between containers and the outside world, and provides predictable communication delay.

[0072] Container layer: composed of multiple function block containers, each container encapsulates multiple microservices and has a daemon process built-in to receive external events and trigger internal microservices.

[0073] Figure 2 The overall architecture of the event-driven soft bus is shown. The architecture mainly includes the following core modules:

[0074] 1. Container and microservice layer: composed of multiple containers, each container contains microservices responsible for specific industrial control tasks. Containers communicate with the soft bus through interfaces, and the identification information, resource requirements, and communication configurations of containers are managed through the registration center.

[0075] 2. Registration center: records the microservice identifiers, resource requirements, and communication configurations of all registered containers, and allocates exclusive memory areas for them in the memory pool. Communication topology information is stored in the connection descriptor table for subsequent event scheduling and data routing.

[0076] 3. Memory pool: allocates data exchange areas for each container and microservice in the system, supporting efficient data access and transmission.

[0077] 4. Connection descriptor table: stores the communication topology information of the system, including the connection relationship between containers and microservices, target addresses, and event routing rules, for event scheduler to query.

[0078] 5. Event queue and event return queue: the system has event queue and return queue to cache input events and subsequent triggered events, realizing efficient event delivery and processing.

[0079] 6. Event scheduler: responsible for extracting events from the event return queue, querying the connection descriptor table to determine the target container and target microservice, and distributing events to the corresponding event queue to ensure the real-time and efficient scheduling of the system. At the same time, the scheduler maintains the specific transmission order of the event queue through specific scheduling strategies, ensuring the real-time and determinacy of the overall system.

[0080] Container communication initialization and communication process are carried out according to the flowchart of Figure 3 .

[0081] When a new container is created, communication initialization is performed, first mapping the shared memory pool to the process address space and allocating memory blocks. At the same time, the new container sends the IP address of the host it is located in, the function block ID, and the communication intention through broadcast during initialization. The broadcast information is listened to by multiple devices and stored in the local database for subsequent quick query and communication positioning.

[0082] When the container needs to communicate with other containers, the sending container first queries the relevant information of the target container from the database. According to the query result, the communication path is selected. If the target container is local, efficient communication is realized through the soft bus; if the target container is in a remote node, the deterministic network is used for communication to the soft bus of the remote device, and then the communication between the containers is realized through the soft bus of the remote device.

[0083] As shown in Figure 4 , the soft bus communication timing of the system is as follows:

[0084] 1. Task execution of container A: after receiving the input event, container A starts the microservice module to process the task and generates output data.

[0085] 2. Data writing into the memory pool: container A writes the output data directly into the specified position of the shared memory pool, and the memory pool is responsible for efficient management of the data exchange area.

[0086] 3. Return of data address: the memory pool generates a memory address offset value for storing the output data, which is used for subsequent transmission and access.

[0087] 4. Event reporting to the soft bus: container A packages the event containing the data address information and submits it to the soft bus as event feedback, indicating that the current task execution has been completed.

[0088] 5. Event scheduling: after receiving the event, the soft bus queries the downstream target container and scheduling strategy through the connection descriptor table to determine the subsequent event transmission path and activation time.

[0089] 6. Event distribution to container B: the soft bus distributes the event to the target container B according to the scheduling result, and the metadata carried by the event contains the address information of the output data in the shared memory pool.

[0090] 7. Data access of container B: after receiving the event, container B extracts the shared memory address from the event metadata and directly accesses the input data through the zero-copy mechanism to avoid redundant data transmission.

[0091] 8. Task execution of container B: after obtaining the input data, container B starts the corresponding microservice logic to process the task, completes the parsing and further calculation of the input data.

[0092] Figure 5 The hierarchical layout of the shared memory in the soft bus system is shown.

[0093] The shared memory region 1 is mainly used for event scheduling and transmission. The event queue and the return queue are used for buffering events generated from each container and events to be transmitted after scheduling, supporting efficient access and asynchronous scheduling of events. The reserved event queue dynamically allocates memory for new events that may occur during system operation, ensuring the stability and processing capacity of the system in a high-concurrency scenario.

[0094] The shared memory region 2 is mainly used for data storage, and the memory allocation adopts a hierarchical block structure. It is divided into multiple layers of memory blocks such as 4MB, 2MB, and 4KB, arranged in descending order. Each layer contains a number of memory blocks (such as memory block 1, memory block 2, memory block 3, etc.) and a reserved memory block, used to store data of different sizes. The multi-layer memory block design combines the memory alignment principle and the data transmission characteristics in industrial control systems, avoiding fragmentation and improving data access speed, and adapting to the mixed transmission requirements of large and small data blocks. The system's reserved memory block is used to handle sudden data transmission requirements.

[0095] The above describes the preferred embodiments of the present application in detail. It should be understood that those skilled in the art can make many modifications and changes to the present application without creative labor, based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the existing technology according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A deterministic soft bus communication method for a virtualized industrial control system, characterized in that: Using soft bus communication, containers interact with each other through an intermediate message bus.

2. The deterministic soft bus communication method for a virtualized industrial control system according to claim 1, wherein: Data flow is managed by a unified bus.

3. The deterministic soft bus communication method for a virtualized industrial control system according to claim 1, wherein: By using shared memory and direct memory mapping, data is directly shared between producers and consumers, and data is transmitted with zero copy.

4. The deterministic soft bus communication method for a virtualized industrial control system according to claim 1, wherein: The system architecture includes the hardware layer, operating system layer, intermediate support layer, and container layer; Hardware layer: physical resources of edge computing nodes; Operating system layer: above the hardware layer, deploy a lightweight real-time operating system or a Linux system that supports real-time scheduling features; Middle support layer: including container engine and soft bus; The container engine is responsible for the lifecycle management and resource isolation of functional block containers; the soft bus provides a unified and efficient communication mechanism, supporting event transmission and data exchange between containers; The soft bus uses an event-driven mechanism to provide a unified communication interface between containers and between containers and the outside world, and provides predictable communication delays. Container layer: includes multiple functional block containers, each of which encapsulates multiple microservices and has a built-in daemon to receive external events and trigger internal microservices.

5. The deterministic soft bus communication method for a virtualized industrial control system according to claim 1, wherein: Includes container and microservice layer, registration center, memory pool, connection descriptor table, event queue and event return queue, and event scheduler; Container and microservice layer: responsible for industrial control tasks; Registration center: records the microservice identifier, resource requirements, and communication configuration of the registered container, and allocates a dedicated memory area in the memory pool; Memory pool: allocates data exchange areas for containers and microservices in the system; Connection descriptor table: storage system communication topology information; Event queue and event return queue: The system has an event queue and an event return queue to cache input events and subsequent trigger events; Event scheduler: Responsible for extracting events from the event return queue, querying the connection descriptor table to determine the target container and target microservice, and distributing events to the event queue.

6. The deterministic soft bus communication method for a virtualized industrial control system according to claim 5, wherein: The registration center manages the identification information, resource requirements and communication configuration of the container, and the container communicates with the soft bus through an interface.

7. The deterministic soft bus communication method for a virtualized industrial control system according to claim 5, wherein: The communication topology information includes the connection relationship between each container and microservice, the target address and the event routing rules.

8. The deterministic soft bus communication method for a virtualized industrial control system according to claim 5, wherein: The event scheduler maintains the specific transmission order of the event queue through a scheduling strategy.

9. The deterministic soft bus communication method for a virtualized industrial control system according to claim 5, wherein: The memory pool is arranged in a hierarchical block structure, and memory blocks of different sizes are divided in advance.

10. The deterministic soft bus communication method for a virtualized industrial control system according to claim 1, wherein: The following steps are involved: Step 1: Task execution of the first container: After receiving the input event, the first container starts the microservice module to process the task and generate output data; Step 2: Data is written to the memory pool: The first container writes the output data directly to the specified location in the shared memory pool. The memory pool is responsible for managing the data exchange area. Step 3: Return data address: The memory pool generates and returns the memory address offset value of the stored output data for subsequent transmission and access; Step 4: Event reporting to the soft bus: The first container packages the event containing the data address information and submits it to the soft bus as an event return, indicating that the current task execution has been completed; Step 5, event scheduling: After receiving the event, the soft bus queries the target container and scheduling strategy through the connection descriptor table to determine the subsequent event transmission path and activation timing; Step 6: Event distribution to the second container: The soft bus distributes the event to the target second container according to the scheduling result. The metadata carried by the event includes the address information of the output data in the shared memory pool. Step 7: Second container data access: After receiving the event, the second container extracts the shared memory address from the event metadata and directly accesses the input data through the zero-copy mechanism; Step 8: Second container task execution: After obtaining the input data, the second container starts the corresponding microservice logic to perform task processing and complete the parsing and further calculation of the input data.

Images