Ethernet / IP-based high-concurrency data processing methods, devices, and storage media

Abstract

This disclosure proposes a high-concurrency data processing method, apparatus, and storage medium based on Ethernet / IP. The method includes: receiving message data sent by multiple node devices, sequentially caching the message data into a first message queue in the form of data response events, assigning an event handler component to each data response event in the first message queue, processing the assigned data response event based on a pre-configured CIP object model to obtain target data, and storing the target data in a data storage module. This enables various node devices to communicate using the Ethernet / IP protocol and process data response events separately using a CIP object model constructed with unified rules, thereby achieving data interconnection and interoperability. Furthermore, this embodiment can also utilize message queues to achieve multiplexing and high-concurrency data processing.

Description

Technical Field

[0001] This disclosure relates to the field of industrial data processing technology, and in particular to a high-concurrency data processing method, apparatus and storage medium based on Ethernet / IP. Background Technology

[0002] With the rapid development of big data, artificial intelligence, and the Internet of Things, the informatization of coal mines is steadily progressing towards intelligent and smart mines. In the application of fieldbus technology, due to different business needs and technical specifications from various manufacturers, hundreds of subsystems (node ​​devices) have gradually formed. These subsystems, with different functions and communication methods, play their roles in various production stages underground, but they also have significant shortcomings for the overall intelligentization of the mine. Furthermore, in practical applications, the coal mine data processing system needs to process data from each node device. As the scale of the mine system expands, the number of node devices also increases. At this point, the data concurrency and high-performance processing capabilities of the data processing system become crucial. Summary of the Invention

[0003] This disclosure proposes a high-concurrency data processing method, apparatus, and storage medium based on Ethernet / IP, aiming to at least partially solve one of the technical problems in related technologies.

[0004] The first aspect of this disclosure proposes a high-concurrency data processing method based on Ethernet / IP, applied to a data processing system. The data processing system communicates with multiple node devices based on the Ethernet / IP protocol. The method includes: receiving message data sent by multiple node devices and sequentially caching the message data into a first message queue in the form of data response events; assigning an event handler component to each data response event in the first message queue; processing the assigned data response event based on a pre-configured CIP object model to obtain target data; and storing the target data in a data storage module.

[0005] A second aspect of this disclosure provides a high-concurrency data processing device based on Ethernet / IP, applied to a data processing system that communicates with multiple node devices based on the Ethernet / IP protocol. The device includes: a first receiving module for receiving message data sent by multiple node devices and sequentially caching the message data into a first message queue in the form of data response events; an allocation module for allocating each data response event in the first message queue to an event handler component; a processing module for processing each event handler component based on a pre-configured CIP object model to obtain target data; and a storage module for storing the target data in the data storage module.

[0006] A third aspect of this disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the Ethernet / IP-based high-concurrency data processing method of this disclosure.

[0007] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the Ethernet / IP-based high-concurrency data processing method disclosed in embodiments of this disclosure.

[0008] In this embodiment, message data sent by multiple node devices is received and sequentially cached into a first message queue in the form of data response events. Each data response event in the first message queue is assigned an event handler component. Each event handler component processes the assigned data response event based on a pre-configured CIP object model to obtain the target data and stores the target data in the data storage module. This enables the various node devices to communicate using the Ethernet / IP protocol and process the data response events using a CIP object model constructed with unified rules, thereby achieving data interconnection and interoperability. Furthermore, this embodiment can also utilize message queues to achieve multiplexing and high-concurrency data processing.

[0009] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0010] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0011] Figure 1 This is a flowchart illustrating a high-concurrency data processing method based on Ethernet / IP provided according to an embodiment of this disclosure;

[0012] Figure 2 This is a schematic diagram of a data acquisition and processing system architecture provided according to embodiments of this disclosure;

[0013] Figure 3 This is a flowchart illustrating a high-concurrency data processing method based on Ethernet / IP provided according to another embodiment of this disclosure;

[0014] Figure 4 This is a schematic diagram illustrating the interaction process between the data processing system and the application layer according to embodiments of this disclosure;

[0015] Figure 5 This is a schematic diagram of a high-concurrency data processing device based on Ethernet / IP according to another embodiment of the present disclosure;

[0016] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0017] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0018] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0019] It should be noted that the execution subject of the Ethernet / IP-based high-concurrency data processing method in this embodiment can be an Ethernet / IP-based high-concurrency data processing device. This device can be implemented by software and / or hardware, and can be configured in an electronic device, which may include, but is not limited to, a terminal, a server, etc.

[0020] Figure 1 This is a flowchart illustrating a high-concurrency data processing method based on Ethernet / IP according to an embodiment of this disclosure. This method is applied to a data processing system, such as... Figure 1 As shown, the method includes:

[0021] S101: Receive message data sent by multiple node devices and cache the message data sequentially into the first message queue in the form of data response events.

[0022] In this embodiment of the disclosure, the data processing system can also be referred to as a data acquisition server. It can be a data processing system applied in any industrial scenario, and the data processing system communicates with multiple node devices (also referred to as subsystems) based on the Ethernet / IP protocol (referred to as "EIP").

[0023] In a specific application scenario, the data processing system could be, for example, a data acquisition and processing system for a coal mine. The node devices could be any possible electronic devices used for data acquisition in the coal mine, such as various sensors and controllers. After acquiring data, the node devices would need to send the data to the data processing system for processing in the form of messages (i.e., message data).

[0024] To enable the data processing system to handle events from each node device at high concurrency, the data acquisition and processing system in this embodiment may, for example, adopt a producer-consumer design pattern, with the data processing system acting as the consumer and each node device acting as the producer. Figure 2 This is a schematic diagram of a data processing system architecture provided according to embodiments of this disclosure, such as... Figure 2 As shown, this data processing system includes, for example, an ESDK (EtherNet / IP Scanner Developers Kit) protocol stack component (EIP protocol stack in the diagram), a scheduler, an event handler component (i.e., EIP Event Handler), a CIP object model (i.e., EIP device model), a data storage module, and any other possible modules. Figure 2 In this context, “EIP Adapter#1”, “EIP Adapter#2”, and “EIP Adapter#x” represent multiple node devices.

[0025] The CIP object model is a solution for interoperability and interchangeability of network devices (i.e., node devices) based on objects. As a communication protocol for automated data transmission between devices, CIP views each network device as a collection of objects, each object as a set of device-related data, and each piece of device data as an object attribute. It provides a complete definition of devices in the network through device object descriptions. CIP provides end-users with essential control, configuration, and data acquisition services for automation systems, offering interoperability and interchangeability of industrial automation devices over Ethernet for the automation field. To further improve interchangeability in CIP networks composed of multiple vendor node devices, CIP defines a set of standard object model specifications, or "device rules." Devices implemented according to a standard "device rule" will respond to all the same commands and exhibit the same network behavior as other devices executing the same rule. The specific specifications are as follows:

[0026] The CIP object model describes the data each object should contain using Class, Instance, and Attribute, and uses Service Code to describe the operations supported by the object. Specifically: A class is an abstract collection of instances with similar functionality. An instance is a concrete object within a class; different instances may have different attributes, but common attributes shared by all instances of a class are called class attributes. The object model specification addresses all device nodes running on the CIP network and their accessible data in a unified way, including: Node Address, Class ID, Instance ID, Attribute ID, and Service Code. The Node Address is an integer value assigned to the device node on the CIP network (typically an IP address on EtherNet / IP networks, and a MAC address on DeviceNet and ControlNet networks). The Class ID is an integer value assigned to each class accessible on the network. The Instance ID is an integer value used to distinguish different instances of the same class. The Attribute ID is an integer value representing the number of an attribute field in a class or instance. Service Code: A service request code represented by an integer value in a specific object instance or object class.

[0027] Furthermore, the CIP object model is a collection of objects, including the following:

[0028] 1. Connection Manager. Responsible for managing the opening and closing of connections on the network, and providing transmission destinations for implicit and explicit connection requests.

[0029] 2. Unconnected Message Manager (UCMM). Primarily provides cross-network message services through message parsing based on connectionless transmission methods, and can perform message duplication detection and retry services. It's important to note that in CIP, UCMM is not a true object, but rather a functional component related to the specific product implementation.

[0030] 3. Message Router. Responsible for receiving explicit messages from UCMM or the transport layer, removing the message header, parsing the data, and routing the message to the destination object based on the class and attribute path to be accessed.

[0031] 4. Identifying the object. This includes all network-related services and attributes when the product connects to the network, such as providing device-related information like VendorID, IP address, and port number.

[0032] 5. Network-specific objects. These provide underlying network configuration and status information. For example, DeviceNet and ControlNet networks use different network-specific object classes.

[0033] 6. Target audience.

[0034] 7. Composite objects. Used to implement the transmission and reception of node data on the network.

[0035] In the CIP specification, a device contains three types of objects: required objects, application objects, and vendor-defined objects.

[0036] Among them, the required objects are the objects that must be selected to achieve the basic requirements of the equipment. They are one of the prerequisites for achieving equipment interchangeability and interoperability. They include connection objects (explicit message connection objects and implicit message connection objects), message routing objects, identification (identity) objects, and network-specific objects.

[0037] Application objects define the data encapsulated in the device. These objects correspond to different devices. For example, a motor object in a drive system contains attribute data describing the frequency, rated current, and motor size; an analog input object for an I / O device contains attribute data such as the type of analog input and the current value.

[0038] Vendor-defined objects refer to special objects that are not specified in the specification but are constructed by the vendor itself. These objects can be accessed using the same methods as required objects and application objects. It is unclear whether application objects and vendor-defined objects are required.

[0039] Composite objects (with a fixed class number of 0x04) provide a mechanism for flexibly combining attributes from different classes and objects and mapping them to individual attributes within the composite object. This mapping mechanism significantly improves the efficiency of information exchange over a network. Composite objects offer a message producer-consumer pattern, allowing produced messages to be sent to all consumers, implemented using UDP. CIP distinguishes between input and output composite objects. Input and output are defined from the perspective of data inflow / outflow relative to the composite object itself. Input objects aggregate information from other objects on the same device and send it to the network; output objects receive network data and write it into the attributes of other application objects.

[0040] Furthermore, the CIP object model defines explicit and implicit connection communication. CIP is a connection-based protocol. Once a connection is established, a unique connection identifier (Connection CID) is obtained through negotiation between the two parties. The definition and format of the CID are network-specific. Using the CID information, connection messages do not need to contain all connection-related information; only the CID is required, thus improving network transmission efficiency. Establishing a connection requires the use of the Unconnected Message Manager (UCMM). UCMM messages are a way to send data requests to devices that have not previously established a connection. The UCMM object is responsible for handling requests and responses to explicit unconnected messages. Using the UCMMForward_Open service request, along with connection request parameters (such as message timeout, maximum data window size, unicast or multicast, etc.), the communication connection is established, and the connection is identified using a unique CID identifier. If two devices initiate a bidirectional connection, both parties hold two CID connection identifiers. In the CIP protocol suite, ControlNet, EtherNet / IP, and CompoNet support accessing application objects via UCMM messages (i.e., accessing object data within a device via UCMM messages even when not connected), while in DeviceNet, UCMM messages can only be used to initiate connection requests. All connections in a CIP network can be categorized as I / O (implicit) connections, either explicitly or via message. Explicit message connections provide a general, multi-purpose communication channel between two devices, offering typical request / response network communication based on TCP / IP and supporting only point-to-point communication. "Explicit" means that the request message specifies the class, object, attribute, and request service code information to be accessed. I / O connections provide dedicated, special-purpose communication paths between two devices or between a single device and multiple devices. Specific application data is transmitted via I / O connections, based on the UDP protocol, supporting unicast or multicast communication; this is also called an implicit connection. "Implicit" means that the source and composition of the message data are identified through the CID.

[0041] In this embodiment of the disclosure, a CIP object model can be constructed for each node device based on the object model specification of the aforementioned standard.

[0042] In practical applications, the data processing system first loads the node information of the node devices (e.g., "EIPAdapter#1", "EIP Adapter#2") that need to collect data. This node information includes, for example, the node device's IP address and device name, and is not restricted in this regard. Next, it loads the CIP object model corresponding to the node device, and the information of this CIP object model is stored in a database (e.g., a MySQL database). Then, the data processing system initiates a connection request to each node device based on the node information and waits for the connection result. If the node device fails to connect (due to network connectivity issues or the node being offline), it attempts to reconnect until a successful connection is established. After successfully connecting to a node device, the system requests data from each node device. The ESDK protocol stack can act as a gateway in the system, responsible for initiating connection requests and requesting data from the node devices. Finally, each node device can respond to the request to collect data and send the collected data to the data processing system in the form of a message.

[0043] In this scenario, the data processing system of this embodiment first receives message data from each node device via the ESDK protocol stack; further, the ESDK protocol stack submits the message data sent by each node device to the scheduler in the form of events (i.e., data response events); further, the scheduler caches the data response events of one or more node devices in a non-blocking manner to a message queue configured by the scheduler, which is referred to as the first message queue (corresponding to...). Figure 2 (EIP event queue).

[0044] S102: Assign an event handler component to each data response event in the first message queue.

[0045] In this embodiment, the data processing system can configure event handler components (EIPEvent Handler) according to the number of node devices, that is, each node device can correspond to one event handling component. Specifically, the event handler component in this embodiment needs to be dynamically generated from persistent CIP object model information in the database after the data processing system establishes a connection with the node devices, and then uniformly registered with the scheduler to accept scheduling from the scheduler.

[0046] After the event handler component persists the CIP object model, the CIP object model is stored in memory in the following table format:

[0047] Table 1 Communication Configuration Storage Structure

[0048] Device ID Network Address 1 172.20.1.10

[0049] Table 2 Application Object Storage Structure (Determined at runtime)

[0050]

[0051]

[0052] Table 3. Storage structure of composite objects (determined at runtime)

[0053]

[0054] Each communicable device is assigned a Device ID and its IP address is recorded. In the EIP protocol, the explicit connection port number is fixed at 0xAF12, and the implicit connection port number is fixed at 0x08AE, therefore, port number information does not need to be recorded. Following the ESDK protocol stack, at system startup, each device is allocated one explicit connection for cyclically accessing application object data (service code: GET_ATTR_LST). Each composite object on the device must maintain a separate implicit connection for receiving implicit datagrams. After the connection is established, the CID tag recording the session needs to be updated in the corresponding data structure in the processor component. The application object model is determined by structured triple information of "class, instance, attribute," and data type information is added to indicate the data length. This approach can adapt to existing application objects and is compatible with future application object extensions; extending to new application objects does not require program modification. An offset field is added to the mapping relationship between composite instance data and specific application object attributes, allowing skipping uninteresting parts of the implicit datagram during data parsing and parsing only the specific data of interest to the system. This table only describes the minimum field requirements in the communication process. When persistently storing data in the database, additional information such as device descriptions, application object attribute descriptions, and data scaling parameters can be added as needed. In addition to the data model described above, the event handling functions in the model include: a connection request function (explicit / implicit), connection establishment / timeout / disconnection event response functions, an explicit datagram parsing function, an explicit data request function, and an implicit datagram parsing function. These functions are actively invoked by the scheduler after the component is registered with it and upon the arrival of an event.

[0055] After caching the data response events to the first message queue as described above, the scheduler in this embodiment can further use the event loop thread to allocate each data response event in the first message queue to an event handler component through a multiplexing strategy.

[0056] In this embodiment, the event queue (first message queue) created by the scheduler separates the data arrival event from the data parsing task. When new message data arrives, it is cached in the queue as a data response event, allowing the protocol stack call to return as quickly as possible. The event queue then introduces another worker thread to synchronously and orderly allocate event handler components. This achieves high network concurrency, improves system data throughput, and resolves the problem of uneven workload distribution.

[0057] S103: Each event handler component processes the assigned data response events based on a pre-configured CIP object model to obtain the target data.

[0058] The CIP object model views each node device as a collection of objects, corresponding one-to-one with the node devices in the network. This CIP object model is used to manage its communication session identifiers with specific devices, application objects and composite object data, and data request and response data parsing.

[0059] In this embodiment, each event handler component parses and processes the assigned data response event (message data, which can be implicit or explicit data) based on the configured CIP object model to determine the data to be collected. This data is called the target data, such as the actual data collected by the node device. Thus, multiple event handler components can parse the message data from multiple node devices in parallel to obtain the required target data, achieving a high-concurrency event response effect.

[0060] S104: Store the target data in the data storage module.

[0061] In other words, the data processing system stores the processed target data into the data storage module.

[0062] In this embodiment, message data sent by multiple node devices is received and sequentially cached into a first message queue in the form of data response events. Each data response event in the first message queue is assigned an event handler component. Each event handler component processes the assigned data response event based on a pre-configured CIP object model to obtain the target data and stores the target data in the data storage module. This enables the various node devices to communicate using the Ethernet / IP protocol and process the data response events using a CIP object model constructed with unified rules, thereby achieving data interconnection and interoperability. Furthermore, this embodiment can also utilize message queues to achieve multiplexing and high-concurrency data processing.

[0063] Figure 3 This is a flowchart illustrating a high-concurrency data processing method based on Ethernet / IP according to another embodiment of this disclosure, as shown below. Figure 3 As shown, the method includes:

[0064] S301: Receives message data sent by multiple node devices and caches the message data sequentially into the first message queue in the form of data response events.

[0065] For a detailed description of S301, please refer to the above embodiments, which will not be repeated here.

[0066] S302: Obtain the identity information carried by each data response event and determine the target CIP object model corresponding to the identity information.

[0067] In this embodiment, the message data of the node device may carry the message's identity information (CID information), such as the node device number and name, without limitation. Furthermore, each CIP object model in this embodiment may have corresponding identity information.

[0068] For example, the identity information carried in the message data sent by the three node devices "EIP Adapter#1", "EIP Adapter#2", and "EIP Adapter#3" are "#1", "#2", and "#3" respectively. Correspondingly, the identity information of the CIP object model of the above node devices are "#1", "#2", and "#3" respectively.

[0069] In this case, in the operation of assigning each data response event in the first message queue to an event handler component, this embodiment first obtains the identity information carried by each data response event (i.e., message data); further, it determines the CIP object model corresponding to the identity information. This CIP object model can be called the target CIP object model. For example, if the node device is "EIP Adapter #1", then the corresponding target CIP object model is "#1".

[0070] S303: Assigns data response events to the event handler component configured for the target CIP object model.

[0071] Furthermore, the data response events are assigned to the event handler components that configure the target CIP object model.

[0072] For example, the data response event of "EIP Adapter #1" is assigned to the event handler component that deploys the "#1" CIP object model, the data response event of "EIP Adapter #2" is assigned to the event handler component that deploys the "#2" CIP object model, and the data response event of "EIP Adapter #3" is assigned to the event handler component that deploys the "#3" CIP object model.

[0073] For a detailed description of S303, please refer to the above embodiments, which will not be repeated here.

[0074] S304: Receive control messages sent by the application layer and determine the target event handler component that matches the control message.

[0075] The application layer is controlled by the user, such as... Figure 2 As shown, the application layer includes various applications, such as advanced applications, control-related applications, and other applications, without any restrictions.

[0076] In this embodiment, the data processing system, during the processing of data response events, such as... Figure 2 As shown, the data processing system can also receive control messages (control requests) sent by the application layer. Specifically, the control messages sent by the application layer reach the control message gateway of the scheduler through the broker. These control messages are used to control node devices, such as controlling a node device (e.g., a water pump) to open or close a valve, or controlling the start / stop of conveyor belts, coal mining machines, transfer machines, etc., or performing any other possible control on the node devices without restriction.

[0077] Furthermore, the scheduler determines the event handler component that matches the control message, i.e., schedules the control message. The event handler component that matches each control message is called the target event handler component; that is, control messages from different applications need to be processed by different event handler components.

[0078] For example, if control message 1 is used to control node device "EIP Adapter #1" (e.g., a water pump) to stop operating, then the target event handler component corresponding to control message 1 is the event handler component that configures the "#1" CIP object model.

[0079] S305: Cache control messages to the second message queue configured for the target event handler component.

[0080] Furthermore, a message queue can also be introduced during the execution of control messages (control requests) in this embodiment. Specifically, in this embodiment, a message queue can be configured in each event handler component, and this message queue is referred to as the second message queue (corresponding to...). Figure 2 (The "Control Request Queue" in the middle). After receiving control messages from the application layer, each control message can be cached in the second message queue configured for the corresponding target event handler component.

[0081] S306: Each event handler component processes the assigned data response events based on a pre-configured CIP object model to obtain the target data.

[0082] Specifically, during the processing of data response events by each event handler component, it first needs to determine whether the second message queue is empty. If it is empty, the allocated data response events are processed. If the second message queue is not empty, the event handler component interrupts the polling processing of data response events and responds to the control messages cached in the second message queue. That is to say, each event handler component prioritizes the execution of application layer control messages.

[0083] Some embodiments, Figure 4 This is a schematic diagram illustrating the interaction process between the data processing system and the application layer according to embodiments of this disclosure, such as... Figure 4 As shown, "application" refers to the application layer, and "front-end" corresponds to the data processing system in this solution. The application layer first sends a control request (carrying control information) to the data processing system and starts clock timer 1 (the first timer). The data processing system receives the control request sent by the application layer and verifies it to determine whether the request should be executed by the system. If the verification passes, it determines the target event handler component that matches the control message to process the request and sends a request confirmation message (i.e., request confirmation) to the application layer. Further, the application layer has several cases: Case 1: If the application layer receives a request confirmation message that fails to receive (e.g., the result field of the request confirmation message is "fail"), then the control is terminated. Case 2: If the application layer receives a successful request confirmation message (e.g., the result field of the request confirmation message is "fail"), then the control is terminated. If the lt field is "success", the first timer automatically terminates and the second timer (clock timer 2) starts. Case 3: If the application layer does not receive the request confirmation message until the first timer times out, control is considered terminated. After processing the request, the data processing system sends the control response message to the application layer to display the control message execution result and starts the third timer (clock timer). If the third timer times out, the control response information is resent, with a maximum of 3 resentments. The application layer starts the second timer and waits to receive the control response message. If the control response message is not received within the timeout period, control is considered failed. Further, after successfully receiving the control response information, the application layer displays the control result and sends a response confirmation message to the data processing system. The data processing system receives the response confirmation message and ends the processing of the control message. Thus, this embodiment can respond to application layer requests while collecting data.

[0084] S307: Cache the target data corresponding to each data response event to the third message queue in sequence.

[0085] Specifically, such as Figure 2 As shown, in this embodiment, a message queue can be configured in the data storage module. This message queue is referred to as the third message queue (corresponding to...). Figure 2 (Medium task queue).

[0086] In practical applications, as the number of connected node devices increases, data concurrency also increases, and database access becomes more frequent. Therefore, this embodiment configures a connection pool for the data storage module. The connection pool contains multiple database connections used to read and write data. Each database connection can have different states, such as "idle," "in use," or "locked." By establishing a database connection pool and corresponding connection usage management strategies, database connections can be reused efficiently and securely, avoiding the overhead of frequent connection establishment and closure. At system startup, the connection pool pre-creates a portion of idle connections for later use. As concurrent data increases and the number of available idle database connections becomes insufficient, the connection pool automatically creates connections until it reaches its configured maximum total number of connections. Conversely, when the number of idle connections in the connection pool becomes excessive during operation, it will appropriately destroy available connections to bring the number of idle connections to a dynamically stable value.

[0087] In addition, this embodiment also configures a thread pool for the data storage module. The thread pool is configured with multiple threads (Tread), such as Tread#1, Tread#2, etc. When the thread pool is created, the number of worker threads in the pool is preset. The worker threads take tasks from the task queue and execute them.

[0088] In the operation of storing the target data of each node device into the data storage module, the storage process of each target data can be encapsulated as a task object. First, the target data is cached into the third message queue in the form of a data stream.

[0089] S308: Each thread in the thread pool retrieves the target data for a data response event from the third message queue, allocates an idle database connection from the connection pool, and sets the database connection to a locked state.

[0090] Furthermore, a thread is invoked from the thread pool to retrieve the target data of a data response event from the third message queue, that is, a task object is retrieved from the third message queue using a thread; furthermore, a task router is used to allocate an idle database connection from the connection pool for each task object (i.e., the target data), and the state of the database connection is changed from "idle" to locked (i.e., "locked").

[0091] S309: Store the target data to the data storage module based on the database link.

[0092] Furthermore, based on the locked database link, the corresponding target data is stored in the data storage module.

[0093] In some embodiments, after storage in the data storage module, the database connection is released and set to an idle state, awaiting use by the next task. This allows for the recycling of database connections in the connection pool.

[0094] In practical applications, since database connection interruptions in the connection pool are not a high-probability event, the task router (connection selector) does not check the status of a connection when retrieving an idle connection. When a connection interruption occurs, task execution will fail, and an attempt will be made to repair the interruption error when returning the connection resource to the connection pool. In most cases, database reconnection failures will not occur unless the network is unrepairable or the database service crashes. If reconnection fails within the timeout period, it will be attempted again the next time it is retrieved for use. Failed tasks will be permanently deleted and will not be attempted to be put back into the task queue for re-execution. This is because data exchange within the same device is frequent, and the latest data always overwrites the old data; it simply needs to wait for the next data update.

[0095] Therefore, this embodiment uses thread pools, connection pools, and message queues to store data in the data storage module, which can improve data storage efficiency.

[0096] In this embodiment, message data sent by multiple node devices is received and sequentially cached into a first message queue in the form of data response events. Each data response event in the first message queue is assigned an event handler component. Each event handler component processes the assigned data response event based on a pre-configured CIP object model to obtain the target data and store the target data in the data storage module. This enables the various node devices to communicate using the Ethernet / IP protocol and process the data response events using a CIP object model constructed with unified rules, thereby achieving data interconnection and interoperability. Furthermore, this embodiment can utilize message queues to achieve multiplexing and high-concurrency data processing. In addition, this embodiment can respond to application layer requests while collecting data. Moreover, this embodiment uses thread pools, connection pools, and message queues to store data in the data storage module, which can improve data storage efficiency.

[0097] To implement the above embodiments, this disclosure also proposes a high-concurrency data processing device based on Ethernet / IP.

[0098] Figure 5 This is a schematic diagram of a high-concurrency data processing apparatus based on Ethernet / IP provided according to another embodiment of the present disclosure.

[0099] like Figure 5As shown, the Ethernet / IP-based high-concurrency data processing device 50 includes:

[0100] The first receiving module 501 is used to receive message data sent by multiple node devices and cache the message data in the form of data response events into the first message queue in sequence.

[0101] The allocation module 502 is used to allocate each data response event in the first message queue to an event handler component;

[0102] Processing module 503 is used for each event handler component to process the assigned data response events based on a pre-configured CIP object model to obtain the target data; and

[0103] Storage module 504 is used to store the target data into the data storage module.

[0104] In some embodiments, the allocation module 502 is specifically used to: obtain the identity information carried by each data response event and determine the target CIP object model corresponding to the identity information; and allocate the data response event to the event handler component that configures the target CIP object model.

[0105] In some embodiments, the apparatus further includes: a second receiving module, configured to receive a control message sent by the application layer and determine a target event handler component that matches the control message; a first caching module, configured to cache the control message in a second message queue configured for the target event handler component; and a processing module, specifically configured to: when the second message queue is not empty, interrupt the processing of data response events by the event handler component and respond to the control message cached in the second message queue.

[0106] In some embodiments, the apparatus further includes: a second caching module for sequentially caching the target data corresponding to each data response event to a third message queue; and a storage module specifically configured to: call each thread in the thread pool to obtain the target data of a data response event from the third message queue, allocate an idle database connection from the connection pool, and set the database connection to a locked state; and store the target data in the data storage module based on the database connection.

[0107] In some embodiments, the storage module is specifically used to: release the database connection and set the database connection to an idle state.

[0108] In some embodiments, the first receiving module is specifically used for: loading the node information and corresponding CIP object model of the node devices for which data needs to be collected; configuring the node information and CIP object model of each node device in multiple event handler components; and initiating a connection request to each node device, and receiving the data response event sent by each node device when the connection is successful.

[0109] In some embodiments, the second receiving module is specifically configured to: receive a control request carrying a control message sent by the application layer, and if the control request passes verification, determine a target event handler component matching the control message and send a request confirmation message to the application layer; after responding to the control message cached in the second message queue, the module further includes: sending a control response message to the application layer and starting a third timer to enable the application layer to display the execution result of the control message; if the third timer times out, resending the control response information; and receiving a response confirmation message from the application layer; wherein, the application layer starts a first timer when sending the control request; and if the application layer receives a failed request confirmation message, it determines that control is terminated; if the application layer receives a successful request confirmation message, it terminates the first timer and starts a second timer until a control response message is received; if the application layer does not receive a request confirmation message until the first timer times out, it determines that control is terminated.

[0110] In this embodiment, message data sent by multiple node devices is received and sequentially cached into a first message queue in the form of data response events. Each data response event in the first message queue is assigned an event handler component. Each event handler component processes the assigned data response event based on a pre-configured CIP object model to obtain the target data and stores the target data in the data storage module. This enables the various node devices to communicate using the Ethernet / IP protocol and process the data response events using a CIP object model constructed with unified rules, thereby achieving data interconnection and interoperability. Furthermore, this embodiment can also utilize message queues to achieve multiplexing and high-concurrency data processing.

[0111] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0112] To implement the above embodiments, this disclosure also proposes a computer program product that, when executed by an instruction processor, performs a high-concurrency data processing method based on Ethernet / IP as proposed in the foregoing embodiments of this disclosure.

[0113] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 6The electronic device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0114] like Figure 6 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0115] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0116] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0117] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive".

[0118] although Figure 6Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0119] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0120] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0121] The processing unit 16 executes various functional applications by running programs stored in the system memory 28, such as implementing the Ethernet / IP-based high-concurrency data processing method mentioned in the foregoing embodiments.

[0122] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0123] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0124] It should be noted that in the description of this disclosure, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0125] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0126] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0127] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0128] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0129] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0130] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0131] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An Ethernet / IP-based high-concurrency data processing method, characterized in that, The application is applied to a data processing system which communicates with a plurality of node devices based on an Ethernet / IP protocol, and the method comprises the following steps: receiving message data sent by the plurality of node devices through an ESDK protocol stack, and submitting the message data sent by each node device to a dispatcher in the form of a data response event, and caching the data response event to a first message queue configured by the dispatcher in a non-blocking manner by the dispatcher; allocating each data response event in the first message queue to an event processor component, wherein the event processor component is dynamically generated from persistent CIP object model information in a database after the data processing system establishes a connection with the node device, and is uniformly registered to the dispatcher to accept scheduling by the dispatcher; each event processor component performs parsing processing on the allocated data response event based on a preconfigured CIP object model to determine target data to be collected; and storing the target data to a data storage module; The method further comprises the following steps: receiving a control message sent by an application layer, and determining a target event processor component matched with the control message, wherein the control message is used to control the node device; caching the control message to a second message queue configured by the target event processor component; and the event processor component performs processing on the allocated data response event based on a preconfigured CIP object model, comprising: in the case that the second message queue is not empty, the event processor component interrupts processing of the data response event, and responds to the control message cached in the second message queue.

2. The method of claim 1, wherein, The step of allocating each data response event in the first message queue to an event processor component comprises the following steps: obtaining identity information carried by each data response event, and determining a target CIP object model corresponding to the identity information; and allocating the data response event to an event processor component configured with the target CIP object model.

3. The method of claim 1, wherein, After obtaining the target data, the method further comprises the following steps: caching the target data corresponding to each data response event to a third message queue in sequence; and the step of storing the target data to a data storage module comprises the following steps: calling each thread in a thread pool to obtain target data of a data response event from the third message queue, allocating a database link in an idle state from a connection pool, and setting the database link to a locked state; and storing the target data to the data storage module based on the database link.

4. The method of claim 3, wherein, After storing the target data to the data storage module based on the database link, the method further comprises the following steps: releasing the database link, and setting the database link to an idle state.

5. The method of claim 1, wherein, The step of receiving a control message sent by an application layer, and determining a target event processor component matched with the control message comprises the following steps: receiving a control request sent by the application layer and carrying a control message, and in the case that the control request passes verification, determining a target event processor component matched with the control message and sending a request confirmation message to the application layer; The method further comprises: sending a control response message to the application layer and starting a third timer to enable the application layer to display the control message execution result, and resending the control response message if the third timer expires; and receiving a response confirmation message from the application layer; wherein the application layer starts a first timer when sending the control request, and determines that the control is terminated if the application layer receives a failed request confirmation message, and terminates the first timer and starts a second timer until the control response message is received if the application layer receives a successful request confirmation message, and determines that the control is terminated if the application layer does not receive the request confirmation message until the first timer expires.

6. An Ethernet / IP-based high-concurrency data processing device, characterized in that, The data processing system is applied to a data processing system, and the data processing system communicates with a plurality of node devices based on an Ethernet / IP protocol, and comprises: a first receiving module, configured to receive packet data sent by the plurality of node devices through an ESDK protocol stack, and submit the packet data sent by each node device to a dispatcher in the form of a data response event, and cache the data response event to a first message queue configured by the dispatcher in a non-blocking manner by the dispatcher; an allocation module, configured to allocate each data response event in the first message queue to an event processor component, wherein the event processor component is dynamically generated from persistent CIP object model information in a database after the data processing system establishes a connection with the node device, and is uniformly registered to the dispatcher to accept scheduling of the dispatcher; a processing module, configured to parse and process the allocated data response event based on a preconfigured CIP object model by each event processor component to determine target data to be collected; and a storage module, configured to store the target data to a data storage module. The apparatus further comprises: a second receiving module, configured to receive a control message sent by an application layer, and determine a target event processor component matched with the control message, wherein the control message is used to control a node device; a first caching module, configured to cache the control message to a second message queue configured by the target event processor component; and the processing module is specifically configured to interrupt processing of the data response event in the event processor component and respond to the control message cached in the second message queue if the second message queue is not empty.

7. The apparatus of claim 6, wherein, The allocation module is specifically configured to: obtain identity information carried by each data response event, and determine a target CIP object model corresponding to the identity information; and allocate the data response event to an event processor component configured with the target CIP object model.

8. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to execute the method according to any one of claims 1-5. The computer instructions are used to enable the computer to execute the method according to any one of claims 1-5.

Images