Data edge end acquisition system and method, electronic equipment and storage medium

Abstract

The invention provides a data side end acquisition system and method, electronic equipment and a storage medium, the system comprises a configuration module, an acquisition module, a preprocessing module and an external service module, the configuration module is used for configuring metadata, including a target protocol driving type of target equipment, target equipment information and target point location information, the acquisition module is used for loading a target protocol driver according to a target protocol driver type, establishing connection with target equipment through the target protocol driver according to the target equipment information, and performing data acquisition on the target equipment based on the target point location information to obtain original acquisition data; the preprocessing module is used for preprocessing the original collection data to obtain target collection data, and the external service module is used for externally providing the target collection data and / or metadata; the system can flexibly configure metadata, so that protocol drivers or devices are integrated through simple configuration, the system can adapt to diversified protocol drivers and devices, and the flexibility, compatibility and expandability of the system are effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of industrial Internet of Things data collection, and specifically to a data edge collection system, method, electronic device and storage medium. Background Art

[0002] Data collection in the industrial production process has become increasingly important. Through data collection, enterprises can conduct more detailed monitoring and analysis of the operating status of production equipment, product quality, energy consumption, etc., thereby improving production efficiency and reducing costs. However, traditional data collection methods mostly rely on centralized servers or manual operations, which are difficult to meet the modern industry's needs for real-time, accurate and large-scale data processing.

[0003] At present, although there are some data edge collection systems based on the Industrial Internet of Things, they often find it difficult to cope with the diverse needs of industrial sites, especially when collecting data from a variety of equipment types. Problems such as poor equipment compatibility, difficult maintenance, lack of flexibility and scalability often occur. Summary of the invention

[0004] In view of the shortcomings of the prior art mentioned above, the present application provides a data edge acquisition system, method, electronic device and storage medium to solve the technical problems that the above-mentioned existing data edge acquisition systems are difficult to meet the diverse needs of industrial sites, especially when collecting data from a variety of types of equipment, and often have shortcomings such as poor equipment compatibility, difficult maintenance, insufficient flexibility and scalability.

[0005] The present application provides a data edge acquisition system, which includes: a configuration module, which is used to configure metadata, and the metadata includes a target protocol driver type, target device information and target point information of a target device; an acquisition module, which is used to load a target protocol driver according to the target protocol driver type, establish a connection with the target device through the target protocol driver according to the target device information, and perform data acquisition on the target device based on the target point information to obtain original acquired data; a preprocessing module, which is used to preprocess the original acquired data to obtain target acquired data; and an external service module, which is used to provide the target acquired data and / or the metadata to the outside.

[0006] In one embodiment of the present application, the acquisition module includes a driver management unit for configuring and / or changing the protocol driver.

[0007] In one embodiment of the present application, the configuration module includes an input unit, a connection configuration unit, a device configuration unit and a point configuration unit; the input unit is used for the user to input information; the connection configuration unit is used to obtain the target protocol driver type input by the user through the input unit and configure it; the device configuration unit is used to obtain the initial device information input by the user through the input unit, and convert the initial device information into target device information that conforms to the target protocol driver type; the point configuration unit is used to obtain the initial point information input by the user through the input unit, and convert the initial point information into target point information that conforms to the target protocol driver type.

[0008] In one embodiment of the present application, the external service module includes a data open sub-module and a cloud-edge collaboration sub-module: the data open sub-module is used to provide the target collection data and / or the metadata to a terminal, and the terminal includes at least one of a client and a database; the cloud-edge collaboration sub-module is used to upload the target collection data and / or the metadata to the cloud.

[0009] In one embodiment of the present application, the data opening submodule includes at least one of a push unit, a storage unit, an OPC UA server and an external interface; the push unit is used to push the target acquisition data to the client; the storage unit is used to transfer the target acquisition data to the database for storage in a preset storage method; the OPC UA server is used to open access rights to the client so that the client can pull the target acquisition data; the external interface is used to provide a query interface for the client to query the target acquisition data and / or the metadata.

[0010] In one embodiment of the present application, the cloud-edge collaboration sub-module is also used to execute control instructions issued by the cloud, and the control instructions include at least one of data pulling instructions and device control instructions. The data pulling instructions are used to pull the target collection data and / or the metadata, and the device control instructions are used to turn the device on or off.

[0011] In one embodiment of the present application, the original collected data is preprocessed, including: determining a target processing strategy based on the target point information, and processing the original collected data according to the target processing strategy to obtain intermediate collected data, the target processing strategy including at least one of range screening, scaling, and negation; converting the intermediate collected data according to a preset format to obtain the target collected data.

[0012] In one embodiment of the present application, a data edge collection method is also provided, the method comprising: configuring metadata, the metadata comprising a target protocol driver type, target device information and target point information of a target device; loading a target protocol driver according to the target protocol driver type, establishing a connection with the target device through the target protocol driver according to the target device information, and performing data collection on the target device based on the target point information to obtain original collected data; preprocessing the original collected data to obtain target collected data, and providing the target collected data and / or the metadata to the outside.

[0013] In one embodiment of the present application, an electronic device is also provided, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the data edge collection method as described above.

[0014] In one embodiment of the present application, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the data edge collection method as described above.

[0015] Beneficial effects of the present invention: The present invention provides a data edge acquisition system, method, electronic device and storage medium. The system is designed by layering configuration, acquisition, processing and external services. Each module performs its own function in the overall operation of the system, and the collaboration between modules ensures the efficiency and accuracy of the entire process from data configuration, acquisition, processing to external opening. Moreover, through the configuration module, the system can flexibly configure metadata, so that newly added protocol drivers or devices can be integrated through simple configuration, without large-scale changes to the system, to adapt to a variety of protocol drivers and devices, effectively improving the flexibility, compatibility and scalability of the system. In addition, users only need to perform simple configuration through the configuration module to adapt to new equipment or points, without complex code modifications or system reconstruction, thereby effectively reducing the maintenance cost of the system.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of an implementation environment of a data edge acquisition system shown in an exemplary embodiment of the present application;

[0018] Figure 2 is a block diagram of a data edge acquisition system shown in an exemplary embodiment of the present application;

[0019] Figure 3 It is a brief class diagram of a custom TestTcp protocol driver shown in a specific embodiment of the present application;

[0020] Figure 4 It is a functional structure diagram of an industrial Internet of Things data edge collection system shown in a specific embodiment of the present application;

[0021] Figure 5 It is a workflow diagram of an industrial Internet of Things data edge collection system shown in a specific embodiment of the present application;

[0022] Figure 6 is a flow chart of a data edge collection method shown in an exemplary embodiment of the present application;

[0023] Figure 7 It is a structural schematic diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0024] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0025] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0026] It should be noted that in this application, "first", "second", etc. are only used to distinguish similar objects, and are not used to limit the order or precedence of similar objects. The variations of "including", "having", etc. described above indicate that the scope covered by the subject of the word is not exclusive except for the examples shown by the word.

[0027] It is understood that the various numbers, step numbers, and other reference numerals recorded in this application are distinguished for the convenience of description and are not intended to limit the scope of this application. The size of the reference numerals in this application does not mean the order of execution. The execution order of each process should be determined by its function and internal logic.

[0028] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0029] It should be noted that most existing data acquisition systems have fixed and limited protocol driver types for data collection, which makes it impossible to meet a wide range of collection needs, let alone support customized collection needs. When facing the data collection and management of a large number of diverse devices, its stability and data processing capabilities are difficult to guarantee. Most of them lack flexible expansion architecture and cannot adapt to the ever-changing needs of industrial sites. At the same time, there are delays and low processing efficiency when processing large-scale data, which affects the overall production optimization and management decisions.

[0030] To solve these problems, the embodiments of the present application respectively propose a data edge collection system, a data edge collection method, an electronic device, a computer-readable storage medium and a computer program product, which will be described in detail below.

[0031] See also Figure 1 , Figure 1 It is a schematic diagram of an implementation environment of a data edge collection system shown as an exemplary embodiment of the present application.

[0032] like Figure 1 As shown, the implementation environment may include industrial equipment 110, edge 120, terminal 130 and cloud 140, wherein the industrial equipment 110 may be an industrial PLC, an intelligent device, a precision sensor, or a third-party system, the edge 120 may be at least one computer device among microcomputers, embedded computers, neural network computers, etc., or an independent physical server, or a server cluster or distributed system composed of multiple physical servers, the terminal 130 may be a client such as a mobile phone, a tablet, a computer, a wearable device, or a database such as iotDb, influxDb, mysql, sqlite, etc., and the cloud 140 may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, which are not limited here. The edge 120 is deployed on the production line formed by the industrial equipment 110 to configure metadata, collect data from the industrial equipment 110, pre-process the collected data, and provide the collected data and / or metadata to the terminal 130 and the cloud 140.

[0033] Schematically, the edge 120 configures metadata, wherein the metadata includes a target protocol driver type, target device information, and target point information of a target device in the industrial device 110, loads a target protocol driver according to the target protocol driver type, establishes a connection with the target device through the target protocol driver according to the target device information, performs data collection on the target device based on the target point information to obtain original collected data, and pre-processes the original collected data to obtain target collected data, so as to provide target collected data and / or metadata to the terminal 130 and / or the cloud 140.

[0034] It should be noted that the edge data collection system provided in the embodiment of the present application is generally arranged in the edge 120 , and the edge data collection method is generally executed by the edge 120 .

[0035] See also Figure 2 , Figure 2 is a block diagram of a data edge acquisition system shown in an exemplary embodiment of the present application. The system can be applied to Figure 1 The implementation environment shown is specifically configured in the edge 120. The system can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the system is applicable.

[0036] like Figure 2 As shown, the exemplary edge data collection system includes:

[0037] The configuration module 210 is used to configure metadata, and the metadata includes the target protocol driver type, target device information and target point information of the target device; the acquisition module 220 is used to load the target protocol driver according to the target protocol driver type, establish a connection with the target device through the target protocol driver according to the target device information, and collect data from the target device based on the target point information to obtain the original collected data; the preprocessing module 230 is used to preprocess the original collected data to obtain the target collected data; the external service module 240 is used to provide the target collected data and / or metadata to the outside.

[0038] In one embodiment of the present application, the configuration module 210 can be a software component, a hardware chip, or an operating terminal such as a mobile phone or a computer. The user creates the current data acquisition task through the configuration module 210, and performs metadata configuration on the current data acquisition task, including configuring the target protocol driver type, target device information, and target point information of the target device. The target device refers to the industrial equipment targeted by the current data acquisition task and from which data needs to be obtained, that is, the industrial equipment designated as the data source in the current data acquisition task, wherein the industrial equipment can be an industrial PLC (Programmable Logic Controller), a smart device, a precision sensor, or a third-party system. The target protocol driver type is also the protocol driver type of the target device, and the protocol driver type refers to the type of industrial protocol data acquisition driver. The industrial protocol data acquisition driver, also known as a protocol driver or a protocol driver package, refers to a driver for realizing data acquisition and communication of industrial equipment. The target device information is also the device information of the target device, including at least one of the parameters such as device name, device type, IP (Internet Protocol) address, and port number. The target point information is also the point information of the point to be sampled in the target device, including at least one of the parameters such as the point address, data type, point name, etc., wherein the point refers to a certain type of data on the industrial equipment, for example: carbon monoxide or carbon dioxide in environmental protection collection equipment.

[0039] The acquisition module 220 can be a software component or a control chip such as MCU (Microcontroller Unit), ECU (Electrical Control Unit), etc., which communicates with industrial equipment and collects data from industrial equipment by running protocol drivers. Protocol drivers of different protocol driver types can be configured in advance in the acquisition module 220. After the configuration module 210 completes the metadata configuration, the acquisition module 220 matches the target protocol driver from a variety of protocol drivers according to the target protocol driver type, so as to load and run the target protocol driver, so that the target protocol driver uses the target device information to establish a communication connection with the target device, and requests the target device to obtain data based on the target point information, or monitors the data of the target device based on the target point information, so as to realize data collection and analysis and obtain the original collected data.

[0040] The preprocessing module 230 can be a software component or a processor such as a DSP (Digital Signal Processor) or a CPU (Central Processing Unit). The preprocessing module 230 preprocesses the original acquisition data collected by the acquisition module 220, including at least one of data cleaning, scaling processing, normalization processing, etc., to obtain target acquisition data.

[0041] The external service module 240 can be a software component such as an API (Application Programming Interface) or a message queue, or it can be a hardware device such as a network interface card, a wireless communication device, or a fiber optic transceiver. The external service module 240 transmits data and / or metadata to an external target such as the cloud or a terminal.

[0042] In one embodiment of the present application, the acquisition module 220 includes a driver management unit for configuring and / or changing the protocol driver.

[0043] In this embodiment, a driver management unit is built into the acquisition module 220, and the configuration and modification of the protocol driver are performed through the driver management unit, and the custom protocol driver integration is opened, which supports users to develop protocol driver packages according to specific needs and upload them to the driver management unit, so as to realize the collection of various unconventional industrial data. Schematically, the data edge acquisition system may also include an SDK (Software Development Kit), and users can develop new drivers according to their own needs and through the SDK to integrate them into the data edge acquisition system for use, thereby effectively solving the complex and changeable data acquisition needs. Of course, the SDK can also be independent of the data edge acquisition system. After the user completes the development of the custom protocol driver through the SDK, the custom protocol driver can be integrated into the system for use by uploading, which is not limited here.

[0044] In addition, the acquisition module 220 also includes a protocol driver library for storing protocol drivers of various types, such as Modbus, OPC UA, OPC DA, Siemens S7, IEC 104, AB Micro 800 and other industry standard protocol drivers and driver jar packages independently developed by users (i.e., custom protocol drivers).

[0045] Schematically, in terms of implementing custom protocol driver integration, the acquisition module 220 uses a reflection mechanism to load and manage the driver jar package. By dynamically loading the driver jar package, the reflection technology is used to create an instance of the driver class at runtime, and its method is called to perform the data acquisition task. By adopting a reflection mechanism to load and manage the driver jar package, the data edge acquisition system of the present application has significant advantages in flexibility, decoupling, maintainability, and availability, and can better meet the diverse data acquisition needs in the industrial Internet of Things environment, as follows:

[0046] 1. High flexibility: Users can develop different protocol driver packages according to actual needs, and the system can seamlessly load and use these driver packages without modifying the core code of the system. This provides great convenience for system expansion and upgrade.

[0047] 2. Strong decoupling: Through the reflection mechanism, the system and the specific protocol driver are loosely coupled. The system does not need to know the specific driver class at compile time, but only needs to dynamically load and call the driver class according to the configuration information at runtime, which reduces the dependency between the system and the driver and improves the maintainability of the system.

[0048] 3. Easy to maintain: When a protocol driver needs to be updated or replaced, you only need to replace the corresponding jar package without recompiling the entire system. This simplifies system maintenance and reduces maintenance costs.

[0049] 4. Support hot loading: The system can realize hot loading of driver jar packages, that is, when the driver jar package is updated during system operation, the system can automatically identify and load the new driver package without restarting the system. This improves the system availability and user experience.

[0050] As the core component of the data edge acquisition system, the acquisition module 220 focuses on efficient data acquisition and fine message parsing tasks. It fully loads the various information configured in the configuration module 210, and flexibly retrieves the required protocol drivers from the protocol driver library, seamlessly connecting various data sources. By building a stable communication link, it can freely send and receive communication messages, and then use its powerful parsing ability to convert the received message data into a unified format to ensure the smooth delivery of the data stream to the preprocessing module 230. At the same time, the acquisition module 220 can not only have a rich and diverse protocol driver built in to support data acquisition of various protocol driver types, but also creatively open up the protocol driver management function, giving users unprecedented flexibility, so that users can independently develop exclusive protocol driver jar packages according to actual needs, and easily upload them to this system to achieve a plug-and-play convenient experience, which not only enhances the user's customization capabilities and greatly improves the system's protocol scalability, but also ensures its wide compatibility with diverse communication protocols, highlighting the flexibility and adaptability of the system, so that the system can cope with more diversified acquisition scenarios and changing application needs.

[0051] In one embodiment of the present application, a connection is established with a target device according to target device information through a target protocol driver, and data is collected from the target device based on target point information, including: under the condition that the target protocol driver is a custom protocol driver, the custom protocol driver initializes a communication pipeline according to the target device information and listens to the connection request of the target device; if the connection request of the target device is listened to, a communication link is established to receive data sent by the target device through the established communication link; the received data is decoded by a message decoder to obtain an independent data packet, and data is extracted from the independent data packet according to the target point information to obtain the original collected data. This embodiment realizes data collection through a monitoring mode, wherein the target device can be an industrial device in a production line.

[0052] In another embodiment of the present application, a connection is established with a target device according to target device information through a target protocol driver, and data is collected from the target device based on target point information, including: under the condition that the target protocol driver is a custom protocol driver, the custom protocol driver generates a data pull request according to the target point information, and converts it into a message that conforms to the protocol format corresponding to the custom protocol driver; according to the target device information, the converted message is sent to the target device, so that the target device returns the original collected data. This embodiment realizes data collection through a pull mode, wherein the target device can be an external third-party system or server.

[0053] For example, the following describes the workflow of a custom protocol driver using a user-defined TestTcp protocol driver. The custom communication protocol Test is based on the common TCP communication method and has a specific message format. The main function of the TestTcp protocol driver is to implement data collection and data distribution through the Test protocol. Figure 3 , Figure 3 This is a brief class diagram of a custom TestTcp protocol driver shown in a specific embodiment of the present application. Figure 3 As shown in the figure, in the TestTcp protocol driver, the core main class is TestTcpSocketGateway, which is the startup entry of the acquisition gateway and the core entry of data delivery. The message channel initializer TestTcpChannelInitializer of data acquisition is responsible for initializing the communication channel to ensure the smooth establishment of the data acquisition link. In terms of message processing, TestTcpFrameDecoder is used as a message decoder to handle the TCP sticky packet problem and ensure that each received message can be correctly parsed. The socket server session is used to monitor and respond to incoming message requests and is responsible for the message monitoring and processing tasks of the communication link. The TestTcpSession class is further responsible for encoding the response message to ensure that the response data is correctly returned according to the protocol. In data delivery, TestTcpConverterUtils plays a key role. It is responsible for converting the sent data into a message format that conforms to the Test protocol format to ensure the correctness of the message format. In addition, the data delivery process also involves the use of the Socket client, which is responsible for sending the converted data to the target device. Through this series of process designs, the TestTcp protocol driver realizes the complete process from data acquisition, message processing to data delivery, demonstrating the application method and flexibility of custom protocol drivers in the system.

[0054] In one embodiment of the present application, the configuration module 210 includes an input unit, a connection configuration unit, a device configuration unit and a point configuration unit; the input unit is used for the user to input information; the connection configuration unit is used to obtain the target protocol driver type input by the user through the input unit and configure it; the device configuration unit is used to obtain the initial device information input by the user through the input unit, and convert the initial device information into target device information that conforms to the target protocol driver type; the point configuration unit is used to obtain the initial point information input by the user through the input unit, and convert the initial point information into target point information that conforms to the target protocol driver type.

[0055] In this embodiment, the configuration module 210 provides detailed configuration information for the data acquisition task, and its structure is divided into three core components: connection configuration, device configuration, and point configuration, and each part presents a clear parent-child hierarchical relationship. In the connection configuration stage, the user can select an appropriate protocol driver and input the target protocol driver type of the target device through the input unit, so that the connection configuration unit configures the target protocol driver type and establishes the communication basis for data acquisition. Subsequently, the device configuration stage is entered under the determined connection configuration, and the user can input the initial device information of the target device through the input unit, so that the device configuration unit dynamically generates a device configuration information form that conforms to the target protocol driver type according to the target protocol driver type determined in the connection configuration, as the target device information, thereby recording the key parameters such as the IP address, port number, and device type of the target device in detail. Finally, the point configuration stage goes deep into the device, and the user inputs the initial point information of the target device through the input unit, so that the point configuration unit dynamically generates a point configuration information form that conforms to the target protocol driver type according to the target protocol driver type determined in the connection configuration, as the target point information, to accurately specify the point address, data type, and point name of the point to be collected, and ensure the accuracy and efficiency of data acquisition.

[0056] Indicatively, the device configuration information form format corresponding to each protocol drive type can be pre-set in the device configuration unit, so that the device configuration unit can dynamically generate the device configuration information form based on the initial device information according to the device configuration information form format corresponding to the target protocol drive type. Similarly, the point configuration information form format corresponding to each protocol drive type can be pre-set in the point configuration unit, so that the point configuration unit can dynamically generate the point configuration information form based on the initial point information according to the point configuration information form format corresponding to the target protocol drive type.

[0057] This embodiment achieves a high degree of intelligence and customization by dynamically generating all configuration information forms according to the selected protocol driver. Once the user selects a specific protocol driver by determining the protocol driver type in the connection configuration, the device configuration information form and point configuration information form under it will be immediately adjusted to adapt to the specific requirements of the selected protocol driver. This not only significantly improves the flexibility and adaptability of the configuration module, but also provides great convenience for its subsequent expansion and upgrade, ensuring that the system can stably and efficiently cope with various complex data acquisition tasks.

[0058] In one embodiment of the present application, the original collected data is preprocessed, including: determining a target processing strategy based on target point information, and processing the original collected data according to the target processing strategy to obtain intermediate collected data, the target processing strategy including at least one of range screening, scaling, and negation; converting the intermediate collected data according to a preset format to obtain target collected data.

[0059] In this embodiment, the original collected data belonging to different points have different processing methods. Therefore, the preprocessing algorithm corresponding to each point can be pre-configured in the preprocessing module 230 as the processing strategy corresponding to each point, so that the preprocessing module 230 can determine the target preprocessing algorithm, that is, the target processing strategy, from the preprocessing algorithms corresponding to each point according to the point name in the target point information, use the target preprocessing algorithm to process the original collected data to obtain intermediate collected data, and convert the format of the intermediate collected data to obtain target collected data that conforms to the preset format.

[0060] For example, the preprocessing module 230 first checks whether the original collected data meets the upper and lower limit requirements, i.e., the range, specified in the target preprocessing algorithm, and determines whether to retain the data beyond the range according to the target preprocessing algorithm. Next, the preprocessing module 230 determines whether the target preprocessing algorithm requires negation of the value, and if so, the data is negated. Finally, according to the scaling type specified in the target preprocessing algorithm, such as linear scaling or square root scaling, the data is scaled accordingly.

[0061] In this embodiment, the preprocessing module 230 performs preprocessing operations on the original collected data according to the preprocessing algorithms corresponding to different points, thereby ensuring that the original collected data can be converted according to established rules, providing accurate and consistent data input for subsequent data processing and analysis.

[0062] In one embodiment of the present application, the external service module 240 includes a data open sub-module and a cloud-edge collaboration sub-module: the data open sub-module is used to provide the target collected data and / or metadata to the terminal, and the terminal includes at least one of a client and a database; the cloud-edge collaboration sub-module is used to upload the target collected data and / or metadata to the cloud.

[0063] In this embodiment, the data open sub-module can actively push the target collection data and / or metadata to the client, or the client can pull and query the target collection data and / or metadata through the data open sub-module according to demand. The data open sub-module can also store the target collection data and / or metadata in an external database, providing a variety of data usage methods.

[0064] The cloud-edge collaboration submodule is mainly used to realize the efficient interaction of data between the edge data collection system and the cloud. As the edge, it is responsible for reporting the target collection data and / or metadata to the cloud for aggregation, so that the cloud can manage the edge data and realize the ability of cloud-edge collaboration. Schematically, the cloud-edge collaboration submodule can include an MQTT client. After configuring the connection information and authentication information of the cloud, the cloud-edge collaboration submodule establishes a two-way communication connection with the MQTT Broker on the cloud through the built-in MQTT client to ensure real-time data transmission and interaction. The cloud-edge collaboration submodule can monitor the metadata. When the metadata changes, the cloud-edge collaboration submodule can actively report the updated metadata and the latest target collection data to the cloud, ensuring the cloud's real-time control of the status and data of industrial equipment.

[0065] In one embodiment of the present application, the data opening submodule includes at least one of a push unit, a storage unit, an OPC UA server and an external interface; the push unit is used to push the target collection data to the client according to a preset storage method; the storage unit is used to transfer the target collection data to a database for storage; the OPC UA server is used to open access rights to the client so that the client can pull the target collection data; the external interface is used to provide a query interface for the client to query the target collection data and / or metadata.

[0066] In this embodiment, the data opening submodule mainly provides a variety of data external use methods, supporting push, database storage, external API, and open data through OPC UA server. Among them, the push unit can be a message middleware such as RocketMQ, KAFKA, MQTT, etc., which pushes the target collected data to the client and other third parties for use. The push unit also supports push according to the specified point. The push strategy can be selected in real time or according to the specified scanning cycle, and supports custom push templates and message order management.

[0067] The storage unit supports storing the target collected data in an external database in a specified database and a specified storage method. Schematically, multiple database connection methods can be built into the storage unit, including IoTDB, InfluxDB, MySQL, and SQLite, and support the expansion of more database types. The storage unit can determine the target database according to the point to which the target collected data belongs to realize the storage of the specified database; multiple storage methods can also be built into the storage unit. After configuring the target database connection information and successfully connecting, the target collected data can also be selectively stored according to the storage method corresponding to the point. For example, the target collected data can be stored outside the preset fluctuation range through the threshold dead zone storage method, which can greatly save data storage space. The storage unit can also optimize the storage structure for time series databases and relational databases. Specifically, a wide table structure is used for time series databases to reduce associations by increasing data redundancy and facilitate query; a narrow table structure is used for relational databases to reduce data redundancy. The data expiration time can also be set through the storage unit so that the database can clear the stored data according to the data expiration time to optimize the storage space management of the database.

[0068] The OPC UA (OLE for Process Control Unified Architecture) server is built into the data edge collection system and supports server endpoint management, security policy settings, user permission management, and maximum connection number management. As a data service end, it opens access rights to multiple clients and other third-party systems. Clients and other third-party systems can connect to the data edge collection system through the OPC UA protocol to obtain target collection data and / or metadata. Specifically, clients and other third-party systems can configure the required collection point information according to needs and pull data on demand.

[0069] The external interface is also called the external API. Through the external API, a rich data query interface is provided for the client and other third-party systems to select and call as needed. Indicatively, the external API can provide historical collection data query interface, the latest collection data query interface, data delivery interface, metadata query interface, etc.

[0070] In one embodiment of the present application, the cloud-edge collaboration submodule is also used to execute control instructions issued by the cloud, the control instructions include at least one of data pulling instructions and device control instructions, the data pulling instructions are used to pull target collection data and / or metadata, and the device control instructions are used to turn the device on or off.

[0071] In this embodiment, the cloud-edge collaboration submodule can receive control from the cloud to enable the cloud to achieve active control capabilities. The cloud can send control instructions to pull the specified data of the edge in real time or perform remote operations to perform data analysis, device management or fault diagnosis. In addition, the cloud-edge collaboration submodule also supports the breakpoint resume function to ensure that when the network is unstable or interrupted, the data can continue to be transmitted after the connection is restored, ensuring the integrity and consistency of the data.

[0072] See also Figure 4 , Figure 4 FIG. 1 is a functional structure diagram of an industrial Internet of Things data edge collection system shown in a specific embodiment of the present application. Figure 4 As shown, the industrial Internet of Things data edge collection system includes a metadata configuration module, a data collection module, a data preprocessing module, a data opening module and a cloud-edge collaboration module. Among them, the metadata configuration module is used to collect the necessary metadata configuration such as protocol driver type, device type, point address, point data type, etc., which can be divided into connection configuration, device configuration and point configuration; the data acquisition module connects to data sources such as industrial equipment through metadata loading and protocol driver loading on the basis of the former configuration, collects corresponding data and parses it. Among them, the data acquisition module can configure multiple protocol drivers such as Modbus, OPC UA, OPC DA, IEC 104, Siemens S7 and driver jar packages developed by users themselves; the data preprocessing module can configure multiple preprocessing algorithms. After the data acquisition module collects data, it preprocesses the original collected data according to the preprocessing algorithm, including scaling, upper and lower limit, negation and other operations to achieve data filtering and conversion; the data opening module mainly provides the use of target collected data to the outside in various forms, including pushing data through message queues such as RocketMQ, KAFKA, MQTT, specifying IoTDB, InfluxDB, MySQL and SQLite and other databases to store data, providing data to the outside through API, and providing data through OPC UA server opens data in other forms; the cloud-edge collaboration module acts as the edge to configure cloud platform (cloud) information, authenticate and connect with the cloud, realize cloud-edge data interaction, and report metadata and target collection data to the cloud.

[0073] The input of the industrial IoT data edge acquisition system is the connection information, equipment information, point information and other related metadata configurations of industrial equipment. The output is the collected data after multiple preprocessing, and provides a variety of output methods. Each module can interact with data through a standardized interface to ensure the efficient transmission and processing of data within the system. Each module performs its duties in the overall operation of the system, and the collaboration between modules ensures the efficiency and accuracy of the entire process from data acquisition, processing to opening. In addition, the system has a high degree of flexibility and scalability, and can quickly adjust and expand functions according to different industrial site needs, and adapt to a variety of industrial protocol drivers and equipment. At the same time, the system adopts a layered design, so that new protocol drivers or devices can be integrated through simple configuration without large-scale changes to the existing system. The system can not only operate stably in a complex industrial environment, but also quickly respond to the dynamic changes of industrial equipment, provide real-time data support, and greatly improve the intelligence level of production management and data utilization efficiency.

[0074] It should be understood that the industrial Internet of Things data edge collection system is an example of a data edge collection system, and accordingly, the metadata configuration module is an example of the configuration module 210, the data collection module is an example of the collection module 220, the data preprocessing module is an example of the preprocessing module 230, the data opening module is an example of a data opening sub-module, and the cloud-edge collaboration module is an example of a cloud-edge collaboration sub-module.

[0075] See also Figure 5 , Figure 5 FIG. 1 is a flowchart of an industrial Internet of Things data edge collection system shown in a specific embodiment of the present application. Figure 5 As shown in the figure, some modules of the industrial Internet of Things data edge collection system are sequentially associated, and some modules are independent of each other. The workflow of the industrial Internet of Things data edge collection system is as follows:

[0076] 1. The user configures the connection information, device information and point information of the collected data through the metadata configuration module. After the configuration is completed, the metadata configuration module transmits the collected information, namely metadata, to the data collection module and the cloud-edge collaboration module.

[0077] 2. The data acquisition module loads the corresponding protocol driver according to the above configured acquisition information, establishes a connection with data sources such as industrial equipment through the protocol driver, and performs data acquisition to transmit the collected data, i.e., the original collected data, to the data preprocessing module. Among them, the data acquisition module can collect data in two ways: monitoring mode and pull mode. Specifically, when the industrial Internet of Things data edge acquisition system acts as a server, the data acquisition module can use the monitoring mode for data acquisition, for example, using the TCP protocol as a server to monitor data requests from the client, and receiving data streams in real time as original collected data; when the industrial Internet of Things data edge acquisition system acts as a client, the data acquisition module can use the pull mode for data acquisition, for example, communicating with Siemens equipment through the Siemens S7 protocol, and the data acquisition module sends a request message containing the collection point information to the Siemens equipment. After receiving the request, the Siemens equipment feeds back the required point data to the data acquisition module as original collected data.

[0078] 3. The data preprocessing module screens, cleans and processes the collected data according to the preprocessing algorithm configured in advance by the user to ensure the accuracy and availability of the data, and provides the formatted collected data obtained after processing, namely the target collected data, to the data opening module and the cloud-edge collaboration module for use.

[0079] 4. Both the data open module and the cloud-edge collaboration module rely on the above modules, but the two are not interdependent. The data open module transfers data to a third party in different ways according to the configuration. Specifically, the data open module supports four independently running data open methods, namely: data push, designated database storage, open query through external API, and open data through OPC UA server. Among them, push and designated database storage can be carried out by creating independent tasks. The tasks run independently and do not interfere with each other. Once a new push task or storage task is started, the formatted collected data will be sent to the specified target according to the task requirements; the external API provides a rich data query interface, responds to the client's query request, and provides formatted collected data. As a data service end, the OPC UA server opens access rights to multiple clients. The client can configure the required collection point information according to demand and pull data on demand. The cloud-edge collaboration module is responsible for realizing cloud-based synchronization and collaborative processing of data. It connects to the cloud platform by configuring connection information and reports metadata and collected data to the cloud. The data reporting of the cloud-edge collaboration module is divided into two modes: active reporting and passive reporting. In the active reporting mode, the cloud-edge collaboration module dynamically synchronizes the updates of the target collected data and metadata to the cloud. In the passive reporting mode, the cloud initiates a data synchronization request. After receiving the request, the cloud-edge collaboration module uploads the specified data to the cloud, thereby realizing close collaboration and data interaction between the edge and the cloud.

[0080] One of the core competitiveness of this industrial Internet of Things data edge collection system lies in its convenient and fast deployment method. The system is designed to be highly flexible and supports multiple cross-platform deployment modes. It can be easily installed on Windows systems and can also run efficiently in Linux environments. It is worth mentioning that the system does not need to rely on any external middleware, which greatly simplifies the deployment process. On the Windows platform, the system also provides a one-click installation package, and users can complete the deployment with simple operations without complex configuration or dependent installation. This design greatly reduces the operation and maintenance costs and technical barriers, allowing the system to quickly adapt to various industrial site environments and meet the diverse needs of different companies.

[0081] See also Figure 6 , Figure 6 is a flow chart of a data edge collection method shown in an exemplary embodiment of the present application. The data edge collection method can be applied to Figure 1 The implementation environment shown in FIG. 1 is specifically implemented by the edge 120 in the implementation environment. It should be understood that the data edge collection method can also be applied to other exemplary implementation environments and be specifically implemented by devices in other implementation environments. This embodiment does not limit the implementation environment to which the data edge collection method is applicable. Figure 6 As shown, in an exemplary embodiment, the data edge collection method includes at least steps S610 to S630, which are described in detail as follows:

[0082] Step S610: configure metadata.

[0083] Step S620, loading the target protocol driver according to the target protocol driver type, establishing a connection with the target device according to the target device information through the target protocol driver, and performing data collection on the target device based on the target point information to obtain original collected data.

[0084] Step S630, pre-processing the original collected data to obtain target collected data, and providing the target collected data and / or metadata to the outside.

[0085] In step S610, the metadata includes the target protocol driver type, target device information and target point information of the target device, wherein the target protocol driver type can be one of the industry standard protocol drivers such as Modbus, OPC UA, OPC DA, Siemens S7, IEC 104, AB Micro 800, and the type of driver jar package independently developed by the user; the target device information includes at least one of the device name, device type, IP address, port number, etc. of the target device; the target point information refers to the point information of the point to be sampled in the target device, including at least one of the point address, data type, point name, etc.

[0086] In step S620, the target protocol driver can be matched from a plurality of pre-set protocol drivers according to the target protocol driver type, and the target protocol driver can be loaded and run so that the target protocol driver can establish a communication connection with the target device using the target device information, and request data from the target device based on the target point information, or monitor the data of the target device based on the target point information, to realize data collection and analysis and obtain the original collected data.

[0087] In step S630, the original collected data may be subjected to at least one of pre-processing operations such as data cleaning, scaling, and normalization to obtain target collected data, and the target collected data and / or metadata may be transmitted to the cloud, terminal, and other external devices through software components such as APIs and message queues, or hardware devices such as network interface cards, wireless communication devices, and optical fiber transceivers.

[0088] It can be seen that this edge data collection method can flexibly configure metadata, so that newly added protocol drivers or devices can be integrated through simple configuration, which can adapt to a variety of protocol drivers and devices, effectively improve the flexibility, compatibility and scalability of edge data collection, and provide strong support for the intelligent management of industrial enterprises. In addition, users only need to make simple configurations to adapt to new devices or points, without complex code modifications or system reconstruction, thereby effectively reducing maintenance costs.

[0089] It should be noted that the data edge collection method provided in the above embodiment and the data edge collection system provided in the above embodiment belong to the same concept, and the implementation process of each step has been described in detail in the system embodiment and will not be repeated here.

[0090] This embodiment also provides an electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the data edge collection method provided in the above-mentioned embodiments.

[0091] See also Figure 7 , Figure 7 is a schematic diagram of a structure of an electronic device shown in an exemplary embodiment of the present application. It should be noted that: Figure 7 The electronic device 700 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0092] like Figure 7As shown, the electronic device 700 includes a processor 701, a memory 702 and a communication bus 703; the communication bus 703 is used to connect the processor 701 and the memory 702; the processor 701 is used to execute the computer program stored in the memory 702 to implement one or more methods in the above embodiments.

[0093] This embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor of a computer, the computer executes the data edge acquisition method as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0094] This embodiment also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the data edge collection method provided in each of the above embodiments.

[0095] The electronic device provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run the computer program so that the electronic device executes each step of the above method.

[0096] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0097] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0098] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the execution includes the steps of the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM (read-only memory), RAM (random access memory), magnetic disk or optical disk and other media that can store program codes.

[0099] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A data edge acquisition system, characterized in that: The system comprises: A configuration module, used to configure metadata, wherein the metadata includes a target protocol driver type, target device information, and target point information of a target device; An acquisition module, used for loading a target protocol driver according to the target protocol driver type, establishing a connection with the target device according to the target device information through the target protocol driver, and performing data acquisition on the target device based on the target point information to obtain original acquisition data; A preprocessing module, used for preprocessing the original collected data to obtain target collected data; The external service module is used to provide the target acquisition data and / or the metadata to the outside.

2. The data edge acquisition system according to claim 1, characterized in that: The acquisition module includes a driver management unit for configuring and / or changing the protocol driver.

3. The data edge acquisition system according to claim 1, characterized in that: The configuration module includes an input unit, a connection configuration unit, a device configuration unit and a point configuration unit; The input unit is used for the user to input information; The connection configuration unit is used to obtain the target protocol driver type input by the user through the input unit and configure it; The device configuration unit is used to obtain the initial device information input by the user through the input unit, and convert the initial device information into target device information that conforms to the target protocol driver type; The point configuration unit is used to obtain the initial point information input by the user through the input unit, and convert the initial point information into target point information that conforms to the target protocol drive type.

4. The data edge acquisition system according to claim 1, characterized in that: The external service module includes a data opening submodule and a cloud-edge collaboration submodule: The data opening submodule is used to provide the target acquisition data and / or the metadata to a terminal, wherein the terminal includes at least one of a client and a database; The cloud-edge collaboration submodule is used to upload the target collection data and / or the metadata to the cloud.

5. The data edge acquisition system according to claim 4, characterized in that: The data opening submodule includes at least one of a push unit, a storage unit, an OPC UA server and an external interface; The push unit is used to push the target collection data to the client; The storage unit is used to transfer the target acquisition data to the database for storage according to a preset storage method; The OPC UA server is used to open access rights to the client so that the client can pull the target collection data; The external interface is used to provide a query interface for the client to query the target acquisition data and / or the metadata.

6. The data edge acquisition system according to claim 4, characterized in that: The cloud-edge collaboration submodule is also used to execute control instructions issued by the cloud, and the control instructions include at least one of data pulling instructions and device control instructions. The data pulling instructions are used to pull the target collection data and / or the metadata, and the device control instructions are used to turn the device on or off.

7. The data edge acquisition system according to claim 1, characterized in that: Preprocessing the original collected data includes: Determine a target processing strategy according to the target point information, and process the original collected data according to the target processing strategy to obtain intermediate collected data, wherein the target processing strategy includes at least one of range screening, scaling, and negation; The intermediate collected data is converted according to a preset format to obtain the target collected data.

8. A data edge collection method, characterized in that: The method comprises: Configuration metadata, the metadata including target protocol driver type, target device information and target point information of the target device; Loading a target protocol driver according to the target protocol driver type, establishing a connection with the target device through the target protocol driver according to the target device information, and performing data collection on the target device based on the target point information to obtain original collected data; The original collected data is preprocessed to obtain target collected data, and the target collected data and / or the metadata are provided externally.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the data edge collection method as described in claim 8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is enabled to execute the data edge collection method as described in claim 8.

Images