Industrial data acquisition method and system based on OPC

Abstract

The invention discloses an OPC-based industrial data acquisition method and system. The method comprises the steps of determining each industrial device needing data acquisition; communication protocols of all industrial devices are integrated into a unified OPC server through an OPC data fusion module, and point location information needing to be collected is set through a configuration list of data point locations; configuring connection between the OPC server and each industrial device, and allowing each industrial device to scan from the OPC server and obtain corresponding industrial data; a real-time data processing module is used for preprocessing the obtained industrial data and storing the processed industrial data in a specified database; and providing a data access interface to allow other systems or applications to access and acquire the stored industrial data through the interface. Through the method, the difficulty of industrial data acquisition caused by protocol diversity of industrial equipment is solved, and the effectiveness and usability of production data are improved.

Description

Technical Field

[0001] The present invention relates to the field of industrial information technology, and in particular to an industrial data acquisition method and system based on OPC. Background Art

[0002] In the field of industrial automation, data collection is the basis for realizing intelligent control. In the process of product production, manufacturing enterprises must collect, process and monitor the data of the production process in order to ensure the efficiency and stability of product production. A fast and reliable data collection system is a strong guarantee to ensure that manufacturing enterprises can work stably during production.

[0003] Due to the increasing complexity of industrial equipment, the communication protocols used by the equipment are diverse, which increases the difficulty of managing the data of equipment using different communication protocols in the same factory. For example, programmable logic controllers, distributed control systems, robots, etc. are common control devices. They may use different communication protocols, which makes data collection and management complicated. In existing industrial control systems, due to the variety of communication protocols used by equipment manufacturers, data collection work faces compatibility and efficiency issues.

[0004] In addition, with the expansion of production scale and the improvement of intelligent demand, the real-time and accuracy requirements for data collection are becoming higher and higher. Therefore, how to achieve data integration and communication between different devices and communication protocols is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0005] In order to solve the above technical problems, this application proposes the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides an industrial data collection method based on OPC, comprising:

[0007] Identify the industrial equipment that needs to collect data;

[0008] The communication protocols of various industrial devices are integrated into a unified OPC server through the OPC data fusion module, and the point information to be collected is set using the configuration list of data points;

[0009] Configure the connection between the OPC server and each industrial device, allowing each industrial device to scan and obtain the corresponding industrial data from the OPC server;

[0010] Use the real-time data processing module to pre-process the acquired industrial data and store the processed industrial data in the specified database;

[0011] A data access interface is provided to allow other systems or applications to access and obtain stored industrial data through the interface.

[0012] In a possible implementation, the determining of each industrial device for which data collection is required includes: determining a data collection target, device status data, and device parameter data; and determining each industrial device according to the data collection target, device status data, and device parameter data.

[0013] In a possible implementation, the communication protocols of various industrial devices are integrated into a unified OPC server through the OPC data fusion module, including:

[0014] Carry out targeted communication configuration for the communication ports in each industrial equipment;

[0015] After the configuration is complete, set the IP address and remote access port of the database server;

[0016] Data is converted through communication networks and configuration information, and the communication protocols of various industrial equipment are integrated into a unified OPC server to summarize and unify the format.

[0017] In a possible implementation, configuring the connection between the OPC server and each industrial device to allow each industrial device to scan and obtain corresponding industrial data from the OPC server includes:

[0018] Set the address, port, and data point information of the OPC server to be collected;

[0019] Centralize the data in each industrial device through the intelligent gateway;

[0020] The industrial data of each industrial equipment’s status information, sensor data and process parameters are uploaded to the cloud platform through network communication.

[0021] In a possible implementation, the preprocessing of the acquired industrial data by the real-time data processing module and storing the processed industrial data in a designated database includes:

[0022] Set the preset standard format for each acquired industrial data according to the data format, data type and usage requirements provided by the equipment;

[0023] Clean the acquired industrial data according to the preset standard format, and clean the data with diverse structures and complex contents into data in a standard format;

[0024] During the cleaning process, redundant information is filtered and stored.

[0025] In a possible implementation, the obtained industrial data are cleaned according to a preset standard format, including: missing value cleaning, format content cleaning and logical error cleaning.

[0026] In a possible implementation, the missing value cleaning includes:

[0027] Determine the missing or empty values ​​in various industrial data;

[0028] Calculate the proportion of missing values ​​for each field in the data based on the pattern of missing values ​​and the characteristics of the data set;

[0029] Develop strategies based on the missing proportion and the importance of the field.

[0030] In a possible implementation, the strategies are formulated according to the missing ratio and the importance of the field, including:

[0031] If the pattern of missing values ​​is missing completely at random and the rows or columns where the missing values ​​are located are not important, then delete the rows or columns with missing values;

[0032] If the pattern of missing values ​​is completely nonrandom and related to unobserved variables, each data set is analyzed by multiple imputation or the missing values ​​are predicted by a predictive model;

[0033] If the pattern of missing values ​​is missing at random, the missing values ​​are filled by the mean, median or mode, or the K nearest neighbor algorithm is used to predict the missing values, or a machine learning model is used to predict the missing values, or the missing content is filled according to domain knowledge or business logic.

[0034] In a second aspect, an embodiment of the present application provides an industrial data acquisition system based on OPC, including:

[0035] A determination module, used to determine each industrial device that needs to collect data;

[0036] OPC data fusion module is used to integrate the communication protocols of various industrial equipment into a unified OPC server, and use the configuration list of data points to set the point information to be collected;

[0037] The data acquisition module is used to configure the connection between the OPC server and each industrial device, allowing each industrial device to scan and obtain the corresponding industrial data from the OPC server;

[0038] A real-time data processing module is used to pre-process the acquired industrial data and store the processed industrial data in a specified database;

[0039] The data access interface module is used for other systems or applications to access and obtain stored industrial data through the interface.

[0040] Compared with the prior art, the beneficial effects of this application are:

[0041] This application provides an efficient data platform to avoid duplication and waste in R&D testing, production processes, and occasional use, solve the difficulties in industrial data collection caused by the diversity of protocols of industrial equipment, and improve the effectiveness and ease of use of production data.

[0042] The OPC data fusion module in this application can easily set the point information to be collected through the configuration list of data points. There is no restriction on device type and quantity. It is compatible with the communication protocols of all popular and common control manufacturers in the current market, can link multiple devices, and adopts asynchronous processing to ensure the efficiency of data collection in the case of large data volume and high collection frequency, thereby improving the speed of data collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of a process flow of an industrial data acquisition method based on OPC provided in an embodiment of the present application;

[0044] Figure 2 An overall block diagram of an industrial data acquisition method based on OPC provided in an embodiment of the present application;

[0045] Figure 3 The data processing flow provided for the embodiment of the present application;

[0046] Figure 4 A schematic diagram of an OPC-based industrial data acquisition system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The present solution is described below in conjunction with the accompanying drawings and specific implementation methods.

[0048] Figure 1 A flow chart of an industrial data collection method based on OPC provided in an embodiment of the present application is shown in FIG. Figure 1 and Figure 2 , an industrial data collection method based on OPC in this embodiment includes:

[0049] S101, determining each industrial device that needs to collect data.

[0050] In this embodiment, the target of data collection, device status data and device parameter data are determined, and each industrial device is determined according to the target of data collection, device status data and device parameter data. In this embodiment, industrial devices such as programmable logic controllers, distributed control systems and robots can be selected.

[0051] S102, integrating the communication protocols of various industrial devices into a unified OPC server through the OPC data fusion module, and using the configuration list of data points to set the point information to be collected.

[0052] In this embodiment, targeted communication configuration is performed on the communication ports in each industrial device. After the configuration is completed, the IP address and remote access port of the database server are set, data conversion is performed through the communication network and configured information, and the communication protocols of each industrial device are integrated into a unified OPC server to summarize and unify the format.

[0053] The data fusion module in this embodiment can easily set the point information to be collected through the configuration list of data points, without restrictions on device type and quantity, and is compatible with all common control manufacturers' communication protocols in the current market. It can link multiple devices and use asynchronous processing to ensure that the efficiency of data collection can still be guaranteed under the conditions of large data volume and high collection frequency, thereby improving the speed of data collection. The acquired data information can be acquired and viewed through the data configuration client using the OPC UA / OPC DA communication method. The data collected in this application collects production data at the same time in real time based on time, which greatly improves the effectiveness and ease of use of production data.

[0054] S103, configuring the connection between the OPC server and each industrial device, allowing each industrial device to scan and obtain corresponding industrial data from the OPC server.

[0055] In this embodiment, the address, port, and data point information that need to be collected of the OPC server are set, the data in each industrial device is centralized through the intelligent gateway, and uploaded to the cloud platform using network communication to obtain the industrial data of the status information, sensor data, and process parameters of each industrial device.

[0056] S104, using a real-time data processing module to pre-process the acquired industrial data and store the processed industrial data in a designated database.

[0057] In this embodiment, the real-time data processing module integrates multiple commonly used databases, such as Sqlserver / Oracle / Mysql, etc., integrates multiple database drivers, and sets the corresponding database information in the configuration file. It can accurately and effectively store the collected information automatically in the specified database in the specified manner according to the configuration information.

[0058] In the process of industrial data collection, some data points may not be collected successfully due to various reasons, resulting in missing values ​​in the data set. Figure 3In this embodiment, the acquired industrial data are set into a preset standard format according to the data format, data type and usage requirements provided by the equipment, and the acquired industrial data are cleaned according to the preset standard format, and the data with diverse structures and complex contents are cleaned into data in a standard format. During the cleaning process, redundant information is filtered and processed before being stored.

[0059] Among them, the acquired industrial data are cleaned according to the preset standard format, including: missing value cleaning, format content cleaning and logical error cleaning. Missing value cleaning includes: determining the missing values ​​or null values ​​in each industrial data, calculating the missing value ratio of each field in the data according to the missing value pattern and the characteristics of the data set, and formulating strategies according to the missing ratio and the importance of the field. Ensure that all data items meet the preset format and content requirements. There may be differences in data format and content. Format content cleaning includes unifying data formats and verifying whether the data content meets the expected range or standard (such as temperature values ​​should be within a reasonable range, and there should be no negative values ​​or extreme cases far exceeding the normal value). Identify and correct logical inconsistencies in the data. Correct erroneous data, mark it as anomaly, or make reasonable estimates based on the context.

[0060] In this embodiment, if the pattern of missing values ​​is completely random missing and the row or column where the missing value is located is not important, the row or column with the missing value is deleted. If the pattern of missing values ​​is completely non-random missing and is related to unobserved variables, each data set is analyzed by multiple interpolation or the missing values ​​are predicted by a prediction model.

[0061] If the pattern of missing values ​​is missing at random, the missing values ​​are filled by the mean, median or mode, or the K nearest neighbor algorithm is used to predict the missing values, or a machine learning model is used to predict the missing values, or the missing content is filled according to domain knowledge or business logic.

[0062] In this embodiment, using the average value to fill the missing values ​​includes: using the main function to merge the training data and the label data, and then dividing the data according to different categories. For each category, the main function is called to calculate the average value of the category, and these average values ​​are used to fill the missing values ​​in the category data. Finally, all processed data segments are merged and the processed feature data and label data are returned.

[0063] S105, providing a data access interface, allowing other systems or applications to access and obtain the stored industrial data through the interface.

[0064] Corresponding to the OPC-based industrial data acquisition method provided in the above embodiment, the present application also provides an embodiment of an OPC-based industrial data acquisition system.

[0065] See also Figure 4 , an industrial data acquisition system 20 based on OPC provided in an embodiment of the present application includes:

[0066] The determination module 201 is used to determine each industrial device that needs to collect data.

[0067] The OPC data fusion module 202 is used to integrate the communication protocols of various industrial devices into a unified OPC server, and use the configuration list of data points to set the point information that needs to be collected.

[0068] The data acquisition module 203 is used to configure the connection between the OPC server and each industrial device, allowing each industrial device to scan and obtain corresponding industrial data from the OPC server.

[0069] The real-time data processing module 204 is used to pre-process the acquired industrial data and store the processed industrial data in a designated database.

[0070] The data access interface module 205 is used for other systems or applications to access and obtain the stored industrial data through the interface.

[0071] An OPC-based industrial data acquisition system in this embodiment also includes a production control module. Through the control point configuration of the control module, when it is necessary to output control information to the device, the required control information is written into the configuration table, and the control data is accurately sent to the device through the configured point information to realize the device control function.

[0072] In addition, an OPC-based industrial data acquisition system in this embodiment also has historical data storage and processing functions, wherein the collected real-time data is processed, and it has historical data processing functions of minutes / hours / days, and the corresponding average values, cumulative values, and changes in switch quantities are realized through configuration information, and classified statistics are performed.

[0073] The historical data storage includes the integration of multiple common databases and the fusion of multiple database drivers, which realizes the automation and intelligence of data collection and storage, and improves the efficiency and accuracy of data storage. Through configuration file drive and scalability design, the system has good flexibility and configurability, and can meet the data storage needs in different scenarios.

[0074] In the embodiments of the present application, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0075] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0076] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An industrial data acquisition method based on OPC, characterized in that: include: Identify the industrial equipment that needs to collect data; The communication protocols of various industrial devices are integrated into a unified OPC server through the OPC data fusion module, and the point information to be collected is set using the configuration list of data points; Configure the connection between the OPC server and each industrial device, allowing each industrial device to scan and obtain the corresponding industrial data from the OPC server; Use the real-time data processing module to pre-process the acquired industrial data and store the processed industrial data in the specified database; A data access interface is provided to allow other systems or applications to access and obtain stored industrial data through the interface.

2. The industrial data acquisition method based on OPC according to claim 1, characterized in that: The industrial equipment that needs to collect data includes: Determine the data collection objectives, equipment status data and equipment parameter data; Each industrial equipment is determined according to the data collection target, equipment status data and equipment parameter data.

3. The industrial data acquisition method based on OPC according to claim 1, characterized in that: The OPC data fusion module integrates the communication protocols of various industrial devices into a unified OPC server, including: Carry out targeted communication configuration for the communication ports in each industrial equipment; After the configuration is complete, set the IP address and remote access port of the database server; Data is converted through communication networks and configuration information, and the communication protocols of various industrial equipment are integrated into a unified OPC server to summarize and unify the format.

4. The industrial data acquisition method based on OPC according to claim 1, characterized in that: The configuration of the connection between the OPC server and each industrial device allows each industrial device to scan and obtain corresponding industrial data from the OPC server, including: Set the address, port, and data point information of the OPC server to be collected; Centralize the data in each industrial device through the intelligent gateway; The industrial data of each industrial equipment’s status information, sensor data and process parameters are uploaded to the cloud platform through network communication.

5. The industrial data acquisition method based on OPC according to claim 1, characterized in that: The method of preprocessing the acquired industrial data using the real-time data processing module and storing the processed industrial data in a designated database includes: Set the preset standard format for each acquired industrial data according to the data format, data type and usage requirements provided by the equipment; Clean the acquired industrial data according to the preset standard format, and clean the data with diverse structures and complex contents into data in a standard format; During the cleaning process, redundant information is filtered and stored.

6. The OPC-based industrial data acquisition method according to claim 5, characterized in that: The obtained industrial data are cleaned according to a preset standard format, including: missing value cleaning, format content cleaning and logical error cleaning.

7. The OPC-based industrial data acquisition method according to claim 6, characterized in that: The missing value cleaning includes: Determine the missing or empty values ​​in various industrial data; Calculate the proportion of missing values ​​for each field in the data based on the pattern of missing values ​​and the characteristics of the data set; Develop strategies based on the missing proportion and the importance of the field.

8. The OPC-based industrial data acquisition method according to claim 7, characterized in that: According to the missing ratio and the importance of the field, strategies are formulated respectively, including: If the pattern of missing values ​​is missing completely at random and the rows or columns where the missing values ​​are located are not important, then delete the rows or columns with missing values; If the pattern of missing values ​​is completely nonrandom and related to unobserved variables, each data set is analyzed by multiple imputation or the missing values ​​are predicted by a predictive model; If the pattern of missing values ​​is missing at random, the missing values ​​are filled by the mean, median or mode, or the K nearest neighbor algorithm is used to predict the missing values, or a machine learning model is used to predict the missing values, or the missing content is filled according to domain knowledge or business logic.

9. An industrial data acquisition system based on OPC, characterized in that: include: A determination module, used to determine each industrial device that needs to collect data; OPC data fusion module is used to integrate the communication protocols of various industrial equipment into a unified OPC server, and use the configuration list of data points to set the point information to be collected; The data acquisition module is used to configure the connection between the OPC server and each industrial device, allowing each industrial device to scan and obtain the corresponding industrial data from the OPC server; A real-time data processing module is used to pre-process the acquired industrial data and store the processed industrial data in a specified database; The data access interface module is used for other systems or applications to access and obtain stored industrial data through the interface.

Images