Universal industrial internet data acquisition method and system

Abstract

The invention provides a universal industrial internet data acquisition method and system, which solves the problems of industrial information data retrieval, scheduling and the like, and comprises the following steps: S1, network data acquisition; s2, data analysis processing; s3, receiving and classifying the data; and S4: data storage. The method has the advantages of good retrieval scheduling effect, high data acquisition precision and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial Internet, and specifically relates to a general industrial Internet data collection method and system. Background Art

[0002] The process of industrial production and manufacturing can be simplified as the factory purchasing equipment, setting up production lines, purchasing raw materials, workers entering the factory for production, and finally selling and delivering. The roles of the industrial chain involved in the process include equipment manufacturers, equipment agents or integrators, raw material suppliers, factories, and customers. Therefore, the concept of the industrial Internet includes not only the connection of the entire production and manufacturing process in the factory, especially the equipment connection of heavy asset industries, but also the coordination and connection of the entire industrial chain from equipment manufacturers, suppliers to factories, and then to customers. The perception layer of the industrial Internet of Things is the first entrance to data. In reality, since the sources of perception layer data are very diverse, from various multi-source heterogeneous devices and systems, how to obtain data from these devices and systems is the first threshold faced by the industrial Internet of Things. For large quantities of industrial data, the current industrial Internet lacks general retrieval and scheduling methods.

[0003] In order to solve the shortcomings of the existing technology, people have conducted long-term exploration and proposed various solutions. For example, a Chinese patent document discloses a data acquisition system based on industrial Internet identification [202310950370.0], which includes an encryption center, a device identification module, a data preprocessing module, a data acquisition module and a data acquisition server; the device identification module is used to assign a unique industrial Internet identification code to industrial equipment, the industrial Internet identification code is associated with a communication sequence, and the equipment data of the industrial equipment, the industrial Internet identification code and the communication sequence are summarized into a data identification package; the data identification package is obtained through the data preprocessing module and outlier calibration and screening, data load verification and distributed encryption are performed, a data stream is generated according to the data identification package, and a communication channel is constructed for transmission; the corresponding number of data acquisition nodes are allocated according to the flow value of the data stream, and the node status is judged to control the acquisition process; distributed storage is performed through the data acquisition server, and several storage points are generated for data disaster recovery.

[0004] The above solution solves the problem of data collection security to a certain extent, but the solution still has many shortcomings, such as large-scale industrial information data retrieval and poor scheduling effect. Summary of the invention

[0005] The purpose of the present invention is to address the above problems and provide a general industrial Internet data collection method with reasonable design and good data retrieval and scheduling effects.

[0006] The purpose of the present invention is to provide a universal industrial Internet data acquisition system that meets the data fusion requirements in response to the above problems.

[0007] To achieve the above object, the present invention adopts the following technical solution: a general industrial Internet data collection method, comprising the following steps:

[0008] S1: Network data collection;

[0009] S2: data analysis and processing;

[0010] S3: data receiving classification;

[0011] S4: Data storage.

[0012] In the above-mentioned general industrial Internet data collection method, step S1 includes the following steps:

[0013] S11: Use Modbus or CAN bus to collect and summarize data from the data acquisition unit in real time;

[0014] S12: Calibrate and sample the data and use genetic algorithm to build a data acquisition model;

[0015] S13: Provide real-time data feedback and cyclically update the data sampling model.

[0016] In the above-mentioned general industrial Internet data collection method, step S12 includes the following steps:

[0017] S121: Establish a data calibration model and set model parameters;

[0018] S122: Determine the measured value and the actual value of each data calibration point, and determine the model and model parameters between adjacent data calibration points;

[0019] S123: Determine the population and population size of the model between adjacent data calibration points, and use a genetic algorithm to optimize individual variables;

[0020] S124: Initialize the population, determine the objective function and set the termination condition, use the objective function to obtain all objective function values ​​of individuals in the population, and obtain the best and worst individuals;

[0021] S125: Update the population and calculate the objective function of individuals in the new population;

[0022] S126: Compare the new and old population individuals to determine the advantages and disadvantages of the new and old individuals. If the new individual is better than the old individual, the old individual will be replaced, otherwise the old individual will be retained;

[0023] S127: Determine whether the termination condition is met, if so, output the optimal solution of the population, otherwise return to step S124.

[0024] In the above-mentioned general industrial Internet data collection method, step S2 adopts a convolutional neural network model for data analysis. The convolutional neural network includes an input layer, a residual layer, a fully connected layer and an output layer. The residual layer has several convolution blocks and each convolution block has a convolution layer, a batch standard layer and an excitation layer.

[0025] In the above-mentioned general industrial Internet data collection method, step S2 includes the following steps:

[0026] S21: Recover missing data by calculating the average or median of adjacent data to recover missing data, or use the following interpolation method to recover data:

[0027]

[0028] where x i is a missing value in the data set, NaN is a non-data character, if x i If it is empty, interpolation is performed;

[0029] S22: Process discrete data using the following empirical rules:

[0030]

[0031] Where X represents Avg(x i )+2σ(x i ), σ represents the standard deviation of x;

[0032] S23: Normalize the data using the minimum-maximum method:

[0033]

[0034] In the above-mentioned general industrial Internet data collection method, a cloud server is used to classify data in step S3, which includes the following steps:

[0035] S31: Use cloud server to fuse data, and detect and extract correlation features from fused data;

[0036] S32: using a convolutional neural network to classify the extracted correlations, and combining feature compression to perform dimensionality reduction on the classification results;

[0037] S33: Use fuzzy clustering algorithm to classify data.

[0038] In the above-mentioned general industrial Internet data collection method, step S31 includes the following steps:

[0039] S311: Use extreme learning machine to optimize data features and build a data linear programming model in combination with cloud servers;

[0040] S312: Decompose the data features in a linear programming model to obtain data information entropy values, and fuse the data in the cloud server in combination with matching related detection;

[0041] S313: Detect the fused data and extract the correlation features, the expression of which is:

[0042]

[0043] Among them, N I 、N R 、N S are the average mutual information feature and state distribution set of the data respectively.

[0044] In the above-mentioned general industrial Internet data collection method, step S32 includes the following steps:

[0045] S321: extracting features from data using a correlation feature extraction model, and constructing a feature training set using the extracted correlation features;

[0046] S322: construct a base classifier composed of a convolutional neural network, and train the convolutional neural network model using features in the feature training set;

[0047] S323: constructing a probability feature space and training a meta-classifier composed of a support vector machine;

[0048] S324: The base classifier is used to predict the correlation extraction features, and after the probability features are generated, the meta classifier is used to predict to obtain the correlation prediction result.

[0049] In the above-mentioned general industrial Internet data collection method, any one or more combinations of cloud storage, edge storage and local storage are used in step S4.

[0050] A universal industrial Internet data collection system adopts the above universal industrial Internet data collection method.

[0051] Compared with the existing technology, the advantages of the present invention are: data collection adopts genetic algorithm to construct data collection model and update it in real time, and the data adopts multi-point calibration method to improve the collection accuracy; the collected data is analyzed by convolutional neural network, missing and discrete data are processed, and finally normalized to ensure the data fusion effect; the cloud server classifies the data, specifically adopts convolutional neural network to extract correlation features and build a classifier to classify the data, thereby improving the data retrieval scheduling rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the method of the present invention;

[0053] Figure 2 It is a flow chart of network data collection of the present invention;

[0054] Figure 3 is a flow chart of the data parsing process of the present invention;

[0055] Figure 4 It is a flow chart of data receiving separation of the present invention.

[0056] Figure 5 It is an architecture diagram of the convolutional neural network of the present invention. DETAILED DESCRIPTION

[0057] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] like Figure 1-5 As shown, a general industrial Internet data collection method includes the following steps:

[0059] S1: Network data collection;

[0060] S2: data analysis and processing;

[0061] S3: data receiving classification;

[0062] S4: Data storage.

[0063] Specifically, step S1, based on the existing data acquisition system, uses a genetic algorithm to optimize the data processing model according to the problem that the data measurement value is not synchronized with the actual value, so as to achieve high-precision real-time data acquisition, which includes the following steps:

[0064] S11: Use Modbus or CAN bus to collect and summarize data from the data acquisition unit in real time;

[0065] S12: Calibrate and sample the data and use genetic algorithm to build a data acquisition model;

[0066] S13: Provide real-time data feedback and cyclically update the data sampling model.

[0067] In order to improve the accuracy of data collection, step S12 uses a multi-point calibration method to divide the data produced in the industrial generation process into multiple intervals according to time. The parameters of each interval are determined by the calibration points, which specifically includes the following steps:

[0068] S121: Establish a data calibration model and set model parameters;

[0069] S122: Determine the measured value and the actual value of each data calibration point, and determine the model and model parameters between adjacent data calibration points;

[0070] S123: Determine the population and population size of the model between adjacent data calibration points, and use a genetic algorithm to optimize individual variables;

[0071] S124: Initialize the population, determine the objective function and set the termination condition, use the objective function to obtain all objective function values ​​of individuals in the population, and obtain the best and worst individuals;

[0072] S125: Update the population and calculate the objective function of individuals in the new population;

[0073] S126: Compare the new and old population individuals to determine the advantages and disadvantages of the new and old individuals. If the new individual is better than the old individual, the old individual will be replaced, otherwise the old individual will be retained;

[0074] S127: Determine whether the termination condition is met, if so, output the optimal solution of the population, otherwise return to step S124.

[0075] After determining the model parameters, the number of individuals in the population, individual variables, objective function and termination conditions, first calculate the individual objective function to obtain the best and worst individuals, then update the population, calculate the objective function of the individuals in the new population and compare it with the individuals in the old population, replace the inferior individuals in the old population with the superior individuals in the new population, generate a new population composed of superior individuals, then iterate until the termination adjustment is met, and finally output the optimal individual of the population and determine the model parameters.

[0076] Furthermore, step S2 adopts a convolutional neural network model for data analysis, and the convolutional neural network includes an input layer, a residual layer, a fully connected layer and an output layer. The residual layer has a plurality of convolutional blocks and each convolutional block has a convolutional layer, a batch standard layer and an excitation layer.

[0077] Furthermore, in step S2, the data analysis needs to consider the issues of missing data compensation, discrete data processing and data normalization to ensure the accuracy of subsequent data classification training, which includes the following steps:

[0078] S21: Since data is missing due to transmission damage or hardware problems during acquisition, it is necessary to recover the missing data by calculating the average or median value of adjacent data, or using the following interpolation method to recover the data:

[0079]

[0080] where x i is a missing value in the data set, NaN is a non-data character, if x i If it is empty, interpolation is performed;

[0081] S22: For discrete data, use the empirical rule to process. In the normal distribution of data, almost all data will fall within three standard deviations of the mean. According to the empirical rule, 68% of the data will be distributed within the first standard deviation, 95% will fall within the second standard deviation, and 99.7% will fall within the first three standard deviations of the mean. Use the following method to remove outliers and recover data:

[0082]

[0083] Where X represents Avg(x i )+2σ(x i ), σ represents the standard deviation of x;

[0084] S23: Normalize the data using the minimum-maximum method:

[0085]

[0086] The specific processing methods are as follows:

[0087]

[0088] where x ij is the original value of the jth data in the ith interval, Y ij is the jth data processing value in the ith interval, minx ij is the lowest value of the original value, max x ij The highest value of the original value is obtained by normalization, and the comparison sequence is obtained. The optimal value of each data constitutes the reference sequence.

[0089] In addition, the cloud server is used to classify the data in step S3, which includes the following steps:

[0090] S31: The cloud server is used to fuse the data, and the fused data is tested and the correlation features are extracted. The correlation coefficient reflects the correlation between the comparison sequence and the reference sequence at each calibration point. The calculation of the correlation coefficient is as follows:

[0091]

[0092] Then select the average value of the correlation coefficient to get the correlation between the comparison sequence and the reference sequence:

[0093]

[0094] S32: using a convolutional neural network to classify the extracted correlations, and combining feature compression to perform dimensionality reduction on the classification results;

[0095] S33: Use fuzzy clustering algorithm to classify data.

[0096] At the same time, in order to realize the retrieval scheduling of data by the cloud server, step S31 needs to build a distributed data model in the cloud server state, which specifically includes the following steps:

[0097] S311: Use extreme learning machine to optimize data features and build a data linear programming model in combination with cloud servers;

[0098] S312: Decompose the data features in a linear programming model to obtain data information entropy values, and fuse the data in the cloud server in combination with matching related detection;

[0099] S313: Detect the fused data and extract the correlation features, the expression of which is:

[0100]

[0101] Among them, N I 、N R 、N S are the average mutual information feature and state distribution set of the data respectively.

[0102] It can be seen that step S32 includes the following steps:

[0103] S321: extracting features from data using a correlation feature extraction model, and constructing a feature training set using the extracted correlation features;

[0104] S322: construct a base classifier composed of a convolutional neural network, and train the convolutional neural network model using features in the feature training set;

[0105] S323: constructing a probability feature space and training a meta-classifier composed of a support vector machine;

[0106] S324: The base classifier is used to predict the correlation extraction features, and after the probability features are generated, the meta classifier is used to predict to obtain the correlation prediction result.

[0107] Obviously, any one or more combinations of cloud storage, edge storage and local storage are used in step S4.

[0108] A general industrial Internet data acquisition system adopts the above-mentioned general industrial Internet data acquisition method. The acquisition system specifically includes a data acquisition module, a data analysis module, a data receiving module and a data storage module. In order to realize data interaction, industrial production tasks perform data acquisition and aggregation of multiple data by distinguishing different data addresses in an independent loop. The data acquisition system uses CAN bus and Modbus for data aggregation and 4G / 5G wireless transmission of data.

[0109] To sum up, the principle of this embodiment is: a genetic algorithm is used to construct a data acquisition model and update it in real time; the collected data is analyzed using a convolutional neural network, missing and discrete data are processed, and finally normalized; the cloud server classifies the data, specifically using a convolutional neural network to extract correlation features and build a classifier for data classification.

[0110] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

[0111] Although the terms such as data collection, data analysis, data reception, and data storage are used more frequently in this article, the possibility of using other terms is not excluded. These terms are used only to more conveniently describe and explain the essence of the present invention; interpreting them as any additional restrictions is contrary to the spirit of the present invention.

Claims

1. A general industrial Internet data collection method, characterized in that: The steps include: S1: Network data collection; S2: data analysis and processing; S3: data receiving classification; S4: Data storage.

2. A general industrial Internet data collection method according to claim 1, characterized in that: The step S1 comprises the following steps: S11: Use Modbus or CAN bus to collect and summarize data from the data acquisition unit in real time; S12: Calibrate and sample the data and use genetic algorithm to build a data acquisition model; S13: Provide real-time data feedback and cyclically update the data sampling model.

3. A general industrial Internet data collection method according to claim 2, characterized in that: The step S12 comprises the following steps: S121: Establish a data calibration model and set model parameters; S122: Determine the measured value and the actual value of each data calibration point, and determine the model and model parameters between adjacent data calibration points; S123: Determine the population and population size of the model between adjacent data calibration points, and use a genetic algorithm to optimize individual variables; S124: Initialize the population, determine the objective function and set the termination condition, use the objective function to obtain all objective function values ​​of individuals in the population, and obtain the best and worst individuals; S125: Update the population and calculate the objective function of individuals in the new population; S126: Compare the new and old population individuals to determine the advantages and disadvantages of the new and old individuals. If the new individual is better than the old individual, the old individual will be replaced, otherwise the old individual will be retained; S127: Determine whether the termination condition is met, if so, output the optimal solution of the population, otherwise return to step S124.

4. A general industrial Internet data collection method according to claim 1, characterized in that: The step S2 adopts a convolutional neural network model to perform data analysis. The convolutional neural network includes an input layer, a residual layer, a fully connected layer and an output layer. The residual layer has a plurality of convolutional blocks, and each convolutional block has a convolutional layer, a batch standard layer and an excitation layer.

5. A general industrial Internet data collection method according to claim 4, characterized in that: The step S2 comprises the following steps: S21: Recover missing data by calculating the average or median of adjacent data to recover missing data, or use the following interpolation method to recover data: where x i is a missing value in the data set, NaN is a non-data character, if x i If it is empty, interpolation is performed; S22: Process discrete data using the following empirical rules: Where X represents Avg(x i )+2σ(x i ), σ represents the standard deviation of x; S23: Normalize the data using the minimum-maximum method:

6. A general industrial Internet data collection method according to claim 4, characterized in that: In step S3, a cloud server is used to classify data, which includes the following steps: S31: Use cloud server to fuse data, and detect and extract correlation features from fused data; S32: using a convolutional neural network to classify the extracted correlations, and combining feature compression to perform dimensionality reduction on the classification results; S33: Use fuzzy clustering algorithm to classify data.

7. A general industrial Internet data collection method according to claim 6, characterized in that: The step S31 comprises the following steps: S311: Use extreme learning machine to optimize data features and build a data linear programming model in combination with cloud servers; S312: Decompose the data features in a linear programming model to obtain data information entropy values, and fuse the data in the cloud server in combination with matching related detection; S313: Detect the fused data and extract the correlation features, the expression of which is: Among them, N I 、N R 、N S are the average mutual information feature and state distribution set of the data respectively.

8. A general industrial Internet data collection method according to claim 7, characterized in that: The step S32 comprises the following steps: S321: extracting features from data using a correlation feature extraction model, and constructing a feature training set using the extracted correlation features; S322: construct a base classifier composed of a convolutional neural network, and train the convolutional neural network model using features in the feature training set; S323: constructing a probability feature space and training a meta-classifier composed of a support vector machine; S324: The base classifier is used to predict the correlation extraction features, and after the probability features are generated, the meta classifier is used to predict to obtain the correlation prediction result.

9. A general industrial Internet data collection method according to claim 1, characterized in that: In the step S4, any one or more combinations of cloud storage, edge storage and local storage are used.

10. A general industrial Internet data acquisition system, characterized in that: The general industrial Internet data collection method described in any one of claims 1 to 10 above is adopted.

Images