Industrial production Internet of Things data micro-service extraction method, medium and system

Abstract

The invention provides an industrial production Internet of Things data micro-service extraction method, a medium and a system, and belongs to the technical field of electrical digital data processing.The industrial production Internet of Things data micro-service extraction method comprises the steps that firstly, an original data stream is collected and decomposed into stable and fluctuation components, and a stability evaluation matrix is established to construct a data model; and calculating a data time contribution value through the time sequence incidence matrix, and establishing a data weight matrix to calculate a data source weight coefficient. And then constructing a micro-service interface contribution matrix to evaluate interface performance, establishing a data processing pipeline calculation efficiency matrix, and correcting a processing result through a data processing compensation equation set. And finally, establishing a data synchronization consistency matrix, constructing an industrial production Internet of Things data extraction matrix, fusing characteristic parameters of each matrix, realizing real-time adjustment of a data extraction strategy through a feedback mechanism, and ensuring efficient extraction of data. The technical problems of poor data stability and insufficient reliability in the data extraction process of the industrial internet of things at present are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic digital data processing, and in particular, relates to a method, medium and system for extracting microservices from industrial production Internet of Things data. Background Art

[0002] As a product of the deep integration of new-generation information technology and manufacturing, the Industrial Internet of Things has become an important driving force for promoting industrial digital transformation. Traditional industrial production data collection mainly relies on manual records or simple automation equipment. With the widespread application of various intelligent sensors, controllers and actuators, massive amounts of real-time data are generated at industrial production sites. These data include multi-dimensional information such as equipment operating status, process parameters, and environmental indicators. At present, the mainstream industrial Internet of Things data extraction methods mainly use mechanisms such as timed polling, triggered collection, and subscription push to transmit data to upper-level application systems through communication networks such as industrial Ethernet and fieldbus.

[0003] However, the existing industrial IoT data extraction technology has many shortcomings. First, the traditional fixed-period collection method is difficult to adapt to the dynamically changing data characteristics in the industrial production process, and often the sampling frequency is too high, resulting in excessive system load, or the sampling frequency is too low, resulting in the loss of key data. Secondly, in the data processing link, simple data filtering and conversion methods cannot effectively handle outliers and noise, affecting the reliability of the data. In addition, although the existing microservice architecture provides good scalability, it lacks an effective quality assurance mechanism in the process of data distribution, processing and synchronization, resulting in the accuracy and consistency of data processing results. It is difficult to guarantee. In a complex industrial production environment, the equipment status is changeable and the process parameters fluctuate frequently, and these problems are more prominent.

[0004] Especially in large-scale industrial production scenarios, it is difficult for existing technologies to simultaneously ensure the real-time, reliability, and stability of data extraction. When the scale of the production line expands and the number of equipment increases, the synergy between data acquisition nodes decreases, and the efficiency of the data processing pipeline decreases, resulting in data extraction quality that is difficult to meet the actual needs of industrial production. In this case, how to build an efficient and reliable IoT data extraction system to achieve accurate data collection, intelligent processing, and reliable transmission has become a technical problem that needs to be solved urgently. In other words, the existing technology has technical problems such as poor data stability and insufficient reliability in the process of industrial IoT data extraction. Summary of the invention

[0005] In view of this, the present invention provides an industrial production Internet of Things data microservice extraction method, medium and system, which can solve the technical problems of poor data stability and insufficient reliability in the industrial Internet of Things data extraction process in the prior art.

[0006] The present invention is implemented as follows: In a first aspect, the present invention provides an industrial production Internet of Things data microservice extraction method, comprising the following steps: collecting Internet of Things device data at an industrial production site to obtain an original data stream, establishing a stability evaluation matrix and a timing association matrix, constructing a data weight matrix and a microservice interface contribution matrix, calculating a data processing efficiency matrix, constructing a data processing compensation equation group and calculating a data processing compensation value, establishing a data synchronization consistency matrix, and finally constructing and outputting an industrial production Internet of Things data extraction matrix, wherein the industrial production Internet of Things data extraction matrix includes an Internet of Things device data extraction matrix and an Internet of Things data extraction quality matrix.

[0007] Wherein: the original data stream is decomposed and calculated to decompose the original data stream into a stable data component and a fluctuating data component; the stability evaluation matrix includes a stability coefficient of the stable data component and a fluctuation coefficient of the fluctuating data component.

[0008] Wherein: a data model is constructed based on the stability assessment matrix, the data model includes data identification, data type, data source, data timestamp, and data value; and a data time contribution value is calculated based on the timing association matrix.

[0009] Among them: the data weight matrix includes data importance coefficient, data timeliness coefficient, and data normalization coefficient; the microservice interface contribution matrix includes interface call frequency, interface response time, and interface data throughput.

[0010] Among them: the data processing efficiency matrix includes data filtering efficiency coefficient, data filtering threshold, data type conversion matrix, and data aggregation window size; the data synchronization consistency matrix includes data version consistency coefficient, data content consistency coefficient, and data time consistency coefficient.

[0011] Wherein: the data processing compensation equation group includes a data filtering compensation equation, a data conversion compensation equation, a data aggregation compensation equation, and a data fusion compensation equation.

[0012] Among them: the Internet of Things device data extraction matrix includes the Internet of Things device coverage coefficient, process data collection coefficient, production parameter extraction coefficient, and equipment data collection frequency coefficient.

[0013] Among them: the IoT data extraction quality matrix includes data extraction completeness coefficient, data extraction accuracy coefficient, data extraction real-time coefficient, and data extraction consistency coefficient.

[0014] A second aspect of the present invention provides a computer-readable storage medium, in which program instructions are stored. When the program instructions are run in a computer, they are used to execute the above-mentioned industrial production Internet of Things data microservice extraction method.

[0015] The third aspect of the present invention provides an industrial production Internet of Things data microservice extraction system, comprising the above-mentioned computer-readable storage medium, the system is any one of a computer, a server, and a single-chip microcomputer, the computer-readable storage medium is arranged in the system, and the system is provided with a microprocessor for executing program instructions stored in the computer-readable storage medium.

[0016] Compared with the prior art, the present invention provides an industrial production Internet of Things data microservice extraction method, medium and system. The industrial production Internet of Things data microservice extraction method proposed in the present invention realizes the intelligent management of the whole process of data extraction by constructing a multi-layer matrix evaluation system and compensation mechanism. The method first performs a stability analysis on the original data stream, establishes an evaluation matrix including a stability coefficient and a fluctuation coefficient, and provides a reliable basis for subsequent data processing. Through the design of the timing association matrix and the data weight matrix, the system can adaptively adjust the data collection strategy to ensure the timely acquisition and processing of important data. At the same time, the introduction of the microservice interface contribution matrix effectively improves the load balancing capability and service quality of the system.

[0017] In practical applications, the method of the present invention realizes precise control of the data processing process through a group of data processing compensation equations. The data filtering compensation equation can intelligently identify and process abnormal data and improve data quality; the data conversion compensation equation ensures the accurate conversion of different types of data; the data aggregation compensation equation optimizes the time window division strategy; and the data fusion compensation equation realizes the reliable fusion of multi-source data. The synergistic effect of these compensation mechanisms significantly improves the accuracy and reliability of data processing. In addition, the application of the data synchronization consistency matrix ensures the consistency of data in a distributed environment and avoids system errors caused by data asynchrony.

[0018] Finally, the industrial production Internet of Things data extraction matrix constructed by the present invention establishes a complete data quality evaluation system by integrating characteristic parameters of multiple dimensions. This method not only solves the problem of poor data stability in traditional technologies, but also realizes real-time optimization of data extraction strategies through feedback mechanisms, ensuring efficient, stable and reliable extraction of industrial production Internet of Things data. The system can adaptively adjust data collection and processing strategies according to dynamic changes in the production environment, effectively improving the quality and efficiency of data extraction, and solving the technical problems of poor data stability and insufficient reliability in the industrial Internet of Things data extraction process in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flow chart of the method of the present invention.

[0020] Figure 2 The data component decomposition and time series correlation analysis diagram in Example 2; Figure 3 It is the time contribution value decomposition analysis diagram in Example 2; Figure 4 This is a data processing efficiency analysis diagram in Example 2; Figure 5 This is a radar chart of data synchronization consistency in Example 2. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0022] like Figure 1 FIG. 1 is a flowchart of a method for extracting microservices from industrial production Internet of Things data provided by the first aspect of the present invention. The method comprises the following steps: S10, collecting IoT device data at the industrial production site to obtain an original data stream, performing decomposition calculation on the original data stream, and decomposing the original data stream into a stable data component and a fluctuating data component; S20, establishing a stability evaluation matrix, wherein the stability evaluation matrix includes a stability coefficient of the stable data component and a fluctuation coefficient of the fluctuation data component, and constructing a data model based on the stability evaluation matrix, wherein the data model includes a data identifier, a data type, a data source, a data timestamp, and a data value; S30, calculating the data collection period and the data collection timestamp according to the stability evaluation matrix, constructing a time series association matrix, wherein the time series association matrix represents the degree of association between adjacent data within the data collection period, and calculating the data time contribution value based on the time series association matrix; S40, establishing a data weight matrix according to the data time contribution value, the data weight matrix including a data importance coefficient, a data timeliness coefficient, and a data normalization coefficient, and calculating a data source weight coefficient based on the data weight matrix; S50, constructing a microservice interface contribution matrix, wherein the microservice interface contribution matrix includes interface call frequency, interface response time, and interface data throughput, and calculating a data source credibility coefficient and a data source integrity coefficient according to the microservice interface contribution matrix; S60, distributing the original data stream to the microservice node based on the data source weight coefficient, establishing a data processing pipeline, and calculating a data processing efficiency matrix, wherein the data processing efficiency matrix includes a data filtering efficiency coefficient, a data filtering threshold, a data type conversion matrix, and a data aggregation window size; S70, constructing a data processing compensation equation group based on the data processing efficiency matrix, calculating a data processing compensation value according to the data processing compensation equation group, and the corrected data processing result is the sum of the original data processing result and the data processing compensation value; S80, performing data synchronization on the corrected data processing result, and establishing a data synchronization consistency matrix, wherein the data synchronization consistency matrix includes a data version consistency coefficient, a data content consistency coefficient, and a data time consistency coefficient; S90. Construct and output an industrial production Internet of Things data extraction matrix, wherein the industrial production Internet of Things data extraction matrix includes an Internet of Things device data extraction matrix and an Internet of Things data extraction quality matrix. The Internet of Things device data extraction matrix includes an Internet of Things device coverage coefficient, a process data collection coefficient, a production parameter extraction coefficient, and an equipment data collection frequency coefficient. The Internet of Things data extraction quality matrix includes a data extraction completeness coefficient, a data extraction accuracy coefficient, a data extraction real-time coefficient, and a data extraction consistency coefficient. The industrial production Internet of Things data extraction matrix integrates the characteristic parameters of the stability assessment matrix, the timing association matrix, the data weight matrix, the microservice interface contribution matrix, the data processing efficiency matrix, and the data synchronization consistency matrix. The industrial production Internet of Things data extraction strategy is adjusted in real time through a feedback mechanism to ensure efficient extraction of industrial production Internet of Things data.

[0023] The data processing compensation equation group includes a data filtering compensation equation, a data conversion compensation equation, a data aggregation compensation equation, and a data fusion compensation equation; The data filtering compensation equation is used to calculate the data filtering compensation coefficient, the input includes the data filtering efficiency coefficient, the data filtering threshold, and the original data stream, and the output is the data filtering compensation coefficient; The data conversion compensation equation is used to calculate the data conversion compensation coefficient, the input includes the data type conversion matrix and the data normalization coefficient, and the output is the data conversion compensation coefficient; The data aggregation compensation equation is used to calculate the data aggregation compensation coefficient, the input includes the data collection timestamp and the data aggregation window size, and the output is the data aggregation compensation coefficient; The data fusion compensation equation is used to calculate the data fusion compensation coefficient. The input includes the data source weight coefficient, the data source credibility coefficient, and the data source integrity coefficient. The output is the data fusion compensation coefficient.

[0024] The specific implementation of the above steps is described in detail below. The specific implementation of step S10 is to first collect the original data stream through the Internet of Things devices at the industrial production site. In order to extract valuable information, the original data stream needs to be decomposed and calculated. The original data stream is decomposed into a stable data component and a fluctuating data component using a time series decomposition method. The stable data component represents the basic trend of the data, and the fluctuating data component reflects the short-term fluctuations of the data. The specific process of decomposition and calculation is as follows: First, the original data stream is represented as a time series According to the theory of time series analysis, Can be decomposed into stable data components , Fluctuation data component and noise component Three parts, namely Among them, the stable data component It can be expressed as a linear combination: ,in is the decomposition coefficient, is the basis function, is the number of basis functions. Fluctuation data components You can use To indicate that is the volatility coefficient, is the wave basis function, is the number of fluctuation basis functions. By decomposing the original data stream into time series, the stable data component and the fluctuation data component can be effectively extracted, laying the foundation for subsequent data processing and analysis.

[0025] The specific implementation of step S20 is to first establish a stability evaluation matrix . This matrix contains the stability coefficients of the stable data components and the volatility coefficient of the volatility data component . Stability factor It reflects the stability of the data in the time series, and its value range is [0, 1]. It indicates the cross-influence between different data components. By statistically analyzing historical data, the specific values ​​of these coefficients can be determined. With the stability assessment matrix, the data model can be constructed. The data model contains information such as data identification, data type, data source, data timestamp and data value. This information provides an important basis for subsequent data processing and analysis.

[0026] The specific implementation of step S30 is to first calculate the data collection cycle and data collection timestamp based on the stability evaluation matrix. The data collection cycle reflects the frequency of data update, which is usually determined based on data characteristics and application requirements. The data collection timestamp indicates the collection time of each data record. Based on this information, a time series correlation matrix can be constructed. This matrix describes the degree of correlation between adjacent data within the data collection period. The calculation formula is ,in and Respectively Moment and The data value at the moment, and is the corresponding data mean, is the size of the time window. By calculating the time series correlation matrix, we can get the time correlation between the data, which is very important for subsequent data analysis and processing. At the same time, we can also calculate the time contribution value of the data ,in is the time weight, is the time attenuation coefficient, It is the data timestamp. This time contribution value reflects the importance of the data in the time series.

[0027] The specific implementation method of step S40 is to first establish a data weight matrix based on the data time contribution value. The matrix contains the importance coefficient, timeliness coefficient and normalization coefficient of the data. The importance coefficient of the data reflects the relative value of the data in the entire data set and can be calculated based on the time contribution value. The timeliness coefficient of the data indicates the validity of the data in the time series and can be determined based on the timestamp and the data update frequency. The normalization coefficient is used to eliminate the dimensional differences between different types of data so that the data can be processed and analyzed uniformly. With the data weight matrix, the weight coefficient of the data source can be calculated. This coefficient comprehensively reflects the importance, timeliness and normalization of the data, providing a basis for subsequent data distribution.

[0028] The specific implementation method of step S50 is to first construct a microservice interface contribution matrix. The matrix includes three indicators: interface call frequency, interface response time, and interface data throughput. The interface call frequency reflects the degree of use of the interface in the entire system, the response time indicates the real-time performance of the interface, and the data throughput represents the processing capacity of the interface. By analyzing these three indicators, the credibility coefficient and integrity coefficient of the data source can be calculated. The credibility coefficient of the data source indicates the credibility score of the data provided by the data source, and the integrity coefficient reflects the performance of the data source in terms of data coverage and continuity. These two coefficients provide a basis for subsequent data distribution and processing.

[0029] The specific implementation method of step S60 is to first distribute the original data stream to each microservice node according to the data source weight coefficient to establish a data processing pipeline. Then, for each microservice node, a data processing efficiency matrix is ​​calculated. The matrix includes parameters such as data filtering efficiency coefficient, data filtering threshold, data type conversion matrix and data aggregation window size. The data filtering efficiency coefficient reflects the ability to filter noise from the original data, and the filtering threshold determines the intensity of the filtering. The data type conversion matrix describes the conversion relationship between different types of data, and the data aggregation window size affects the granularity of time series aggregation of the data. By setting these parameters reasonably, the efficient operation of the data processing pipeline can be ensured.

[0030] The specific implementation of step S70 is to construct a data processing compensation equation group according to the aforementioned data processing efficiency matrix. The equation group includes four parts: data filtering compensation equation, data conversion compensation equation, data aggregation compensation equation and data fusion compensation equation.

[0031] The data filter compensation equation is used to calculate the filter compensation coefficient ,in is the filter coefficient, the value range is [0.5, 2], is the data value, is the filtering threshold, is the filtering error term, and its value range is [-0.1, 0.1]. This equation can dynamically adjust the filtering compensation coefficient according to the actual data characteristics to ensure the optimal filtering effect.

[0032] The data conversion compensation equation is used to calculate the conversion compensation coefficient ,in To type-convert matrix elements, is the conversion weight, is the conversion error term, and its value range is [-0.1, 0.1]. This equation can calculate the appropriate compensation coefficient according to the conversion relationship between different data types.

[0033] The data aggregation compensation equation is used to calculate the aggregation compensation coefficient ,in is the aggregation window size, is the data timestamp, is the reference timestamp, is the time attenuation coefficient, ranging from 0.01 to 0.1. is the aggregation error term, and its value range is [-0.1, 0.1]. This equation can calculate a reasonable aggregation compensation coefficient based on the distribution of data timestamps.

[0034] The data fusion compensation equation is used to calculate the fusion compensation coefficient ,in is the data source weight coefficient, is the data source credibility coefficient, is the data source integrity coefficient, is the fusion error term, and its value range is [-0.1, 0.1]. This equation comprehensively considers the importance, credibility and integrity of different data sources and calculates the appropriate fusion compensation coefficient.

[0035] By calculating the above data processing compensation equation group, the errors generated in the data processing process can be compensated to ensure that the final data processing results are more accurate and reliable.

[0036] The specific implementation method of step S80 is to first synchronize the corrected data processing results. A data synchronization consistency matrix is ​​established, which contains three indicators: data version consistency coefficient, data content consistency coefficient, and data time consistency coefficient. The data version consistency coefficient reflects the consistency of data versions between different processing nodes, the data content consistency coefficient indicates the consistency of data content between different nodes, and the data time consistency coefficient describes the consistency of timestamps between different nodes. By analyzing and optimizing these three indicators, the synchronization consistency of data in the entire system can be ensured.

[0037] The specific implementation of step S90 is to finally construct and output the industrial production Internet of Things data extraction matrix, which includes two parts: the Internet of Things device data extraction matrix and the Internet of Things data extraction quality matrix.

[0038] The IoT device data extraction matrix includes the IoT device coverage coefficient, process data collection coefficient, production parameter extraction coefficient and device data collection frequency coefficient. The IoT device coverage coefficient reflects the proportion of IoT devices covered by the system to the total number of devices, the process data collection coefficient indicates the completeness of production process data collection, the production parameter extraction coefficient represents the extraction coverage of key production parameters, and the device data collection frequency coefficient describes the frequency of device data collection. These indicators comprehensively reflect the data collection of IoT devices.

[0039] The IoT data extraction quality matrix includes the data extraction completeness coefficient, data extraction accuracy coefficient, data extraction real-time coefficient, and data extraction consistency coefficient. The data extraction completeness coefficient indicates whether all the data required to be collected have been successfully extracted, the data extraction accuracy coefficient reflects the accuracy of the extracted data, the data extraction real-time coefficient represents the timeliness of the data, and the data extraction consistency coefficient describes the consistency of data between different data sources. These indicators comprehensively evaluate the quality of IoT data extraction.

[0040] Finally, the industrial production Internet of Things data extraction matrix integrates the characteristic parameters calculated in all the above steps, such as stability evaluation matrix, timing correlation matrix, data weight matrix, microservice interface contribution matrix, data processing efficiency matrix and data synchronization consistency matrix. Through the comprehensive analysis of these parameters, the extraction effect of industrial production Internet of Things data can be comprehensively evaluated. At the same time, the matrix can also adjust the data extraction strategy in real time through the feedback mechanism to further improve the efficiency and quality of data extraction.

[0041] The calculation process involved in the present invention is described in detail below.

[0042] 1. The decomposition calculation of the original data stream is expressed as follows: ; In the formula, is the original data stream time series; To stabilize the data component; is the fluctuation data component; is the noise component; is the sampling time point.

[0043] Stable data component calculation: ; In the formula, is the decomposition coefficient; is the basis function; is the number of basis functions.

[0044] Fluctuation data component calculation: ; In the formula, is the coefficient of volatility; is the wave basis function; is the number of wave basis functions.

[0045] 2. Construction of stability assessment matrix: ; In the formula, is the stability coefficient; is the cross-influence coefficient; is the matrix index.

[0046] 3. Time series correlation matrix calculation: ; In the formula, is the time series correlation coefficient; for The data value at the moment; for Moment mean; is the time window size.

[0047] Data time contribution value calculation: ; In the formula, Contribute value to time; is the time weight; is the time attenuation coefficient; Is the timestamp.

[0048] 4. Data processing compensation equations: Data filtering compensation equation: ; In the formula, is the filter compensation coefficient; is the filtration coefficient; is the data value; is the filtering threshold; is the filtering error term.

[0049] Data conversion compensation equation: ; In the formula, is the conversion compensation coefficient; Convert matrix elements to types; is the conversion weight; is the conversion error term.

[0050] Data aggregation compensation equation: ; In the formula, is the aggregation compensation coefficient; is the window size; is the timestamp; is the reference time; is the attenuation coefficient; is the aggregate error term.

[0051] Data fusion compensation equation: ; In the formula, is the fusion compensation coefficient; is the weight coefficient; is the credibility coefficient; is the integrity coefficient; is the fusion error term.

[0052] Parameter acquisition method: 1. Basis function coefficients and Obtained by least squares fitting; 2. Stability coefficient Obtained through statistical analysis of historical data, with a range of [0, 1]; 3. Time decay coefficient Determined by experiments, the typical value is 0.01~0.1; 4. Filtration coefficient Dynamically adjusted according to data characteristics, the range is [0.5, 2]; 5. Error term The range is [-0.1, 0.1].

[0053] The following is a detailed explanation of the derivation and establishment process of each equation: 1. Derivation of decomposition calculation equation: First, based on the classic time series decomposition theory, considering the trend, periodicity and randomness of industrial data, an initial decomposition model is constructed: ; In the formula, is the trend item; is a periodic term; is a random item.

[0054] The trend term and the period term are further combined into a stable component, and the random term is subdivided into a fluctuation component and a noise component: ; Stable Component The build process: Step 1: Choose an appropriate set of basis functions , including polynomial and trigonometric functions; Step 2: Apply the least squares method to solve the decomposition coefficients: ; Step 3: Determine the optimal number of basis functions through cross-validation .

[0055] 2. Derivation of stability assessment matrix: Step 1: Construct initial evaluation metrics: ; In the formula, is the stable component variance; is the overall variance.

[0056] Step 2: Introduce the cross-influence coefficient: ; Step 3: Form a complete evaluation matrix: .

[0057] 3. Derivation of data processing compensation equations: (1) Derivation of data filtering compensation equation: Step 1: Define the basic filtering error: ; Step 2: Introduce adaptive coefficient: ; In the formula, To adjust the parameters, is the data variance, is the data mean.

[0058] Step 3: Add the error compensation term to get the final equation: .

[0059] (2) Derivation of data conversion compensation equation: Step 1: Construct a type conversion matrix: ; Step 2: Introduce conversion weights: ; In the formula, is the type distance, is the attenuation parameter.

[0060] Step 3: Form the complete compensation equation: .

[0061] (3) Derivation of data aggregation compensation equation: Step 1: Define the time difference item: ; Step 2: Introduce time decay: ; Step 3: Combine to form the compensation equation:

[0062] .

[0063] Parameter optimization method: 1. and Optimization by gradient descent; 2. and Obtained by maximum likelihood estimation; 3. Dynamic updates using online learning methods; 4. The optimal range of the error term is determined through cross-validation.

[0064] A second aspect of the present invention provides a computer-readable storage medium, in which program instructions are stored. When the program instructions are run in a computer, they are used to execute the above-mentioned industrial production Internet of Things data microservice extraction method.

[0065] The third aspect of the present invention provides an industrial production Internet of Things data microservice extraction system, comprising the above-mentioned computer-readable storage medium, the system is any one of a computer, a server, and a single-chip microcomputer, the computer-readable storage medium is arranged in the system, and the system is provided with a microprocessor for executing program instructions stored in the computer-readable storage medium.

[0066] Specifically, the principle of the present invention is: the technical principle of the present invention is based on the idea of ​​data feature decomposition and multidimensional matrix evaluation. First, by performing stability analysis on the original data stream, the data is decomposed into stable components and fluctuating components. This decomposition method conforms to the essential characteristics of industrial production data, because industrial data usually contains stable baseline values ​​and fluctuating values ​​that change over time. Based on this decomposition, the system can more accurately evaluate the changing characteristics of the data and provide a scientific basis for subsequent processing.

[0067] In the data processing link, the multi-layer matrix evaluation system designed by the present invention forms a complete data quality assurance chain. The stability evaluation matrix quantifies the change characteristics of the data by calculating the stability coefficient and the fluctuation coefficient; the time series correlation matrix captures the time dependency between data and optimizes the sampling strategy; the data weight matrix reasonably allocates processing resources according to the importance and timeliness of the data. This multi-dimensional evaluation method ensures the scientificity and reliability of data processing.

[0068] It is particularly noteworthy that the present invention innovatively introduces a set of data processing compensation equations, which is a compensation mechanism based on a mathematical model. The compensation equations establish an error correction model for data processing by considering the characteristic parameters of multiple links such as data filtering, conversion, aggregation and fusion. This compensation mechanism can effectively reduce the cumulative error in the data processing process and improve the accuracy of the processing results. At the same time, the application of the microservice architecture provides good system scalability, and the design of the data synchronization consistency matrix ensures the consistency of data in a distributed environment.

[0069] A specific embodiment 1 of the present invention is provided below, and the specific implementation method of each step in this embodiment 1 is described in detail as follows.

[0070] The specific implementation of step S10 is as follows. First, the original data stream is collected by the IoT devices at the industrial production site. In order to extract valuable information from the original data stream, it is necessary to decompose it into time series. Specifically, the original data stream is decomposed into Decomposition into stable data components and the fluctuation data component Two parts, namely .in, is the noise component.

[0071] Stable data component It can be expressed as a linear combination: ,in is the decomposition coefficient, is the basis function, is the number of basis functions. This can effectively extract the basic trend of the data.

[0072] Fluctuation data component You can use To indicate that is the volatility coefficient, is the wave basis function, is the number of fluctuation basis functions. This part reflects the short-term fluctuation characteristics of the data in the time series.

[0073] By decomposing the original data stream into time series, the stable data component and the fluctuating data component can be effectively extracted, laying the foundation for subsequent data processing and analysis. The main purpose of this step is to separate the useful information from the noise component in the original data stream.

[0074] The specific implementation of step S20 is as follows. First, establish a stability evaluation matrix , which contains the stability coefficients of the stable data components and the volatility coefficient of the volatility data component ,Right now: ; Among them, the stability coefficient It reflects the stability of the data in the time series, and its value range is . Volatility coefficient It indicates the cross-influence between different data components. The specific values ​​of these coefficients can be determined by statistical analysis of historical data.

[0075] After having the stability assessment matrix, we can build a data model. The data model contains information such as data identification, data type, data source, data timestamp and data value. This information provides an important basis for subsequent data processing and analysis.

[0076] The main purpose of this step is to establish a data model that comprehensively describes the data quality and characteristics, providing basic support for subsequent data processing. The stability evaluation matrix can quantitatively reflect the stability of the data and the relationship between different components.

[0077] The specific implementation of step S30 is as follows. First, according to the stability evaluation matrix obtained in step S20, the data collection period and the data collection timestamp are calculated. The data collection period reflects the frequency of data update, which is usually determined according to data characteristics and application requirements. The data collection timestamp indicates the collection time of each data record.

[0078] With this information, we can construct the temporal correlation matrix . This matrix describes the degree of association between adjacent data within the data collection cycle, and the calculation formula is: ; in, and Respectively Moment and The data value at the moment, and is the corresponding data mean, is the size of the time window.

[0079] At the same time, the time contribution value of the data can also be calculated : ; in, is the time weight, is the time attenuation coefficient, It is the data timestamp. This time contribution value reflects the importance of the data in the time series.

[0080] The main purpose of this step is to establish matrices and parameters that describe the characteristics of the data time series. The time series correlation matrix can quantify the degree of correlation between adjacent data, and the time contribution value reflects the importance of the data in the time series. This information provides a basis for subsequent data analysis and processing.

[0081] The specific implementation of step S40 is as follows. First, according to the data time contribution value calculated in step S30 , establish a data weight matrix. This matrix includes three indicators: data importance coefficient, timeliness coefficient and normalization coefficient.

[0082] The importance coefficient of the data can be calculated based on the time contribution value. Specifically, the higher the importance coefficient of the data, the greater the relative value of the data in the entire data set.

[0083] The timeliness coefficient of data indicates the validity of the data in the time series. It can be determined based on the timestamp and update frequency of the data. The higher the timeliness coefficient, the newer the data is and the more valuable it is for reference.

[0084] Normalization coefficients are used to eliminate the dimensional differences between different types of data so that the data can be processed and analyzed uniformly. Normalization or standardization methods are usually used for processing.

[0085] With the data weight matrix, the weight coefficient of the data source can be calculated This coefficient comprehensively reflects the importance, timeliness and standardization of the data, and provides a basis for subsequent data distribution.

[0086] The main purpose of this step is to establish a weight matrix that fully describes the data characteristics and provides a basis for subsequent data distribution and processing. Through quantitative evaluation of data importance, timeliness and standardization, the weight coefficient of each data source can be reasonably determined.

[0087] The specific implementation of step S50 is as follows. First, construct a microservice interface contribution matrix The matrix contains the interface call frequency , Interface response time and interface data throughput Three indicators, namely: ; Among them, the interface call frequency Reflects the usage of the interface in the entire system and the response time represents the real-time performance of the interface, while the data throughput It represents the processing capability of the interface.

[0088] According to the microservice interface contribution matrix, the credibility coefficient of the data source can be calculated and integrity factor The credibility coefficient indicates the credibility score of the data provided by the data source, while the integrity coefficient reflects the performance of the data source in terms of data coverage and continuity.

[0089] The main purpose of this step is to establish a matrix that describes the characteristics of the microservice interface and calculate the credibility and integrity indicators of the data source based on it. These indicators provide an important basis for subsequent data distribution and processing.

[0090] The specific implementation of step S60 is as follows. First, according to the data source weight coefficient calculated in step S40 , distribute the original data stream to each microservice node and establish a data processing pipeline. Then, for each microservice node, calculate the data processing efficiency matrix . The matrix includes: 1. Data filtering efficiency coefficient : The value range is , which reflects the ability to filter noise from raw data.

[0091] 2. Data filtering threshold : Determines the strength of filtering.

[0092] 3. Data type conversion matrix : Describes the conversion relationship between different types of data.

[0093] 4. Data aggregation window size : Affects the granularity of time series aggregation of data.

[0094] By setting these parameters properly, you can ensure that the data processing pipeline runs efficiently.

[0095] The main purpose of this step is to establish a matrix that describes the efficiency of data processing and provide a basis for subsequent data compensation calculations. By optimizing key parameters such as filtering, conversion, and aggregation, the quality and efficiency of data processing can be improved.

[0096] The specific implementation of step S70 is as follows: The data processing efficiency matrix obtained in step S60 , construct the data processing compensation equation group. The equation group includes:

[0097] 1. Data filtering compensation equation: ; in, is the filtering error term, and its value range is .

[0098] 2. Data conversion compensation equation: ; in, is the conversion error term, and its value range is .

[0099] 3. Data aggregation compensation equation: ; in, is the time attenuation coefficient, and its value range is , is the aggregate error term, and its value range is .

[0100] 4. Data fusion compensation equation: ; in, is the fusion error term, and its value range is .

[0101] Through the calculation of these compensation equations, the errors generated in the data processing process can be compensated to ensure that the final data processing results are more accurate and reliable.

[0102] The main purpose of this step is to establish a data processing compensation mechanism to calculate compensation coefficients for key links such as filtering, conversion, aggregation and fusion to improve the accuracy and stability of data processing.

[0103] The specific implementation of step S80 is as follows: First, the corrected data processing results are synchronized. A data synchronization consistency matrix is ​​established. , which contains the data version consistency coefficient , Data content consistency coefficient and data time consistency coefficient Three indicators, namely: ; Among them, the data version consistency coefficient Reflects the consistency of data versions between different processing nodes, data content consistency coefficient Indicates the consistency of data content between different nodes, data time consistency coefficient It describes the consistency of timestamps between different nodes.

[0104] By analyzing and optimizing these three indicators, the synchronization consistency of data in the entire system can be ensured.

[0105] The main purpose of this step is to establish a matrix that describes the consistency of data synchronization, and evaluate and optimize the three key indicators of version, content, and timestamp to ensure that the data remains consistent throughout the system.

[0106] The specific implementation of step S90 is as follows. Finally, the industrial production Internet of Things data extraction matrix is ​​constructed And output. This matrix includes the IoT device data extraction matrix and IoT Data Extraction Quality Matrix Two parts.

[0107] IoT Device Data Extraction Matrix Include: 1. IoT device coverage factor : Reflects the proportion of IoT devices covered by the system to the total number of devices.

[0108] 2. Process data collection coefficient : Indicates the integrity of production process data collection.

[0109] 3. Production parameter extraction coefficient : Represents the extraction coverage of key production parameters.

[0110] 4. Equipment data collection frequency coefficient : Describes the frequency of collecting device data.

[0111] IoT Data Extraction Quality Matrix include: 1. Data extraction completeness coefficient : Indicates whether all the data that need to be collected have been successfully extracted.

[0112] 2. Data extraction accuracy coefficient : reflects the accuracy of the extracted data.

[0113] 3. Data extraction real-time coefficient : Represents the timeliness of the data.

[0114] 4. Data extraction consistency coefficient : Describes the consistency of data between different data sources.

[0115] In order to better understand and implement the present invention, the following provides Example 2 of a specific application scenario of the present invention: A factory is an enterprise that produces automobile parts. A large number of industrial Internet of Things devices, including various sensors, actuators, etc., are used on the production line to collect various data in the production process in real time. In order to give full play to the value of these data and improve production efficiency and product quality, the factory decided to adopt the industrial production Internet of Things data microservice extraction method proposed in the present invention for data management and application.

[0116] First, the factory’s IT department collects and decomposes the raw data stream on the production line into time series. Decomposition calculation, dividing it into stable data components and the fluctuation data component For data collected by a certain type of sensor, its stable data component It can be expressed as ,in , , , , , , is the sampling period of the sensor. And the fluctuation data component You can use To indicate that , , , . Thus, the original data stream It is decomposed into two parts: stable trend and fluctuating characteristics. Figure 2 This is a diagram of data component decomposition and time series correlation analysis. The upper part shows the original data stream X(t) and its decomposed stable data component S(t) and fluctuating data component F(t). The lower part shows the changing trend of the time series correlation coefficient between adjacent time windows.

[0117] Next, the IT department established a stability assessment matrix For the above sensor data, the stability coefficient , , , indicating that the data is relatively stable in time series. , , , reflecting the existence of certain cross-influences between different data components. With this information, the IT department established a detailed data model, including key information such as data identification, type, source, timestamp and data value.

[0118] Then, based on the stability assessment matrix, the IT department calculated the data collection cycle and timestamp. For this sensor, the data collection cycle is set to 5 minutes, that is, data is collected every 5 minutes. Based on the collection timestamp, the IT department constructed a time series correlation matrix . Take two adjacent time points and As an example, the time series correlation coefficient The calculation is as follows: ; in, and They are Moment and The data value at the moment, and is the corresponding data mean, is the size of the time window. Through this calculation, the IT department obtains the time correlation coefficient between adjacent data. At the same time, the time contribution value of the data is calculated based on the timestamp of the data. : ; in, The time contribution value is the collection timestamp of the data. The time contribution value reflects the importance of the data in the time series. Figure 3This is a time contribution decomposition analysis chart, which shows the three components of the time contribution value and their sum, and uses different colors to show the contribution of the three index items and the final comprehensive contribution value.

[0119] With the above feature description, the IT department then established a data weight matrix. For the above sensor data, its importance coefficient is , the timeliness coefficient is , the normalized coefficient is Taking all these factors into consideration, the weight coefficient of this data source is Determined to be .

[0120] At the same time, the IT department also built a microservice interface contribution matrix Taking a microservice interface that collects sensor data as an example, the interface call frequency Times / minute, response time milliseconds, data throughput Kbytes / second. Based on these indicators, the IT department calculates the credibility coefficient of the data source and integrity factor .

[0121] With the above characteristics, the IT department began to build a data processing pipeline. First, according to the data source weight coefficient , distribute the sensor data stream to the corresponding microservice node for processing. Then, for the microservice node, the IT department sets the following data processing efficiency parameters: 1. Data filtering efficiency coefficient , filter threshold ,in is the standard deviation of the data.

[0122] 2. Data type conversion matrix , indicating that no type conversion is required.

[0123] 3. Data aggregation window size , that is, data aggregation is performed once an hour.

[0124] With these parameters, the IT department constructed the corresponding data processing compensation equations. Figure 4 This is a data processing efficiency analysis diagram, which shows the relationship between data filtering efficiency and data deviation, and marks the position of the filtering threshold θ=3σ.

[0125] Taking the data filtering compensation equation as an example, the calculation is as follows: ; in, is the original data value, is the standard deviation, is the filtering error term. The compensation coefficient Will be used to adjust the final data processing results.

[0126] Similarly, the data conversion compensation equation is: ; Since no data type conversion is required, the compensation factor is close to 1.

[0127] The data aggregation compensation equation is: ; in, is the timestamp of each data point, is the reference timestamp, is the time attenuation coefficient, is the aggregation error term. This compensation coefficient will be used to correct the errors in the data aggregation process.

[0128] Finally, the data fusion compensation equation is: ; Among them, the data source weight coefficient calculated previously is used , Credibility Coefficient and integrity factor , is the fusion error term. The compensation coefficient reflects the weight and credibility of the data from the data source in the fusion process.

[0129] By calculating the above compensation equations, the IT department obtained various compensation coefficients and applied them to the final data processing results to ensure the accuracy and reliability of the data.

[0130] Next, the IT department synchronized the corrected data processing results. A data synchronization consistency matrix was established. , where the data version consistency coefficient , data content consistency coefficient , data time consistency coefficient By optimizing these three indicators, the synchronization consistency of data between different microservice nodes is ensured.

[0131] Finally, the IT department built a comprehensive industrial production IoT data extraction matrix Among them, the IoT device data extraction matrix The indicators are as follows: IoT device coverage factor ; Process data collection coefficient

[0132] Production parameter extraction coefficient ; Equipment data collection frequency coefficient ; The IoT data extraction quality matrix The indicators are: Data extraction completeness factor ; Data extraction accuracy coefficient ; Data extraction real-time coefficient ; Data extraction consistency coefficient ; Combining the above characteristic parameters, the industrial production IoT data extraction matrix of the factory is This reflects the overall quality of data collection. Figure 5 The radar chart for data synchronization consistency shows the three key indicators of data synchronization: version consistency (α), content consistency (β), and time consistency (γ). Through dynamic monitoring and optimization of the matrix, the IT department continuously improves the efficiency and reliability of data collection, meeting the needs of production management, fault diagnosis and other applications.

[0133] It should be noted that the variables involved in the present invention are explained in detail as shown in Table 1 below.

[0134] Table 1 Variable explanation table

[0135] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for extracting microservices from industrial production Internet of Things data, characterized in that: The following steps are involved: Collect IoT device data at industrial production sites to obtain raw data streams, establish a stability assessment matrix and a timing correlation matrix, construct a data weight matrix and a microservice interface contribution matrix, calculate a data processing efficiency matrix, construct a data processing compensation equation group and calculate the data processing compensation value, establish a data synchronization consistency matrix, and finally construct and output an industrial production IoT data extraction matrix, which includes an IoT device data extraction matrix and an IoT data extraction quality matrix.

2. The method for extracting industrial production Internet of Things data microservices according to claim 1, characterized in that: Decomposition calculation is performed on the original data stream to decompose the original data stream into a stable data component and a fluctuating data component; the stability evaluation matrix includes a stability coefficient of the stable data component and a fluctuation coefficient of the fluctuating data component.

3. The method for extracting industrial production Internet of Things data microservices according to claim 1, characterized in that: Building a data model based on the stability assessment matrix, the data model including data identification, data type, data source, data timestamp, and data value; The data time contribution value is calculated based on the time series correlation matrix.

4. The method for extracting industrial production Internet of Things data microservices according to claim 1, characterized in that: The data weight matrix includes data importance coefficient, data timeliness coefficient, and data normalization coefficient; the microservice interface contribution matrix includes interface call frequency, interface response time, and interface data throughput.

5. The method for extracting industrial production Internet of Things data microservices according to claim 1, characterized in that: The data processing efficiency matrix includes data filtering efficiency coefficient, data filtering threshold, data type conversion matrix, and data aggregation window size; the data synchronization consistency matrix includes data version consistency coefficient, data content consistency coefficient, and data time consistency coefficient.

6. The method for extracting industrial production Internet of Things data microservices according to claim 1, characterized in that: The data processing compensation equation group includes a data filtering compensation equation, a data conversion compensation equation, a data aggregation compensation equation, and a data fusion compensation equation.

7. The method for extracting industrial production Internet of Things data microservices according to claim 1, characterized in that: The IoT device data extraction matrix includes an IoT device coverage coefficient, a process data collection coefficient, a production parameter extraction coefficient, and an equipment data collection frequency coefficient.

8. The method for extracting industrial production Internet of Things data microservices according to claim 1, characterized in that: The IoT data extraction quality matrix includes a data extraction completeness coefficient, a data extraction accuracy coefficient, a data extraction real-time coefficient, and a data extraction consistency coefficient.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed in a computer, they are used to execute the industrial production Internet of Things data microservice extraction method according to any one of claims 1 to 8.

10. An industrial production Internet of Things data microservice extraction system, characterized in that: The system comprises the computer-readable storage medium as claimed in claim 9, wherein the system is any one of a computer, a server, and a single-chip microcomputer, the computer-readable storage medium is arranged in the system, and the system is provided with a microprocessor for executing program instructions stored in the computer-readable storage medium.

Images