Industrial data quality detection method, device, computer equipment, storage medium and computer program product

Through multi-dimensional filtering and quality characteristic coding matching, combined with SPC system and streaming processing programs, real-time aggregation and exception recognition of industrial data is realized, solving the problem of low efficiency in quality data management and control in traditional technology, and improving the quality monitoring of industrial production processes ability.

CN118709120BActive Publication Date: 2024-11-15XIAMEN HITHIUM ENERGY STORAGE TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411181755.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-15
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

The quality data control efficiency in traditional industrial production is not high, and it is difficult to effectively monitor and identify abnormal data, resulting in difficulty in quality control of industrial production processes.

Method used

By obtaining the original data collected during industrial production, multi-dimensional filtering and quality characteristic encoding matching, aggregating data and identifying abnormal data, sending alarm information to the terminal that matches the target quality characteristic, using SPC system, streaming processors and publishing and subscribers Message queues implement real-time processing and aggregation of data.

Benefits of technology

It improves data filtering and aggregation efficiency, enhances the accuracy of abnormal data identification and the accuracy of alarm information, thereby improving the quality control efficiency of industrial production processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118709120B_ABST
    Figure CN118709120B_ABST
Patent Text Reader

Abstract

The present application relates to an industrial data quality detection method, device, computer equipment, storage medium and computer program product. The method includes: obtaining various raw data collected during the industrial production process, matching the quality characteristic coding of the raw data that meets the multi-dimensional filtering conditions, and obtaining the initial quality data carrying the quality characteristic coding; based on the aggregation time condition matched by the quality characteristic coding, aggregating the initial quality data carrying the same quality characteristic coding to obtain the first aggregated data with the aggregation parameter added; based on the aggregation capacity condition matched by the quality characteristic coding, aggregating each first aggregated data according to the aggregation parameter to generate the second aggregated data; identifying abnormal data from each second aggregated data, and determining the target quality characteristic corresponding to the abnormal data; sending the alarm information corresponding to the abnormal data to the prompt terminal matching the target quality characteristic. The use of this method can improve the quality control efficiency of the industrial production process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an industrial data quality detection method, device, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of industrial technology, in order to meet the increasingly high automation requirements of industrial production processes, the level of informatization and intelligence of industrial production is getting higher and higher. At the same time, the amount of data generated in the production process is also increasing exponentially. Taking the manufacturing field as an example, in the industrial production process, there will be various types of quality parameters such as equipment parameters, process parameters, and process detection parameters.

[0003] In order to effectively monitor the industrial production process, it is necessary to analyze the quality data, identify abnormal data and issue alarms, and then effectively control the quality of the industrial production process. However, the industrial production process in traditional technology has the problem of low efficiency in quality data control. Summary of the invention

[0004] Based on this, it is necessary to provide an industrial data quality detection method, device, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of quality data management and control in response to the above technical problems.

[0005] In a first aspect, the present application provides an industrial data quality detection method. The method comprises:

[0006] Acquire various raw data collected during the industrial production process, perform quality characteristic coding matching on the raw data that meets the multi-dimensional filtering conditions, and obtain initial quality data carrying the quality characteristic coding;

[0007] Aggregating the initial quality data carrying the same quality characteristic code based on the aggregation time condition matched by the quality characteristic code to obtain first aggregated data with an aggregation parameter added thereto;

[0008] Based on the aggregation capacity condition matched by the quality characteristic code, aggregating each of the first aggregated data according to the aggregation parameter to generate second aggregated data;

[0009] Identify abnormal data from each of the second aggregated data, and determine a target quality characteristic represented by a quality characteristic code carried by the abnormal data;

[0010] The alarm information corresponding to the abnormal data is sent to the prompt terminal matching the target quality characteristics.

[0011] In one embodiment, the method is implemented based on an SPC system, a streaming program, and a publish-subscribe message queue, wherein the message queue includes at least a first theme queue, a second theme queue, a third theme queue, a fourth theme queue, and a fifth theme queue;

[0012] The first subject queue is used to receive various raw data collected and sent during the industrial production process;

[0013] The SPC system is used to consume the raw data from the first subject queue to obtain the initial quality data and send it to the second subject queue;

[0014] The streaming program is used to consume the initial quality data from the second subject queue to obtain the first aggregated data, and send it to the third subject queue; consume the first aggregated data from the third subject queue to obtain the second aggregated data, and send it to the fourth subject queue; consume the second aggregated data from the fourth subject queue to obtain the abnormal data, and send it to the fifth subject queue;

[0015] The SPC system is also used to consume the abnormal data from the fifth topic queue to generate the alarm information.

[0016] In one of the embodiments, the multidimensional filtering condition is a condition group consisting of a plurality of filtering conditions having a hierarchical nested relationship, and each of the condition groups has a quality characteristic code representing the plurality of filtering conditions;

[0017] The step of performing quality characteristic coding matching on the original data meeting the multi-dimensional filtering condition to obtain initial quality data carrying the quality characteristic coding includes:

[0018] According to the hierarchical nesting relationship of each filtering condition, the original data is filtered at multiple levels to obtain valid data that meets any condition group;

[0019] For each of the valid data, based on the target condition group satisfied by the valid data, the quality characteristic code corresponding to the target condition group is added to the valid data to obtain initial quality data carrying the quality characteristic code.

[0020] In one of the embodiments, the aggregation parameters include a sample size and a sample value corresponding to the first aggregated data;

[0021] The step of aggregating each of the first aggregated data according to the aggregation parameter based on the aggregation capacity condition matched by the quality characteristic code to generate second aggregated data includes:

[0022] Based on the aggregation capacity condition matched by the quality characteristic code, grouping each of the first aggregated data to obtain a plurality of data groups;

[0023] For each of the data groups, based on the sample size and the sample value, calculate a plurality of statistical parameters corresponding to the data group;

[0024] The statistical parameters are combined with basic parameters in the data group to obtain second aggregated data corresponding to the data group.

[0025] In one of the embodiments, the first aggregate data are grouped based on the aggregate capacity condition matched by the quality characteristic code to obtain a plurality of data groups, including:

[0026] Based on the quality characteristic code in the first aggregated data, searching for a control chart type that matches the quality characteristic code;

[0027] According to the aggregation capacity condition matched by the control diagram type, each of the first aggregate data is grouped through a sliding window to obtain a plurality of data groups.

[0028] In one of the embodiments, the second aggregated data includes a plurality of statistical parameters;

[0029] The identifying abnormal data from each of the second aggregated data includes:

[0030] Using the statistical parameters contained in each of the second aggregated data as point data, and constructing a control chart matching the parameter type of the statistical parameters;

[0031] Based on the abnormal data determination condition corresponding to the control chart, abnormal data is identified from the point data included in the control chart.

[0032] In one embodiment, the step of sending the warning information corresponding to the abnormal data to the prompt terminal matching the target quality characteristic includes:

[0033] Based on the matching relationship between the quality characteristics and the industrial production management account, searching for a target management account that matches the target quality characteristics;

[0034] Determine a prompt terminal that has logged in the target management account;

[0035] The warning information corresponding to the abnormal data is sent to the prompt terminal.

[0036] In a second aspect, the present application also provides an industrial data quality detection device. The device comprises:

[0037] The raw data processing module is used to obtain various raw data collected during the industrial production process, perform quality characteristic coding matching on the raw data that meets the multi-dimensional filtering conditions, and obtain initial quality data carrying the quality characteristic coding;

[0038] A first aggregation processing module, configured to aggregate the initial quality data carrying the same quality characteristic code based on an aggregation time condition matched by the quality characteristic code to obtain first aggregated data with an aggregation parameter added thereto;

[0039] A second aggregation processing module, configured to aggregate each of the first aggregated data according to the aggregation parameter based on the aggregation capacity condition matched by the quality characteristic code to generate second aggregated data;

[0040] An abnormal data identification module, used to identify abnormal data from each of the second aggregated data, and determine the target quality characteristics represented by the quality characteristic code carried by the abnormal data;

[0041] The alarm processing module is used to send the alarm information corresponding to the abnormal data to the prompt terminal matching the target quality characteristics.

[0042] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0043] Acquire various raw data collected during the industrial production process, perform quality characteristic coding matching on the raw data that meets the multi-dimensional filtering conditions, and obtain initial quality data carrying the quality characteristic coding;

[0044] Aggregating the initial quality data carrying the same quality characteristic code based on the aggregation time condition matched by the quality characteristic code to obtain first aggregated data with an aggregation parameter added thereto;

[0045] Based on the aggregation capacity condition matched by the quality characteristic code, aggregating each of the first aggregated data according to the aggregation parameter to generate second aggregated data;

[0046] Identify abnormal data from each of the second aggregated data, and determine a target quality characteristic represented by a quality characteristic code carried by the abnormal data;

[0047] The alarm information corresponding to the abnormal data is sent to the prompt terminal matching the target quality characteristics.

[0048] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0049] Acquire various raw data collected during the industrial production process, perform quality characteristic coding matching on the raw data that meets the multi-dimensional filtering conditions, and obtain initial quality data carrying the quality characteristic coding;

[0050] Aggregating the initial quality data carrying the same quality characteristic code based on the aggregation time condition matched by the quality characteristic code to obtain first aggregated data with an aggregation parameter added thereto;

[0051] Based on the aggregation capacity condition matched by the quality characteristic code, aggregating each of the first aggregated data according to the aggregation parameter to generate second aggregated data;

[0052] Identify abnormal data from each of the second aggregated data, and determine a target quality characteristic represented by a quality characteristic code carried by the abnormal data;

[0053] The alarm information corresponding to the abnormal data is sent to the prompt terminal matching the target quality characteristics.

[0054] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0055] Acquire various raw data collected during the industrial production process, perform quality characteristic coding matching on the raw data that meets the multi-dimensional filtering conditions, and obtain initial quality data carrying the quality characteristic coding;

[0056] Aggregating the initial quality data carrying the same quality characteristic code based on the aggregation time condition matched by the quality characteristic code to obtain first aggregated data with an aggregation parameter added thereto;

[0057] Based on the aggregation capacity condition matched by the quality characteristic code, aggregating each of the first aggregated data according to the aggregation parameter to generate second aggregated data;

[0058] Identify abnormal data from each of the second aggregated data, and determine a target quality characteristic represented by a quality characteristic code carried by the abnormal data;

[0059] The alarm information corresponding to the abnormal data is sent to the prompt terminal matching the target quality characteristics.

[0060] The above-mentioned industrial data quality detection method, device, computer equipment, storage medium and computer program product, by acquiring various original data collected in the industrial production process, match the quality characteristic coding of the original data that meets the multi-dimensional filtering conditions, can filter the data and use the quality characteristic coding as a component of the quality data to realize the coding marking of the effective quality data, and obtain the initial quality data carrying the quality characteristic coding, which provides a data basis for the subsequent rapid realization of data aggregation and abnormality identification, and then uses the aggregation time condition and aggregation capacity condition matched by the quality characteristic coding as the conditions for data aggregation, aggregates the initial quality data in time and capacity in sequence, associates multiple aggregation processes with quality characteristics, effectively improves the association strength between the aggregated data and the quality characteristics, and identifies abnormal data from each second aggregated data, determines the target quality characteristics represented by the quality characteristic coding carried by the abnormal data, and sends the alarm information corresponding to the abnormal data to the prompt terminal matching the target quality characteristics, which can realize accurate feedback of the alarm information corresponding to the abnormal data based on the quality characteristic coding, improve the quality detection efficiency and the accuracy of abnormal alarms, and thus improve the quality control efficiency of the entire industrial production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 An application environment diagram of an industrial data quality detection method in an embodiment;

[0062] Figure 2 A schematic diagram of a process of industrial data quality detection method in one embodiment;

[0063] Figure 3 A schematic diagram of a message queue storing data through different topics in one embodiment;

[0064] Figure 4 A schematic diagram of the interaction between the SPC system, Flink and Kafka in one embodiment;

[0065] Figure 5 A schematic diagram of a flow chart of an industrial data quality detection method in another embodiment;

[0066] Figure 6 A schematic diagram of an application environment of an industrial data quality detection method in another embodiment;

[0067] Figure 7 is a structural block diagram of an industrial data quality detection device in an embodiment;

[0068] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] The industrial data quality detection method provided in the embodiment of the present application can be applied to Figure 1 The application environment shown in the figure may include a control device 102, an execution system 104, a server 106, and a terminal 108. The application environment may also include a data storage system 110, which may store data that the server 106 needs to process. The data storage system may be integrated on the server 106, or may be placed on a cloud or other server.

[0071] Among them, the control device 102 is used to control the industrial production process. The control device 102 communicates with the execution system 104 through the network. The execution system 104 can collect and send various original data in the industrial production process to the server 106. The server 106 obtains various original data collected in the industrial production process, matches the quality characteristic code of the original data that meets the multi-dimensional filtering conditions, and obtains initial quality data carrying the quality characteristic code. Based on the aggregation time condition matched by the quality characteristic code, the initial quality data carrying the same quality characteristic code is aggregated to obtain first aggregated data with aggregation parameters added. Based on the aggregation capacity condition matched by the quality characteristic code, each first aggregated data is aggregated according to the aggregation parameter to generate second aggregated data, abnormal data is identified from each second aggregated data, and the target quality characteristic represented by the quality characteristic code carried by the abnormal data is determined. The alarm information corresponding to the abnormal data is sent to the prompt terminal 108 that matches the target quality characteristic. The production management personnel corresponding to the prompt terminal 108 can control the industrial production process through the control device 102 based on the alarm information.

[0072] The control device 102 may be, but is not limited to, a computer, a host computer, a device gateway, a PLC (programmable logic controller), etc.

[0073] The execution system 104 may be, but is not limited to, MES (Manufacturing Execution System), QMS (Quality Management System), IOT (Internet of Things), SAP (Enterprise Resource Planning System), etc.

[0074] The server 106 can be implemented as an independent server or a server cluster composed of multiple servers. In the server or server cluster, functional components such as SPC (statistical process control) tools, streaming programs (such as flink) and message queues (such as kafka) can be integrated.

[0075] The terminal 108 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc.

[0076] In one embodiment, Figure 2 As shown, a method for detecting industrial data quality is provided. Figure 1 The server in the example is used to illustrate the following steps:

[0077] Step S202, obtaining various original data collected during the industrial production process, performing quality characteristic coding matching on the original data that meets the multi-dimensional filtering conditions, and obtaining initial quality data carrying the quality characteristic coding.

[0078] Among them, raw data refers to data acquired or generated by control equipment or execution systems involved in the production process of industrial products. In some embodiments, raw data can be data generated by one or more systems such as MES (manufacturing execution system), QMS (quality management system), IOT (internet of equipment system), SAP (enterprise resource planning system), etc. The raw data can specifically include but are not limited to process parameters, equipment parameters, quality inspection parameters, etc.

[0079] Each item of raw data may be actively sent by the control device or execution system, or may be obtained by the server from the control device or execution system by request. For example, in some embodiments, the control device or execution system may actively send the raw data to a subscription-publishing message queue. The control device or execution system, as a producer of data, sends the raw data to the message queue, and the message queue actively sends the raw data to the subscriber for consumption. The subscriber of the data may be the execution object in the server that performs the next step of processing on the raw data.

[0080] Multidimensional filtering conditions refer to the conditions for filtering the original data. Different dimensions have different filtering conditions. By filtering the original data in multiple dimensions through multidimensional filtering conditions, invalid data that does not meet the data requirements of one or more dimensions can be filtered out, and valid data that meets the data requirements of multiple dimensions can be obtained. In some embodiments, the multidimensional filtering conditions can be a combination of conditions consisting of data conditions corresponding to multiple dimensions whose data ranges are successively reduced. For example, in a certain inspection process of product A, the data range corresponding to product A can include factory, wire drawing, process, equipment, inspection items, product code, material, supplier batch, inspection batch and other dimensions from large to small. Correspondingly, the multidimensional filtering conditions set for product A can include factory filtering conditions, wire drawing filtering conditions, process filtering conditions, equipment filtering conditions, inspection item filtering conditions, product code filtering conditions, material filtering conditions, supplier batch filtering conditions, and inspection batch filtering conditions.

[0081] The quality characteristic code is a unique code used to characterize the quality characteristics. The quality characteristic code corresponds to the multidimensional filtering condition, and different multidimensional filtering conditions correspond to different quality characteristic codes. In some embodiments, the quality characteristic code is obtained by combining the codes corresponding to the filtering conditions of each dimension in the multidimensional filtering condition. The condition composition of the multidimensional filtering condition satisfied by the original data determines the quality characteristic code corresponding to the original data. For example, the multidimensional filtering condition includes the conditions of the six dimensions of ABCDEF, the dimension code corresponding to A is 01, the dimension code corresponding to B is 02, the dimension code corresponding to C is 03, the dimension code corresponding to D is 04, the dimension code corresponding to E is 05, and the dimension code corresponding to F is 06. The original data that meets the multidimensional filtering condition has the quality characteristic code 010203040506.

[0082] Initial quality data refers to initial data that has been filtered and cleaned and carries quality characteristic codes. Data filtering and cleaning includes the above-mentioned multidimensional data filtering process. In some embodiments, data filtering and cleaning may also include filtering out data whose values ​​exceed the upper and lower limits, and only retaining data processing processes within the upper and lower limits. In other embodiments, data filtering and cleaning may also include filtering out non-numeric data, and only retaining numerical data. It can be understood that in specific applications, data filtering and cleaning may include a combination of one or more of the above methods, which are not specifically limited here.

[0083] In one embodiment, the server obtains various raw data collected and actively sent by the control device or execution system during the industrial production process, filters the various raw data through multidimensional filtering conditions, and obtains the raw data that meets the multidimensional filtering conditions, and then determines the quality characteristic code that matches the raw data obtained by filtering based on the matching relationship between the multidimensional filtering conditions and the quality characteristic code, and then adds the quality characteristic code to the raw data to obtain the initial quality data carrying the quality characteristic code.

[0084] Step S204: Aggregate the initial quality data carrying the same quality characteristic code based on the aggregation time condition matched by the quality characteristic code to obtain first aggregated data with aggregation parameters added thereto.

[0085] Among them, the aggregation time condition refers to the condition that the time data in each initial quality data needs to meet during the aggregation process of the initial quality data. The specific content of the aggregation time condition is determined by the quality characteristic coding. Different quality characteristic coding can match different aggregation time conditions according to the quality characteristics they represent. Specifically, according to the control chart adapted to the quality characteristics, the specific aggregation time condition is determined. For example, for the mean-range control chart, the mean-standard deviation control chart, and the median-range control chart, the same or different aggregation time conditions can be configured respectively. For the single value-moving range control chart, the aggregation time condition can be configured as no aggregation.

[0086] Specifically, the aggregation time condition can be a specific time interval range or a duration condition. For example, the time range of 10:00:00-10:04:59 is used as the first aggregation time condition, and the time range of 10:05:00-10:09:59 is used as the second aggregation time condition. If the time of the specified category in the initial quality data is within the range of 10:00:00-10:04:59, then the initial quality data can be considered to meet the first aggregation time condition, and if the time of the specified category in the initial quality data is within the range of 10:00:00-10:04:59, then the initial quality data can be considered to meet the second aggregation time condition. For another example, a duration of 5 minutes is used as the aggregation time condition. If the initial quality data is arranged in chronological order, then the interval between the time of a certain initial quality data and the time of the first initial quality data does not exceed 5 minutes, then the initial quality data is considered to meet the aggregation time condition.

[0087] The first aggregated data refers to a piece of aggregated data obtained by aggregating a group of initial quality data that meets the same aggregation time condition. In the process of aggregating the initial quality data, it is necessary to ensure that the clustered initial quality data have the same quality characteristic code so that the first aggregated data obtained after the clustering can be characterized by the quality characteristic code. For example, a group of initial quality data with the same quality characteristic code within 5 minutes of the time difference is aggregated to obtain a piece of aggregated data representing the group of initial quality data.

[0088] The aggregation parameter is a parameter that characterizes the number and average value of the initial quality data aggregated by the first aggregated data. During the aggregation process, the server will count the number of aggregated initial quality data, and obtain the average value through mean calculation based on the specific values ​​of each initial quality data, and then use the number and average value of the initial quality data as the aggregation parameter to obtain the first aggregated data with the aggregation parameter added.

[0089] Step S206 , based on the aggregation capacity condition matched by the quality characteristic code, aggregate the first aggregated data according to the aggregation parameter to generate second aggregated data.

[0090] Among them, the aggregation capacity condition refers to the quantity condition that the aggregated initial quality data needs to meet during the aggregation process of the initial quality data. The specific content of the aggregation capacity condition is determined by the quality characteristic coding. Different quality characteristic coding can match different aggregation capacity conditions according to the quality characteristics they represent. Specifically, according to the control chart adapted to the quality characteristics, the specific aggregation time condition is determined. For example, for the mean-range control chart, the mean-standard deviation control chart, and the median-range control chart, the same or different specific numerical aggregation capacities can be configured respectively. For the single value-moving range control chart, the aggregation time condition can be configured so that the aggregation capacity is fixed to 1. For example, if the clustering capacity of 10 is used as the aggregation capacity condition, 10 consecutive initial quality data can be clustered to obtain one clustering data representing the 10 initial quality data, thereby obtaining the second clustering data.

[0091] The second aggregated data refers to a piece of aggregated data obtained by aggregating a group of continuous initial quality data that meets the quantity condition. In the process of aggregating the initial quality data, it is also necessary to ensure that the clustered initial quality data have the same quality characteristic code so that the second aggregated data obtained after the clustering can be characterized by the quality characteristic code. For example, a group of 10 consecutive first aggregated data with the same quality characteristic code is aggregated again to obtain a piece of aggregated data representing the group of first aggregated data.

[0092] Different from the clustering process performed according to the aggregation time condition, in the process of aggregation according to the aggregation capacity condition, it is also necessary to calculate at least two statistical parameters of the first aggregation data of the group according to the statistical data type corresponding to the control chart, such as at least two of the mean, median, range, standard deviation, maximum value, minimum value, etc. If the control chart is a mean-range control chart, then the mean and range need to be calculated during the clustering process. If the control chart is a mean-standard deviation control chart, then the mean and standard deviation need to be calculated during the clustering process.

[0093] Step S208: identifying abnormal data from each piece of second aggregated data, and determining a target quality characteristic represented by the quality characteristic code carried by the abnormal data.

[0094] Among them, abnormal data refers to data in the second aggregated data that meets the abnormal data determination rule. In some embodiments, different control charts can be drawn according to various statistical parameters in the second aggregated data. The abnormal points in the control chart can be identified by the abnormal data determination rule corresponding to the control chart, thereby identifying the abnormal data in the second aggregated data.

[0095] The second aggregated data carries the corresponding quality characteristic code. For each second aggregated data, regardless of whether it is abnormal data, since the aggregation process is for data with the same quality characteristic code, the various original data aggregated by the second aggregated data all have the same quality characteristic code. For abnormal data, the target quality characteristic represented by the quality characteristic code can be determined based on the quality characteristic code carried by the abnormal data. The target quality characteristic can represent the specific quality characteristic of the abnormality.

[0096] Step S210, sending the alarm information corresponding to the abnormal data to the prompt terminal matching the target quality characteristics.

[0097] Among them, the alarm information is a prompt information used to characterize the abnormality of the data. The prompt content in the alarm information can pre-configure the corresponding alarm information template according to the target quality characteristics matched by the abnormal data. When abnormal data occurs, the alarm information can be generated by filling the specific abnormal data into the alarm information template corresponding to the target quality characteristics. Among them, in some embodiments, the alarm information specifically includes information such as the time of occurrence, factory, wire drawing, equipment, supplier, material, batch, inspection item, abnormal description, etc.

[0098] For different quality characteristics, the association between the quality characteristics and the control objects can be pre-built according to the control objects of the quality characteristics, and the system account of the control object can be bound to the quality characteristics. After detecting abnormal data, the server can generate alarm information corresponding to the abnormal data, match the bound system account according to the target quality characteristics, and send the alarm information to the prompt terminal logged in with the system account.

[0099] The above industrial data quality detection method obtains various original data collected during the industrial production process, matches the quality characteristic coding of the original data that meets the multi-dimensional filtering conditions, and obtains the initial quality data carrying the quality characteristic coding. It can filter the data and use the quality characteristic coding as a component of the quality data to realize the coding marking of the effective quality data, which provides a data basis for the subsequent rapid realization of data aggregation and abnormality identification. By using the aggregation time condition and aggregation capacity condition matched by the quality characteristic coding as the conditions for data aggregation, the initial quality data is aggregated in sequence according to time and capacity, and multiple aggregation processes can be associated with quality characteristics, effectively improving the association strength between the aggregated data and the quality characteristics; by identifying abnormal data from each second aggregated data, and then determining the target quality characteristics represented by the quality characteristic coding carried by the abnormal data, the alarm information corresponding to the abnormal data is sent to the prompt terminal matching the target quality characteristics, and the accurate feedback of the alarm information corresponding to the abnormal data can be realized based on the quality characteristic coding, thereby improving the overall quality detection efficiency and the accuracy of abnormal alarms, so that abnormal alarms can be responded to in a timely and accurate manner, and the quality control efficiency of the industrial production process can be improved.

[0100] In some embodiments, the industrial data quality detection method is implemented based on an SPC system, a streaming processing program, and a publish-subscribe message queue, and the message queue includes at least a first theme queue, a second theme queue, a third theme queue, a fourth theme queue, and a fifth theme queue;

[0101] The first topic queue is used to receive various raw data collected and sent during the industrial production process;

[0102] An SPC system for consuming raw data from the first subject queue to obtain initial quality data and sending it to the second subject queue;

[0103] A streaming program, for consuming the initial quality data from the second subject queue to obtain first aggregated data, and sending the first aggregated data to the third subject queue; consuming the first aggregated data from the third subject queue to obtain second aggregated data, and sending the second aggregated data to the fourth subject queue; consuming the second aggregated data from the fourth subject queue to obtain abnormal data, and sending the abnormal data to the fifth subject queue;

[0104] The SPC system is also used to consume abnormal data from the fifth topic queue to generate alarm information.

[0105] Among them, the streaming program is a data processing program for processing continuous data streams generated in real time, which can process and analyze data in real time and output results in a continuous manner. In some embodiments, the streaming program can be a flink program, which can process various streaming data in real time and improve the speed of data processing.

[0106] A publish-subscribe message queue is a message queue that receives messages generated by a publisher and automatically sends them to a subscriber for consumption by the subscriber. In some embodiments, the publish-subscribe message queue may be a Kafka message queue, which can provide a publish-subscribe message processing mode and effectively improve data processing efficiency.

[0107] In a publish-subscribe message queue, different types of data can be stored by distinguishing different topics, and different topics can be subscribed and consumed by different subscribers. In some embodiments of the present application, the message queue includes at least a first topic queue, a second topic queue, a third topic queue, a fourth topic queue, and a fifth topic queue, which are respectively used to store five different types of data.

[0108] Among them, Figure 3 As shown, the first theme queue is used to store various original data collected and sent during the industrial production process, the second theme queue is used to store the initial quality data obtained by encoding and matching based on the original data, the third theme queue is used to store the first aggregated data obtained by aggregation based on the original data, the fourth theme queue is used to store the second aggregated data obtained by aggregation based on the first aggregated data, and the fifth theme queue is used to store the abnormal data obtained by abnormal identification based on the second aggregated data.

[0109] In some embodiments, the stream processing program cooperates with the publish-subscribe message queue to implement the above industrial data quality detection method. Specifically, taking flink as the stream processing program and kafka as the publish-subscribe message queue as an example, flink, as a data processing framework, can read data from kafka and perform real-time processing and analysis. At the same time, flink can also write the processing results back to kafka to achieve end-to-end real-time data processing and analysis, such as Figure 4 As shown, the specific interaction process includes the following:

[0110] The control device or execution system sends various raw data collected and sent during the industrial production process to the first topic queue of Kafka;

[0111] After receiving the initial quality data, the first topic queue of Kafka sends the initial quality data to the SPC system according to the publish-subscribe relationship;

[0112] The SPC system consumes raw data from the first topic queue of Kafka, performs quality characteristic coding matching on the raw data that meets the multi-dimensional filtering conditions, obtains the initial quality data carrying the quality characteristic coding, and sends it to the second topic queue of Kafka;

[0113] After receiving the initial quality data, the second topic queue of Kafka sends the initial quality data to Flink according to the publish-subscribe relationship;

[0114] Flink consumes the initial quality data from the second topic queue of Kafka, aggregates the initial quality data with the same quality characteristic code based on the aggregation time condition matched by the quality characteristic code, obtains the first aggregated data with the aggregation parameters added, and sends it to the third topic queue of Kafka;

[0115] After receiving the first aggregated data, the third topic queue of Kafka sends the initial quality data to Flink according to the publish-subscribe relationship;

[0116] Flink consumes the first aggregated data from the third topic queue of Kafka, aggregates each first aggregated data according to the aggregation parameters based on the aggregation capacity condition matched by the quality characteristic encoding, generates the second aggregated data and sends it to the fourth topic queue of Kafka;

[0117] After receiving the second aggregated data, the fourth topic queue of Kafka sends the initial quality data to Flink according to the publish-subscribe relationship;

[0118] Flink consumes the second aggregate data from the fourth topic queue of Kafka, identifies abnormal data from each second aggregate data, and sends it to the fifth topic queue of Kafka;

[0119] After receiving the abnormal data, the fifth topic queue of Kafka sends the initial quality data to the SPC system according to the publish-subscribe relationship;

[0120] The SPC system consumes abnormal data from the fifth topic queue of Kafka, determines the target quality characteristics represented by the quality characteristic code carried by the abnormal data, and sends the alarm information corresponding to the abnormal data to the prompt terminal matching the target quality characteristics.

[0121] It can be understood that the first topic queue, the second topic queue, the third topic queue, the fourth topic queue and the fifth topic queue contained in Kafka can be located in different servers in the server cluster.

[0122] In this embodiment, by combining the SPC system, the streaming processing program and the publish-subscribe message queue, the respective advantages of the SPC system, the streaming processing program and the publish-subscribe message queue can be fully utilized to perform streaming data processing on the data generated by each process in real time, which can effectively expand the data processing scale, further improve the data processing efficiency in the data processing process, and thus improve the quality control efficiency in the industrial production process.

[0123] In some embodiments, the multi-dimensional filtering condition is a condition group consisting of multiple filtering conditions with a hierarchical nested relationship, and each condition group has a quality characteristic code that characterizes the multiple filtering conditions.

[0124] Furthermore, the quality characteristic coding is matched for the original data that meets the multi-dimensional filtering conditions to obtain initial quality data carrying the quality characteristic coding, including:

[0125] According to the hierarchical nesting relationship of each filtering condition, the original data is filtered at multiple levels to obtain valid data that meets any condition group; for each valid data, based on the target condition group met by the valid data, the quality characteristic code corresponding to the target condition group is added to the valid data to obtain the initial quality data carrying the quality characteristic code.

[0126] The hierarchical nesting relationship refers to the relationship of inclusion and being included between different dimensions. Taking the factory dimension, wire drawing dimension and process dimension involved in the battery production process as an example, the factory dimension contains multiple wire drawing dimensions, and one wire drawing dimension contains multiple process dimensions.

[0127] From the dimension with the largest data range to the dimension with the smallest data range, the corresponding filter conditions are nested in sequence to obtain a condition group. The data that meets each filter condition in the condition group is valid data. It can be understood that different condition groups can be configured for different quality inspection tasks to obtain the conditions of each dimension that need to be met for the quality inspection task.

[0128] In some embodiments, taking the industrial production process of battery manufacturing as an example, the dimensional conditions of quality characteristic configuration include, from large to small data range: factory, wire drawing, process, equipment, inspection item, product code, material, supplier batch, inspection batch. When upstream systems such as MES, QMS, IOT, and SAP generate initial data such as process parameters, equipment parameters, and quality inspection parameters, the initial data can be filtered according to the corresponding condition group to obtain valid data that meets the corresponding condition group.

[0129] In the process of performing quality checks on multiple types of initial data at the same time, the quality characteristic codes that match the initial data can be determined based on the target condition groups that each initial data satisfies. By adding the quality characteristic codes to the initial data, the quality characteristic codes carrying the initial data can be obtained, which facilitates subsequent data aggregation and abnormal alarms based on the quality characteristic data.

[0130] In this embodiment, by matching the quality characteristic code with the condition group composed of multidimensional filtering conditions, the quality characteristics corresponding to the initial data can be determined while filtering the initial data, and the quality characteristic code representing the quality characteristics is added to the initial data, so that each piece of initial data carries the quality characteristic code, effectively improving the data processing efficiency of subsequent data aggregation and abnormal alarm.

[0131] In some embodiments, the aggregation parameters include a sample size and a sample value corresponding to the first aggregated data.

[0132] Furthermore, if Figure 5 As shown, based on the aggregation capacity condition matched by the quality characteristic code, each first aggregation data is aggregated according to the aggregation parameter to generate second aggregation data, including:

[0133] Step S502: grouping the first aggregated data based on the aggregation capacity condition matched by the quality characteristic code to obtain a plurality of data groups.

[0134] Step S504, for each data group, based on the sample size and sample value, calculate multiple statistical parameters corresponding to the data group.

[0135] Step S506, combining the statistical parameters with the basic parameters in the data group to obtain second aggregated data corresponding to the data group.

[0136] The specific capacity threshold of the aggregate capacity condition is determined based on the quality characteristic code. For different quality characteristic codes, different capacity thresholds are pre-configured according to the control chart corresponding to the quality characteristic represented by the quality characteristic code.

[0137] After determining the aggregation capacity condition matched by the quality characteristic code, the first aggregate data can be grouped according to the time sequence of each first aggregate data and the aggregation capacity to obtain multiple data groups, the number of each data group does not exceed the capacity threshold indicated by the aggregation capacity condition, and the quality characteristic codes carried by each first aggregate data contained in each data group are the same. For each data group, the server can extract the sample size and sample value contained in the first aggregate data, wherein the sample size represents the number of data aggregated by the first aggregate data, and the sample value represents the mean value of the data aggregated by the first aggregate data. According to the sample size and sample value, the server can calculate multiple statistical parameters corresponding to the data group.

[0138] Among them, the statistical parameters that the server needs to calculate can be determined according to the statistical data type corresponding to the control chart, and the statistical parameters can specifically include one or more of the values ​​of mean, median, range, standard deviation, maximum value, minimum value, etc. For example, if the control chart includes a mean-range control chart, the statistical parameters that need to be calculated include the mean and range; if the control chart includes a mean-standard deviation control chart, the statistical parameters that need to be calculated include the mean and standard deviation.

[0139] The basic parameters in the data group are common parameters that each first aggregation parameter in the data group has. The basic parameters may specifically include at least one of the parameters such as quality characteristic code, data group sequence number, sample size, data type, etc.

[0140] In this embodiment, the server obtains the second aggregated data corresponding to the data group by combining the statistical parameters with the basic parameters. Without losing the basic information, the server can combine the basic information with the statistical parameters that can effectively characterize the overall distribution of the aggregated data, thereby achieving effective expression of the aggregated data and providing a data basis for subsequently improving the accuracy of abnormal data identification.

[0141] In one embodiment, each first aggregate data is grouped based on the aggregate capacity condition matched by the quality characteristic code to obtain multiple data groups, including:

[0142] Based on the quality characteristic code in the first aggregate data, a control chart type matching the quality characteristic code is searched; and according to the aggregation capacity condition matched by the control chart type, each first aggregate data is grouped through a sliding window to obtain multiple data groups.

[0143] Among them, the control chart type is information used to characterize the type of control chart for abnormal data identification. Different quality characteristic codes can correspond to one or more control chart types. For each type of control chart, a corresponding aggregation capacity condition can be configured. For example, based on the quality characteristic code in the first aggregated data, the server searches through the control chart type to determine that the control chart types that match the quality characteristic code include a mean-range control chart, a mean-standard deviation control chart, a median-range control chart, and a single value-moving range control chart, wherein the aggregation capacity condition of the mean-range control chart is that the number of aggregations is not greater than M, the aggregation capacity condition of the mean-standard deviation control chart is that the number of aggregations is not greater than N, the aggregation capacity condition of the median-range control chart is that the number of aggregations is not greater than L, and the aggregation capacity condition of the single value-moving range control chart is that the number of aggregations is 1, wherein M, N, and L are all positive integers, and the values ​​of M, N, and L can be the same or different.

[0144] Furthermore, after the server finds the control chart type that matches the quality characteristic code and determines the aggregation capacity condition that the control chart type matches, it can group the aggregated data according to the aggregation capacity threshold through a sliding window to obtain multiple data groups that meet the aggregation capacity condition.

[0145] In this embodiment, the server determines the aggregation capacity condition through the control chart type, and can adapt to the differentiated data aggregation processing of different types of control charts, so that the control chart composed of the second aggregated data obtained by aggregation can realize abnormal data identification with different accuracy requirements, thereby improving data processing efficiency.

[0146] In one embodiment, the second aggregated data includes a plurality of statistical parameters; and identifying abnormal data from each second aggregated data includes:

[0147] The statistical parameters contained in each second aggregated data are used as point data to construct a control chart that matches the parameter type of the statistical parameters; based on the abnormal data judgment conditions corresponding to the control chart, abnormal data are identified from the point data contained in the control chart.

[0148] The statistical parameters in the second aggregated data are data obtained by statistical processing based on the aggregated multiple first aggregated data. Specifically, they may include two or more values ​​such as the mean value, median, range, standard deviation, maximum value, minimum value, etc. According to the parameter type of the statistical parameters required by different types of control charts, the statistical parameters of the parameter type are used as the point data of the control chart to construct the control chart.

[0149] In some embodiments, the abnormal data determination rules of the control chart may specifically include the following rules:

[0150] (1) A point is greater than K standard deviations from the center line;

[0151] (2) K consecutive points are on the same side of the center line;

[0152] (3) K consecutive points, all increasing or all decreasing;

[0153] (4) K consecutive points, staggered up and down;

[0154] (5) K points out of K+1 points are more than 2 standard deviations away from the center line (on the same side);

[0155] (6) K points out of K+1 points are more than 1 standard deviation away from the center line (on the same side);

[0156] (7) K consecutive points within 1 standard deviation of the center line (on either side);

[0157] (8) K consecutive points are greater than 1 standard deviation away from the center line (on either side);

[0158] (9) The values ​​of consecutive K points are the same;

[0159] (10) K consecutive points are outside the control limits.

[0160] When there are point data satisfying any of the above abnormal data determination rules, these point data are the abnormal point data in the control chart, and the second aggregated data represented by the abnormal point data are the abnormal data with abnormalities.

[0161] In this embodiment, a control chart is constructed based on statistical parameters to identify abnormal points, which can quickly and effectively identify abnormal data and improve the efficiency of identifying abnormal data. At the same time, by constructing various types of control charts, abnormal analysis can be performed from multiple different angles, which can facilitate the combination of various types of control charts, quickly locate the causes of data anomalies, and improve the quality control efficiency in the industrial production process.

[0162] In some embodiments, the alarm information corresponding to the abnormal data is sent to a prompt terminal that matches the target quality characteristics, including: based on the matching relationship between the quality characteristics and the industrial production management account, searching for a target management account that matches the target quality characteristics; determining a prompt terminal that is logged in with the target management account; and sending the alarm information corresponding to the abnormal data to the prompt terminal.

[0163] Among them, the matching relationship between quality characteristics and industrial production management accounts can be pre-configured data. By managing production data with different quality characteristics according to industrial production management accounts, effective division of responsibility scope can be achieved. Combined with the processing method of adding quality characteristic codes to the original data, each original data that meets the multi-dimensional filtering conditions can be matched to the corresponding industrial production management account. In the event of abnormal data, it can be quickly and effectively matched to the corresponding management object to handle the abnormal situation accordingly, thereby improving quality control efficiency.

[0164] Furthermore, after determining the abnormal data, the server can determine the target quality characteristics where the abnormality occurs based on the quality characteristic code carried by the abnormal data, and based on the matching relationship between the quality characteristics and the industrial production management account, the target management account that matches the target quality characteristics can be quickly found, and the alarm information corresponding to the abnormal data can be sent to the prompt terminal logged in with the target management account, so that the management object can quickly obtain the situation that the data has abnormalities from the prompt terminal and perform corresponding management and control processing according to the specific abnormal data, thereby improving the efficiency of quality control.

[0165] The present application also provides an application scenario, which applies the above industrial data quality detection method. Specifically, Figure 6 As shown in the figure, taking the battery manufacturing industry as an example, the technical ideas of the industrial data quality detection method in this application scenario include: through Apache Flink streaming processing, real-time docking with the original data collected upstream, filtering, cleaning, and grouping the original data, combining with the SPC algorithm, real-time monitoring and analysis results are obtained, and abnormal data sets are alarmed.

[0166] The entire industrial data quality detection process is divided into five stages: real-time collection of various raw data, aggregation based on aggregation time conditions to obtain first aggregated data, aggregation based on aggregation capacity conditions to obtain second aggregated data, abnormal data monitoring and analysis, and alarm output. For ease of description, in the following embodiments, the first aggregated data is defined as sub-group detailed data, and the second aggregated data is defined as sub-group data.

[0167] The first stage is the generation of SPC raw data (i.e. initial quality data).

[0168] When upstream MES, QMS, IOT, SAP and other systems generate data such as process parameters, equipment parameters, quality inspection parameters, etc., they are sent to the first topic of Kafka.

[0169] The SPC system consumes data from the first topic of Kafka and filters the data according to the following three conditions to obtain the original data that meets the multi-dimensional filtering conditions:

[0170] (1) Filter according to the multi-dimensional filtering conditions configured by quality characteristics. According to the data range from large to small, the dimensions are factory, wire drawing, process, equipment, inspection item, product code, material, supplier batch, inspection batch, etc., and retain the data that matches the conditions.

[0171] (2) Filter according to the upper and lower limits of the quality characteristic configuration and only retain data within the upper and lower limits.

[0172] (3) Filter by data type, filter out non-numeric data, and retain numeric data.

[0173] The SPC system adds the quality characteristic codes and quality-related fields corresponding to the multi-dimensional filtering conditions to the original data, and obtains the initial quality data with quality characteristic codes, which is defined as SPC original data. SPC original data includes: quality characteristic codes, factories, wires, processes, equipment, inspection items, materials, supplier batches, inspection batches, statistical types, product codes, and collection time.

[0174] After generating the SPC raw data, the SPC system sends the SPC raw data to the second topic of Kafka, which can be consumed by the database service and saved to the SPC raw data table in the database. It should be noted that the first stage is only the cleaning and filtering of the data. The generated SPC raw data contains all the contents of the raw data plus the fields related to the quality characteristics.

[0175] The second stage is the generation of SPC subgroup detailed data.

[0176] The flink program consumes the SPC raw data from the second topic of kafka. The flink program will group and perform aggregation calculations based on the statistical chart type configured for the quality characteristics and the aggregation time condition. The grouping key is the quality characteristic code.

[0177] Among them, the mean-range control chart, mean-standard deviation control chart, and median-range control chart are grouped and aggregated according to the corresponding aggregation time conditions; the single value-moving range control does not perform aggregation processing.

[0178] The flink program uses the flink window function to perform aggregation processing, outputs the first aggregate data, and defines the first aggregate data as SPC subgroup detail data.

[0179] The flink program sends the SPC subgroup detail data to the third topic of Kafka, and the database service consumes the data in the third topic of Kafka and saves it to the SPC subgroup detail data in the database. The SPC subgroup detail data includes: quality characteristic code, factory, wire drawing, process, equipment, inspection item, material, supplier batch, inspection batch, statistical type, product code, collection time, original data record id, data type, sample size, and sample value.

[0180] The third stage is the SPC subgroup data generation stage.

[0181] The flink program consumes SPC subgroup detail data from the third topic of kafka. The flink program groups and performs aggregation calculations based on the statistical chart type configured for quality characteristics and the aggregation capacity conditions.

[0182] Among them, the mean-range control chart, mean-standard deviation control chart, and median-range control chart are grouped and aggregated according to the corresponding aggregation capacity conditions; the single value-moving range control is processed according to the aggregation capacity of 1.

[0183] The Flink program sends the SPC subgroup data to the fourth topic of Kafka. The database service consumes the fourth topic of Kafka and saves the SPC subgroup data in the database.

[0184] The SPC subgroup data content includes: quality characteristic code, subgroup number, sample size, subgroup detail data record ID, supplier batch, inspection batch, data type, mean, median, range, standard deviation, maximum value, and minimum value.

[0185] The fourth stage is the abnormal point monitoring and analysis stage.

[0186] The flink program consumes SPC subgroup data from the fourth topic of kafka.

[0187] According to the parameter type of the statistical parameters required by different types of control charts, the statistical parameters of the parameter type are used as the point data of the control chart to construct the control chart.

[0188] In some embodiments, the abnormal data determination rules of the control chart may specifically include the following rules:

[0189] (1) A point is greater than K standard deviations from the center line;

[0190] (2) K consecutive points are on the same side of the center line;

[0191] (3) K consecutive points, all increasing or all decreasing;

[0192] (4) K consecutive points, staggered up and down;

[0193] (5) K points out of K+1 points are more than 2 standard deviations away from the center line (on the same side);

[0194] (6) K points out of K+1 points are more than 1 standard deviation away from the center line (on the same side);

[0195] (7) K consecutive points within 1 standard deviation of the center line (on either side);

[0196] (8) K consecutive points are greater than 1 standard deviation away from the center line (on either side);

[0197] (9) The values ​​of consecutive K points are the same;

[0198] (10) K consecutive points are outside the control limits.

[0199] Use Flink's counting window function and sliding step size to perform real-time calculations and output SPC abnormal point data. Specifically, when there are point data that meet any of the above abnormal data judgment rules, these point data are the abnormal point data in the control chart.

[0200] The flink program saves the abnormal point data to the SPC abnormal point data in the database and sends it to the fifth topic of kafka.

[0201] The SPC abnormal point data content includes: quality characteristic classification code, quality characteristic code, abnormal processing order number, occurrence time, inspection item code, inspection item name, processing status, control chart sub-chart, abnormality judgment sub-group number, control chart type, control chart name, factory, wire drawing, process, equipment, product, supplier, abnormal rule, and statistical type.

[0202] The fifth stage is the alarm information output stage.

[0203] The SPC system consumes the abnormal point data in the fifth topic of Kafka, and queries the corresponding quality characteristics according to the quality characteristic code. It obtains the configured pusher employee number and user information, and generates alarm information. The alarm information specifically includes the time of occurrence, factory, wire, equipment, supplier, material, batch, inspection item, abnormal description, etc. Furthermore, the SPC system pushes the alarm information to the collaborative office platform based on the pusher employee number and user information, so that the pusher can view the alarm information and process it accordingly.

[0204] Based on the above processing, the timeliness of the five processes of SPC, from raw data collection to alarm information output, has been greatly improved. The overall delay has been reduced from more than 30 minutes to less than 30 seconds. On-site personnel can get timely feedback from the system, access and process it in time, improving product quality assurance.

[0205] Furthermore, the combination and application of flink and kafka technologies improves the stability of analysis under large data volumes. Since the amount of data is over 100 million, offline query processing takes a long time and program timeouts and failures may occur. With real-time processing by flink and kafka, the data to be processed is cached in the flink program, the amount of data is reduced, the processing speed is fast, and the stability is greatly improved.

[0206] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0207] Based on the same inventive concept, the embodiment of the present application also provides an industrial data quality detection device for implementing the industrial data quality detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more industrial data quality detection device embodiments provided below can refer to the limitations of the industrial data quality detection method above, and will not be repeated here.

[0208] In one embodiment, Figure 7 As shown, an industrial data quality detection device is provided, including: an original data processing module 702, a first aggregation processing module 704, a second aggregation processing module 706, an abnormal data identification module 708 and an alarm processing module 710, wherein:

[0209] The original data processing module 702 is used to obtain various original data collected during the industrial production process, perform quality characteristic coding matching on the original data that meets the multi-dimensional filtering conditions, and obtain initial quality data carrying the quality characteristic coding;

[0210] A first aggregation processing module 704, configured to aggregate the initial quality data carrying the same quality characteristic code based on the aggregation time condition matched by the quality characteristic code to obtain first aggregated data with an aggregation parameter added thereto;

[0211] A second aggregation processing module 706, configured to aggregate each of the first aggregated data according to the aggregation parameter based on the aggregation capacity condition matched by the quality characteristic code to generate second aggregated data;

[0212] An abnormal data identification module 708 is used to identify abnormal data from each of the second aggregated data, and determine the target quality characteristics represented by the quality characteristic code carried by the abnormal data;

[0213] The alarm processing module 710 is used to send the alarm information corresponding to the abnormal data to the prompt terminal matching the target quality characteristics.

[0214] In one of the embodiments, the industrial data quality detection device includes a message queue based on an SPC system, a streaming processing program, and a publish-subscribe mode, wherein the message queue includes at least a first theme queue, a second theme queue, a third theme queue, a fourth theme queue, and a fifth theme queue;

[0215] The first subject queue is used to receive various raw data collected and sent during the industrial production process;

[0216] The SPC system is used to consume the raw data from the first subject queue to obtain the initial quality data and send it to the second subject queue;

[0217] The streaming program is used to consume the initial quality data from the second subject queue to obtain the first aggregated data, and send it to the third subject queue; consume the first aggregated data from the third subject queue to obtain the second aggregated data, and send it to the fourth subject queue; consume the second aggregated data from the fourth subject queue to obtain the abnormal data, and send it to the fifth subject queue;

[0218] The SPC system is also used to consume the abnormal data from the fifth topic queue to generate the alarm information.

[0219] In one of the embodiments, the multidimensional filtering condition is a condition group consisting of a plurality of filtering conditions having a hierarchical nested relationship, and each of the condition groups has a quality characteristic code representing the plurality of filtering conditions;

[0220] The raw data processing module 702 is specifically used to perform multi-level filtering on the raw data according to the hierarchical nesting relationship of each filtering condition to obtain valid data that meets any one of the condition groups; for each valid data, based on the target condition group met by the valid data, the quality characteristic code corresponding to the target condition group is added to the valid data to obtain initial quality data carrying the quality characteristic code.

[0221] In one of the embodiments, the aggregation parameters include a sample size and a sample value corresponding to the first aggregated data;

[0222] The second aggregation processing module 706 is specifically used to group each of the first aggregation data based on the aggregation capacity condition matched by the quality characteristic code to obtain multiple data groups; for each of the data groups, based on the sample size and the sample value, calculate multiple statistical parameters corresponding to the data group; combine the statistical parameters with the basic parameters in the data group to obtain second aggregation data corresponding to the data group.

[0223] In one of the embodiments, the second aggregation processing module 706 is specifically used to find a control chart type that matches the quality characteristic code based on the quality characteristic code in the first aggregation data; and group each of the first aggregation data through a sliding window according to the aggregation capacity condition matched by the control chart type to obtain multiple data groups.

[0224] In one embodiment, the second aggregated data includes multiple statistical parameters; the abnormal data identification module 708 is used to use the statistical parameters included in each of the second aggregated data as point data to construct a control chart that matches the parameter type of the statistical parameters; based on the abnormal data judgment conditions corresponding to the control chart, identify abnormal data from the point data included in the control chart.

[0225] In one embodiment, the alarm processing module 710 is used to find a target management account that matches the target quality characteristics based on the matching relationship between the quality characteristics and the industrial production management account; determine the prompt terminal logged in with the target management account; and send the alarm information corresponding to the abnormal data to the prompt terminal.

[0226] The above-mentioned industrial data quality detection device, by acquiring various original data collected during the industrial production process, matches the quality characteristic coding of the original data that meets the multi-dimensional filtering conditions, can filter the data and use the quality characteristic coding as a component of the quality data to realize the coding marking of the effective quality data, and obtain the initial quality data carrying the quality characteristic coding, which provides a data basis for the subsequent rapid realization of data aggregation and abnormality identification, and then uses the aggregation time condition and aggregation capacity condition matched by the quality characteristic coding as the conditions for data aggregation, aggregates the initial quality data in time and capacity in sequence, associates multiple aggregation processes with quality characteristics, and effectively improves the association strength between the aggregated data and the quality characteristics. By identifying abnormal data from each second aggregated data, determining the target quality characteristics represented by the quality characteristic coding carried by the abnormal data, and sending the alarm information corresponding to the abnormal data to the prompt terminal that matches the target quality characteristics, it is possible to realize accurate feedback of the alarm information corresponding to the abnormal data based on the quality characteristic coding, improve the quality detection efficiency and the accuracy of abnormal alarms, and thus improve the quality control efficiency of the entire industrial production process.

[0227] Each module in the above industrial data quality detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0228] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an industrial data quality detection method is implemented.

[0229] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0230] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0231] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0232] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0233] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0234] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0235] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0236] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting industrial data quality, characterized in that: The method comprises: Acquire various raw data collected during the industrial production process, perform quality characteristic coding matching on the raw data that meets the multi-dimensional filtering conditions, and obtain initial quality data carrying the quality characteristic coding; Aggregating the initial quality data carrying the same quality characteristic code based on the aggregation time condition matched by the quality characteristic code to obtain first aggregated data with an aggregation parameter added thereto, wherein the aggregation time condition refers to a condition that the time data in each initial quality data needs to satisfy during the aggregation process of the initial quality data; Based on the aggregation capacity condition matched by the quality characteristic code, each of the first aggregated data is aggregated according to the aggregation parameter to generate second aggregated data, wherein the aggregation capacity condition refers to the quantity condition that the aggregated initial quality data needs to meet during the aggregation process of the initial quality data; the aggregation parameter includes the sample quantity and sample value corresponding to the first aggregated data; The step of aggregating each of the first aggregated data according to the aggregation parameter based on the aggregation capacity condition matched by the quality characteristic code to generate second aggregated data includes: Based on the aggregation capacity condition matched by the quality characteristic code, grouping each of the first aggregated data to obtain a plurality of data groups; For each of the data groups, based on the sample size and the sample value, calculate a plurality of statistical parameters corresponding to the data group; Combining the statistical parameter with the basic parameter in the data group to obtain second aggregated data corresponding to the data group; The first aggregated data are grouped based on the aggregated capacity condition matched by the quality characteristic code to obtain a plurality of data groups, including: Based on the quality characteristic code in the first aggregated data, searching for a control chart type that matches the quality characteristic code; According to the aggregation capacity condition matched by the control chart type, each of the first aggregate data is grouped by a sliding window to obtain a plurality of data groups; Identify abnormal data from each of the second aggregated data, and determine a target quality characteristic represented by a quality characteristic code carried by the abnormal data; The alarm information corresponding to the abnormal data is sent to the prompt terminal matching the target quality characteristics.

2. The method according to claim 1, characterized in that The method is implemented based on an SPC system, a streaming program and a publish-subscribe message queue, wherein the message queue comprises at least a first theme queue, a second theme queue, a third theme queue, a fourth theme queue and a fifth theme queue; The first subject queue is used to receive various raw data collected and sent during the industrial production process; The SPC system is used to consume the raw data from the first subject queue to obtain the initial quality data and send it to the second subject queue; The streaming program is used to consume the initial quality data from the second subject queue to obtain the first aggregated data, and send it to the third subject queue; consume the first aggregated data from the third subject queue to obtain the second aggregated data, and send it to the fourth subject queue; consume the second aggregated data from the fourth subject queue to obtain the abnormal data, and send it to the fifth subject queue; The SPC system is also used to consume the abnormal data from the fifth topic queue to generate the alarm information.

3. The method according to claim 1, characterized in that The multidimensional filtering condition is a condition group consisting of a plurality of filtering conditions having a hierarchical nested relationship, and each of the condition groups has a quality characteristic code representing the plurality of filtering conditions; The step of performing quality characteristic coding matching on the original data meeting the multi-dimensional filtering condition to obtain initial quality data carrying the quality characteristic coding includes: According to the hierarchical nesting relationship of each filtering condition, the original data is filtered at multiple levels to obtain valid data that meets any condition group; For each of the valid data, based on the target condition group satisfied by the valid data, the quality characteristic code corresponding to the target condition group is added to the valid data to obtain initial quality data carrying the quality characteristic code.

4. The method according to claim 1, characterized in that: The second aggregated data includes a plurality of statistical parameters; The identifying abnormal data from each of the second aggregated data includes: Using the statistical parameters contained in each of the second aggregated data as point data, and constructing a control chart matching the parameter type of the statistical parameters; Based on the abnormal data determination condition corresponding to the control chart, abnormal data is identified from the point data included in the control chart.

5. The method according to claim 1, characterized in that The step of sending the warning information corresponding to the abnormal data to the prompt terminal matching the target quality characteristic includes: Based on the matching relationship between the quality characteristics and the industrial production management account, searching for a target management account that matches the target quality characteristics; Determine a prompt terminal that has logged in the target management account; The warning information corresponding to the abnormal data is sent to the prompt terminal.

6. An industrial data quality detection device, characterized in that: The device comprises: The raw data processing module is used to obtain various raw data collected during the industrial production process, perform quality characteristic coding matching on the raw data that meets the multi-dimensional filtering conditions, and obtain initial quality data carrying the quality characteristic coding; a first aggregation processing module, configured to aggregate the initial quality data carrying the same quality characteristic code based on an aggregation time condition matched by the quality characteristic code, to obtain first aggregated data with an aggregation parameter added thereto, wherein the aggregation time condition refers to a condition that the time data in each initial quality data needs to satisfy during the aggregation processing of the initial quality data; a second aggregation processing module, configured to aggregate each of the first aggregated data according to the aggregation parameters based on the aggregation capacity condition matched by the quality characteristic code to generate second aggregated data, wherein the aggregation capacity condition refers to the quantity condition that the aggregated initial quality data needs to meet during the aggregation processing of the initial quality data; the aggregation parameters include the sample size and sample value corresponding to the first aggregated data; the second aggregation processing module is specifically configured to search for a control chart type matching the quality characteristic code based on the quality characteristic code in the first aggregated data; group each of the first aggregated data through a sliding window according to the aggregation capacity condition matched by the control chart type to obtain a plurality of data groups; for each of the data groups, based on the sample size and the sample value, calculate a plurality of statistical parameters corresponding to the data group; and combine the statistical parameters with the basic parameters in the data group to obtain second aggregated data corresponding to the data group; An abnormal data identification module, used to identify abnormal data from each of the second aggregated data, and determine the target quality characteristics represented by the quality characteristic code carried by the abnormal data; The alarm processing module is used to send the alarm information corresponding to the abnormal data to the prompt terminal matching the target quality characteristics.

7. The device according to claim 6, characterized in that The device includes a message queue based on an SPC system, a streaming program and a publish-subscribe method, wherein the message queue includes at least a first theme queue, a second theme queue, a third theme queue, a fourth theme queue and a fifth theme queue; The first subject queue is used to receive various raw data collected and sent during the industrial production process; The SPC system is used to consume the raw data from the first subject queue to obtain the initial quality data and send it to the second subject queue; The streaming program is used to consume the initial quality data from the second subject queue to obtain the first aggregated data, and send it to the third subject queue; consume the first aggregated data from the third subject queue to obtain the second aggregated data, and send it to the fourth subject queue; consume the second aggregated data from the fourth subject queue to obtain the abnormal data, and send it to the fifth subject queue; The SPC system is also used to consume the abnormal data from the fifth topic queue to generate the alarm information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.