A multi-modal industrial process automatic fault diagnosis method and system

Abstract

The application relates to a kind of multimodal industrial process automatic fault diagnosis method and system, including utilizing network structure search to carry out automatic network design and parameter automatic optimization to industrial process data, the expressiveness of utilizing multi-channel convolution enhancement network is enhanced, and the automatic fault diagnosis of single mode industrial process is realized.Combined with the information of previous mode for new mode in meta-learning.The application overcomes the problems that the network structure design of existing deep neural network-based industrial process fault diagnosis method is complicated, and parameter optimization is consumed, and the utilization of previous design experience is insufficient, by combining network structure search, multi-channel convolution and meta-learning, considering the connection between industrial multimodal, and has theoretical and practical significance for automatic fault diagnosis of multimodal industrial process.

Description

Technical Field

[0001] This invention belongs to the field of fault diagnosis in industrial production processes, specifically a multimodal industrial process automatic fault diagnosis method and system. Background Technology

[0002] With the continuous upgrading of industrial production equipment and the significant improvement in the automation and digitalization of processes, fault diagnosis technology in industrial processes has become an important component for preventing accidents, ensuring safe industrial production, and promoting robust industrial development. Fault diagnosis utilizes process-related variables to determine the type of equipment failure. In industrial processes, equipment failures can lead to decreased production efficiency, downtime, and even personnel injuries or fatalities. When a failure occurs, accurate fault diagnosis and appropriate measures to quickly restore the process to normal are crucial. Therefore, fault diagnosis is of great significance for ensuring the safe and stable operation of industrial processes.

[0003] Traditional fault diagnosis methods mainly rely on statistical learning and deep learning to build models for fault diagnosis. Model design is difficult and requires experts in related fields, which is very time-consuming. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention relates to an automatic fault diagnosis method and system for multimodal industrial processes. It includes automatically designing networks and optimizing parameters for industrial process data using network structure search, enhancing the network's expressiveness through multi-channel convolution, and achieving automatic fault diagnosis for single-modal industrial processes. Simultaneously, it incorporates meta-learning to learn information from previous modes for use in new modes. By combining network structure search, multi-channel convolution, and meta-learning, and considering the connections between multiple industrial modalities, this invention overcomes the problems of cumbersome network structure design, time-consuming parameter optimization, and insufficient utilization of previous design experience in existing deep neural network-based industrial process fault diagnosis methods. This invention has both theoretical and practical significance for automatic fault diagnosis of multimodal industrial processes.

[0005] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0006] An automated fault diagnosis method for multimodal industrial processes includes the following steps:

[0007] Model building: A network model for meta-learning network structure search is built to search for and train the optimal multi-channel convolutional neural network to obtain a fault diagnosis model;

[0008] Search phase: Operations are applied to the raw industrial production process data of multiple modalities to generate candidate channels, a candidate operation set is defined, and the structure search within the network unit is used to automatically design the network for the industrial process data to obtain the initial network parameters for the new modality; the raw industrial multimodal data is sensor data from the industrial production process.

[0009] Training and optimization phase: The structure between network units is used for training, which enables automatic parameter optimization and network pruning to obtain the fault diagnosis network model corresponding to the new mode.

[0010] Real-time detection: Real-time acquisition and preprocessing of sensor data from the industrial production process to be detected, input into the new modal fault diagnosis network model, to realize fault detection and fault diagnosis of the new modal industrial production process.

[0011] The network model for meta-learning network structure search includes: two branches linked by several convolutional neural network units, with each network unit within and between branches connected by edge operations. The data to be processed is input at the beginning of the two branches, and the fully connected layer for outputting fault diagnosis results is connected at the end of the two branches. The network unit also includes edge operations and nodes. The input within the unit is also divided into two paths, and the output is one path.

[0012] The search phase includes:

[0013] Step 201: Normalize and preprocess the original industrial production process data of multiple modalities to make the data dimensions that meet the search of meta-learning network structure.

[0014] Step 202: Apply operations to the preprocessed data to form candidate channels for multi-channel convolution, and concatenate them with the preprocessed data to generate the input for the network search stage;

[0015] Step 203: Assign weights to the candidate channels to obtain the mixed input;

[0016] Step 204: Define the candidate operation set and assign a weight to each operation;

[0017] Step 205: Iterate through steps 203-204 above, select the Adam optimizer, use the cross-entropy loss function, and backpropagate to adjust the network parameters, channel weight parameters, and operation weight parameters, thereby obtaining the optimal initial network parameters, optimal initial channel parameters, and optimal initial operation parameters as the initial parameters for the new mode.

[0018] Step 3 includes:

[0019] Step 301: Normalize and preprocess the dimensions of the new modality industrial production process data to meet the input data dimensions for meta-learning network structure search;

[0020] Furthermore, the industrial production process data of the new modality and the original industrial production process data of multiple modalities are all data from the same production line, only with different feed ratios. The modality refers to different feed ratios; one ratio represents one modality, multiple modalities represent historical data of multiple ratios, and the new modality represents industrial data with the new ratio.

[0021] Step 302: Use the optimal initial network parameters, optimal initial channel weight parameters, and optimal initial operation parameters as initial parameters to train the new modal industrial production process data of the input network, and obtain the optimized network parameters, channel weight parameters, and operation weights after training;

[0022] Step 303: By optimizing the channel weight parameters and operation weights, select the convolution channels and operations in the network to obtain the fault diagnosis network model corresponding to the new mode.

[0023] Furthermore, the preprocessing includes the following steps:

[0024] Step 1-1: Normalization, with a mapping range of [-1, 1];

[0025] Step 1-2: Process the normalized multimodal industrial production process data into a two-dimensional form C = (n × n) required for meta-learning network structure search, where the dimension of the original data is (1 × n^2). If the dimension of the original data is less than n^2, padding is performed.

[0026] The operation applied to data channel C is as follows: Candidate channels for multi-channel convolution The concatenation process generates the input for the web search stage. ;in, The elements inside are operations;

[0027] The weighting of the input channels is described as follows ;

[0028] Further, mixed inputs are obtained:

[0029] (1)

[0030] in, For network units or nodes within a unit, This represents the weight of the i-th input within input I. This represents the weights of all inputs within input I.

[0031] The candidate operation set includes the following convolution, pooling, and skip operations: 3×3 separable convolution, 5×5 separable convolution, 3×3 dilated convolution, 5×5 dilated convolution, 3×3 max pooling, 3×3 average pooling, keeping the original input, and clearing the original input.

[0032] use Indicates on the edge A set of M candidate operations is given, and each operation is assigned a weight. The Softmax function is used to transform these weights, mixing all operations in operation set O together to obtain a mixed output.

[0033] (2)

[0034] in, This represents the output of a network cell or an operating node within a cell. Representation of operation set Any operation in, Representation of operation set The weight of any operation in the process, Representation of operation set The weights of all operations.

[0035] The three initial parameters of the backpropagation adjustment and optimization network include:

[0036] By combining the mixed input and mixed output into the loss function and optimizing the objective function, and iteratively solving for the three optimization parameters, the optimal initial network parameters can be obtained. Optimal channel weight parameters Optimal operation weight parameters ;

[0037] The objective function is: (3)

[0038] It utilizes the optimal network initial parameters from the previous modes. Optimal channel weight parameters Optimal operation weight parameters Network parameters are obtained by training with new modal data. Channel weight Operating parameters .

[0039] The final network model is generated by obtaining the input channels and performing operations according to the following rules, and then trained as the final fault diagnosis model:

[0040] a. Select the input channels corresponding to the top three weights as the network's input channels;

[0041] b. All network units are generated from the weight matrix obtained after the search is completed;

[0042] c. For each edge of a network unit and each node within a unit, retain the operation with the largest weight. For each intermediate network unit or node within a unit, retain the predecessor nodes corresponding to the two largest edges.

[0043] Furthermore, the final generated network includes 8 network units, of which the 3rd and 6th are reduction units, and the rest are normal units.

[0044] The aforementioned multimodal industrial process automatic fault diagnosis system includes sensing devices, a processor, and a memory; the sensing devices include a temperature sensor, a speed sensor, and a weight sensor, used to collect data from the industrial production process and send it to the processor; the memory stores a program, and the processor reads the program and executes the method steps described above to realize automatic fault diagnosis of the multimodal industrial production process.

[0045] The present invention has the following beneficial effects and advantages:

[0046] This invention provides an automatic fault diagnosis method and system for multimodal industrial production processes. It automatically designs networks and optimizes parameters for industrial process data through network structure search, and enhances the network's expressiveness using multi-channel convolution to achieve automatic fault diagnosis for single-modal industrial processes. Simultaneously, it incorporates meta-learning to learn information from previous modes for use in new modes. By combining network structure search, multi-channel convolution, and meta-learning, this invention considers the connections between multiple industrial modalities, overcoming the problems of cumbersome network structure design, costly parameter optimization, and insufficient utilization of previous design experience in existing deep neural network-based industrial process fault diagnosis methods. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method of the present invention.

[0048] Figure 2 This refers to a normal unit in the method search results.

[0049] Figure 3 This refers to the reduced units in the method search results.

[0050] Figure 4 This is the network structure diagram of the method. Detailed Implementation

[0051] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0053] like Figure 1 The diagram shown is a flowchart of the method of the present invention.

[0054] An automatic fault diagnosis method for industrial production processes involves normalizing industrial production process data and padding it with zeros to convert it into a two-dimensional matrix, forming the original convolutional channels. Then, operations are applied to the original channels to generate candidate convolutional channels for multi-channel convolution. Next, the optimal convolutional channels and weights for the convolutional operations are searched and trained. Finally, the optimal convolutional channels and operations are selected based on the weights to form the final network structure for fault diagnosis. The programming language used in the program execution steps of this invention is not limited to MATLAB, Python, etc.

[0055] In this method, the new modal industrial production process data and the original industrial production process data of multiple modalities are all data from the same production line, only with different feed ratios. The modality refers to different feed ratios; one ratio represents one modality, multiple modalities represent historical data of multiple ratios, and the new modality represents industrial data of the new ratio.

[0056] The network model for meta-learning network structure search described in this method includes: two branches connected by several convolutional neural network units, and the network units within and between branches are linked by edge operations. The data to be processed is input at the beginning of the two branches, and the fully connected layer for outputting fault diagnosis results is connected at the end of the two branches. The network unit also includes edge operations and nodes. The input within the unit is also divided into two paths, and the output is one path.

[0057] The specific steps of this invention are as follows:

[0058] Step 1: Data processing and normalization;

[0059] Step 1-1: Normalize the measured industrial production data using the Min-Max standardization method, with a mapping range of [-1, 1];

[0060] The industrial production data is normalized using the Min-Max standardization method, as shown in the following formula:

[0061]

[0062] in, This represents actual industrial production data. Furthermore, manually labeled fault diagnosis categories are used as category labels for training.

[0063] Steps 1-2: Process the data into multi-channel data required for meta-learning network structure search, in the form of (U×C×H×W). U is the batch size of the data, C is the number of channels, H is the data height, and W is the data width. Through experiments, the number of channels was 7 in the search phase and 3 in the training phase, and U was 64 in both the search and training phases. The original industrial data was processed into four-dimensional data in batches of 64 data points each.

[0064] Step 2: Establish a meta-learning network structure search model, and use this model to perform convolution operations on four-dimensional data to fully extract the features of industrial production process data.

[0065] Step 2-1: Use the meta-learning network structure search model to perform convolution operations on the four-dimensional data to fully extract the features of the industrial process data.

[0066] The meta-learning network structure search model in this example is a multi-channel convolutional neural network, with each layer built on a two-dimensional convolutional model to create a multi-channel convolutional model.

[0067] Step 3: Generate a new channel

[0068] Step 3-1: Apply operation to the raw data channel C Candidate channels for multi-channel convolution .

[0069] Step 3-2: Concatenate with the original data to form the input for the web search stage.

[0070] Step 4: Assign channel weights:

[0071] Assign weights to the input channels A mixed input is obtained through channel weights.

[0072] (1)

[0073] in, For network units or nodes within a unit, This represents the weight of the i-th input within input I. This represents the weights of all inputs within input I.

[0074] Step 5: Select operations such as convolution, pooling, and skip connections (keeping the original input) and assign weights:

[0075] Step 5-1: The candidate operations selected are: 3×3 separable convolutions (sep_conv_3×3), 5×5 separable convolutions (sep_conv_5×5), 3×3 dilated convolutions (dil_conv_3×3), 5×5 dilated convolutions (dil_conv_5×5), 3×3 max pooling (max_pool_3×3), 3×3 average pooling (average_pool_3×3), identity preservation (identity, skip_connect), and zeroing out the original input (zero).

[0076] Step 5-2: Use Indicates on the edge The set of M candidate operations. These candidate operations are discrete, and each operation is assigned a weight. The Softmax function is used to transform these weights, mixing all operations in operation set O together to obtain a mixed output. (2)

[0077] in, This represents the output of a network cell or an operating node within a cell. Representation of operation set Any operation in, Representation of operation set The weight of any operation in the process, Representation of operation set The weights of all operations.

[0078] Step 6: Define the objective function, training channel weights, network parameters, and operation weights:

[0079] Step 6-1: Select the Adam optimizer, use the cross-entropy loss function, and optimize the objective as follows:

[0080] (3)

[0081] Step 6-2: Iterate repeatedly to solve for the three optimization parameters to obtain the optimal initial network parameters. Optimal channel weight parameters Optimal operation weight parameters .

[0082] Step 7: Process the new modal data, train the network based on the optimal initial parameters, and obtain the final fault diagnosis network model.

[0083] Step 7: Preprocess the new modal industrial production process data and convert it into input for meta-learning network structure search.

[0084] Step 8: Utilize the optimal network initial parameters from Step 6 Optimal channel weight parameters Optimal operation weight parameters , as the initial parameters of the network;

[0085] Step 9: Train the network using the new modal data to obtain the network parameters. Channel weight Operating parameters Based on channel weight The top three input channels with the highest weights are selected as the network's input channels. Each network unit is processed using the weight matrix obtained after the search. and In the generation process, each edge of a network unit retains the operation with the largest weight, and each intermediate node retains the predecessor nodes corresponding to the two largest edges. For example... Figure 2 For normal units in the method search results, such as Figure 3 For reducing units in the method search results, such as Figure 4 The diagram shows the network structure of the method. The final network contains 8 network units, of which the 3rd and 6th are reduction units, and the rest are normal units. The connection method between units is the same as the internal connection method of the generation unit.

[0086] Based on the above criteria, the input channels and operations are used to form the final network model, which is then trained as the final fault diagnosis model.

[0087] Step 10: Real-time detection data preprocessing involves normalizing and converting the real-time detection data into an input form, then inputting it into the fault diagnosis model to obtain the fault diagnosis category and achieve real-time detection.

[0088] Meanwhile, the present invention provides an automatic fault diagnosis system for multimodal industrial production processes, including sensing devices, a processor, and a memory; the sensing devices include a temperature sensor, a speed sensor, and a weight sensor, used to collect data during industrial production processes and send it to the processor; the memory stores a program, and the processor reads the program and executes the method steps described above to realize automatic fault diagnosis of multimodal industrial production processes.

[0089] In summary, this invention utilizes network structure search to automatically design networks and optimize parameters for industrial process data. It leverages multi-channel convolution to enhance network expressiveness, achieving automatic fault diagnosis for single-modal industrial processes. Simultaneously, it incorporates meta-learning to learn information from previous modalities for use in new modalities. By combining network structure search, multi-channel convolution, and meta-learning, and considering the connections between multiple industrial modalities, this invention overcomes the problems of cumbersome network structure design, resource-intensive parameter optimization, and insufficient utilization of previous design experience in existing deep neural network-based industrial process fault diagnosis methods. This invention has both theoretical and practical significance for achieving automatic fault diagnosis in industrial production processes.

[0090] The embodiments described above will help those skilled in the art to further understand the present invention, but do not limit the present invention in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

Claims

1. A method for automatic fault diagnosis in multimodal industrial processes, characterized in that, Includes the following steps: Model Establishment: A network model for meta-learning network structure search is established to search for and train the optimal multi-channel convolutional neural network to obtain a fault diagnosis model. The network model for meta-learning network structure search includes: two branches connected by several convolutional neural network units, and the network units within and between the branches are linked by edge operations. The data to be processed is input at the beginning of the two branches, and the fully connected layer for outputting fault diagnosis results is connected at the end of the two branches. The network unit also includes edge operations and nodes. The input within the unit is also divided into two paths, and the output is one path. Search Phase: Operations are applied to the raw industrial production process data of multiple modalities to generate candidate channels, a candidate operation set is defined, and automatic network design is performed on the industrial process data using structure search within network units to obtain the initial network parameters for new modalities; the raw industrial multimodal data is sensor data collected by sensing devices in the industrial production process; the sensing devices include temperature sensors, speed sensors, and weight sensors; Training and optimization phase: The structure between network units is used for training, which enables automatic parameter optimization and network pruning to obtain the fault diagnosis network model corresponding to the new mode. Real-time detection: Real-time acquisition and preprocessing of sensor data from the industrial production process to be detected, input into the new modal fault diagnosis network model, to realize fault detection and fault diagnosis of the new modal industrial production process.

2. The automatic fault diagnosis method for multimodal industrial processes according to claim 1, characterized in that, The search phase includes: Step 201: Normalize and preprocess the original industrial production process data of multiple modalities to make the data dimensions that meet the search of meta-learning network structure. Step 202: Apply operations to the preprocessed data to form candidate channels for multi-channel convolution, and concatenate them with the preprocessed data to generate the input for the network search stage; Step 203: Assign weights to the candidate channels to obtain the mixed input; Step 204: Define the candidate operation set and assign a weight to each operation; Step 205: Iterate through steps 203-204 above, select the Adam optimizer, use the cross-entropy loss function, and backpropagate to adjust the network parameters, channel weight parameters, and operation weight parameters, thereby obtaining the optimal initial network parameters, optimal initial channel parameters, and optimal initial operation parameters as the initial parameters for the new mode.

3. The automatic fault diagnosis method for multimodal industrial processes according to claim 1, characterized in that, The training optimization phase includes: Step 301: Normalize and preprocess the dimensions of the new modality industrial production process data to meet the input data dimensions for meta-learning network structure search; Step 302: Use the optimal initial network parameters, optimal initial channel weight parameters, and optimal initial operation parameters as initial parameters to train the new modal industrial production process data of the input network, and obtain the optimized network parameters, channel weight parameters, and operation weights after training; Step 303: By optimizing the channel weight parameters and operation weights, select the convolution channels and operations in the network to obtain the fault diagnosis network model corresponding to the new mode.

4. The automatic fault diagnosis method for multimodal industrial processes according to claim 3, characterized in that, The operation applied to data channel C is Candidate channels for multi-channel convolution The concatenation process generates the input for the web search stage. ;in, The elements inside are operations; Assign weights to the input channels ; Further, mixed inputs are obtained: （1） in, For network units or nodes within a unit, This represents the weight of the i-th input within input I. This represents the weights of all inputs within input I.

5. The automatic fault diagnosis method for multimodal industrial processes according to claim 1, characterized in that, The candidate operation set includes the following convolution, pooling, and skip operations: 3×3 separable convolution, 5×5 separable convolution, 3×3 dilated convolution, 5×5 dilated convolution, 3×3 max pooling, 3×3 average pooling, keeping the original input, and clearing the original input. use Indicates on the edge A set of M candidate operations is given, and each operation is assigned a weight. The Softmax function is used to transform these weights, mixing all operations in operation set O together to obtain a mixed output. （2） in, This represents the output of a network cell or an operating node within a cell. Representation of operation set Any operation in, Representation of operation set The weight of any operation in the process, Representation of operation set The weights of all operations.

6. The automatic fault diagnosis method for multimodal industrial processes according to claim 2, characterized in that, Backpropagation is used to adjust and optimize the three initial parameters of the network, including: By combining the mixed input and mixed output into the loss function and optimizing the objective function, and iteratively solving for the three optimization parameters, the optimal initial network parameters can be obtained. Optimal channel weight parameters Optimal operation weight parameters ; The objective function is: （3） in, , , The network parameters, channel weights, and operating parameters are obtained by training on the new modality data.

7. The automatic fault diagnosis method for multimodal industrial processes according to claim 1, characterized in that, It utilizes the optimal network initial parameters from the previous modes. Optimal channel weight parameters Optimal operation weight parameters Network parameters are obtained by training with new modal data. Channel weight Operating parameters .

8. The automatic fault diagnosis method for multimodal industrial processes according to claim 7, characterized in that, The final network model is generated by obtaining the input channels and performing operations according to the following rules, and then trained as the final fault diagnosis model: a. Select the input channels corresponding to the top three weights as the network's input channels; b. All network units are generated from the weight matrix obtained after the search is completed; c. For each edge of a network unit and each node within a unit, retain the operation with the largest weight. For each intermediate network unit or node within a unit, retain the predecessor nodes corresponding to the two largest edges.

9. The multimodal industrial process automatic fault diagnosis system according to claim 1, characterized in that, The system includes a sensing device, a processor, and a memory; the sensing device is used to collect data from the industrial production process and send it to the processor; the memory stores a program, and the processor reads the program and executes the steps of the method as described in any one of claims 1-8 to achieve automatic fault diagnosis of the multimodal industrial production process.

Images