Sliding contact line power supply fault diagnosis method based on causal inference and knowledge distillation

Abstract

The invention belongs to the technical field of automobile production line fault diagnosis, and relates to a slide wire power supply fault diagnosis method based on causal inference and knowledge distillation, which comprises the following steps: S1, constructing a first feature model based on a deep neural network, and using a slide wire picture in an automobile production line for training the first feature model; s2, using a variational auto-encoder and the first feature model to carry out a de-confusion operation on the slide wire picture in the automobile production line, and generating an intermediate layer feature map; s3, constructing a second feature model based on the knowledge distillation technology, and training the second feature model; and S4, deploying the second feature model in edge equipment on an automobile production line, and performing carbon deposition fault diagnosis. While the model keeps high diagnosis precision, the scale of the training model can be compressed, the training model can capture correct fault features, the reliability of the output result of the fault diagnosis network model is improved, and the actual requirements of industrial production are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile production line fault diagnosis, and in particular relates to a busbar power supply fault diagnosis method based on causal inference and knowledge distillation. Background Art

[0002] In the automobile production industry, mobile devices in the parts transmission system of the production line need to constantly change their positions. At each different position, the mobile devices must be able to obtain power at any time, which puts forward special requirements for the power supply of the automobile parts transmission system - "mobile power supply". The busbar power supply has the characteristics of a large power supply range, strong current conduction capacity, and stable mechanical properties, making it the first choice for power supply of automobile parts transmission systems. The busbar power supply method is that the collector and the guide rail surface slide relative to each other while maintaining contact, so as to achieve the effect of mobile power supply. However, the carbon brush on the collector is made of soft material. After long-term use, the carbon powder on the carbon brush will fall onto the busbar track. After long-term accumulation, the busbar track will be short-circuited, causing serious damage. Therefore, how to detect faults in a timely and effective manner and discover abnormal operating conditions is the key to ensuring the healthy operation of the automobile production line and improving the safety and availability of the system.

[0003] Deep learning is a branch of machine learning (ML) that has a multi-layer network architecture and can automatically extract abstract features from complex raw data, thereby eliminating some of the reliance on experts. In recent years, with the increasing number of sensor types integrated in industrial production fields such as automobile production lines, the data in fault diagnosis has become increasingly complex. It is expensive and time-consuming to obtain large amounts of data and expertise to support mechanism-based fault diagnosis methods. Therefore, deep learning models are favored in the field of automobile production line fault diagnosis due to their powerful learning and feature extraction capabilities.

[0004] However, fault diagnosis methods based on deep learning have problems such as large network scale, large parameters, and high computing power requirements of the computing platform. It is difficult to deploy fault diagnosis models on embedded platforms and mobile devices with limited computing power. At present, most deep learning models still rely on large servers for calculation and generation. However, in the actual industrial production process, the computing platform of the equipment is mainly a small embedded platform with insufficient computing power and limited storage space. Large-scale fault diagnosis networks are difficult to deploy, while small-scale fault diagnosis networks are difficult to extract correct features. Therefore, it is crucial to lightweight the fault diagnosis network.

[0005] Currently popular neural network lightweighting methods include pruning, quantization, knowledge distillation (KD) and neural network structure search. The knowledge distillation method was proposed by Geoffrey Hinton in 2015. It is the process of transferring knowledge from the first feature model to the second feature model, so that the second feature model can obtain powerful feature extraction capabilities from the first feature model at the expense of a certain accuracy, thereby enabling small models to be deployed on hardware platforms with insufficient computing power. At present, many fault diagnosis models based on deep learning have introduced knowledge distillation technology to achieve lightweighting.

[0006] Knowledge distillation can be divided into three types, namely, "response-based", "feature-based" and "relationship-based" knowledge distillation. At present, most fault diagnosis networks adopt the "response-based" knowledge distillation method, which only uses the output of the first feature model without the information in the hidden layer, resulting in a lack of interpretability. Although the "feature-based" knowledge distillation can obtain the information in the hidden layer, it is not necessarily a fault feature, especially in the image-based knowledge distillation. Due to the poor background diversity of carbon deposit fault image data in the automobile production line, the fault diagnosis model easily misidentifies the background features as fault features, and the intermediate layer feature map transmitted from the first feature model to the second feature model tends to focus on the background features rather than the fault features. When the background features of the automobile production line change slightly, the lightweight fault diagnosis model may fail, seriously endangering the safety of automobile production. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention provides a busbar power supply fault diagnosis method based on causal inference and knowledge distillation, which combines causal inference and knowledge distillation, introduces a variational autoencoder as a background generator to reconstruct the background of the input image and extract image features; calculates the reconstruction error between the background and the input image to generate a feature map containing only fault features; and constructs a loss function based on the reconstruction error to train the model. While maintaining the high diagnostic accuracy of the training model, the scale of the training model can be greatly compressed, and the training model can capture the correct fault features, thereby improving the reliability of the output results of the fault diagnosis network model and meeting the actual needs of industrial production.

[0008] To achieve the above object, the present invention discloses the following technical solution:

[0009] A busbar power supply fault diagnosis method based on causal inference and knowledge distillation, comprising:

[0010] S1: Build a first feature model based on a deep neural network, and use busbar images in the automobile production line to train the first feature model;

[0011] S11: Collect busbar images from the automobile production system and pre-process the busbar images to eliminate image noise; divide the busbar images into four fault levels based on the coverage area, thickness and glossiness of carbon deposits in the busbar images; and construct a training set of images X train And the test set image X test ;

[0012] S12: Construct a first feature model. The first feature model is constructed based on a deep neural network model and uses the divided training set image X train Train them;

[0013] S2: De-obfuscate the busbar image using a variational autoencoder and the first feature model in step S1 to generate an intermediate layer feature map; the first feature model includes three submodules: the intermediate layer feature map generated by the first submodule of the first feature model includes the edge of the carbon deposits and the boundary features between the busbar and the carbon deposits; the second submodule of the first feature model can add the shape contour of the carbon deposits to the intermediate layer feature map; the intermediate layer feature map generated by the third submodule of the first feature model includes the distribution features of the carbon deposits on the busbar;

[0014] S21: The training set image X train Input into the trained first feature model and extract the intermediate layer feature map f output by the first feature model t (x);

[0015] S22: The training set image X train Input the variational autoencoder and extract the intermediate layer feature map f output by the variational autoencoder decoder VAE (x);

[0016] S23: Generate a feature map containing only background features using a variational autoencoder, specifically including: linear shape features of the busbar background, light intensity features of the working environment, and color features of the busbar surface; calculate the intermediate layer feature map f of the first feature model in step S21 t (x) and the intermediate layer feature map f of the variational autoencoder decoder in step S22 VAE (x) to obtain the carbon deposition feature map containing only the carbon deposition fault features; and then use kernel convolution and spatial bilinear interpolation to match the number of channels and spatial resolution for processing, and obtain the intermediate layer feature error map as follows:

[0017] ERROR intermediate =MSE(f t (x),Ψ VAE (f VAE (x)));

[0018] Among them, ERROR intermediateis the feature error map of the middle layer; MSE() is the mean square error function; f t (x) is the middle layer feature map of the first feature model; f VAE (x) is the feature map of the middle layer of the variational autoencoder; Ψ VAE () is a bilinear interpolation function used to match the feature map dimensions of the first feature model and the variational autoencoder; x is the input independent variable of the mean square error function:

[0019] S3: Construct a second feature model based on knowledge distillation technology, and use knowledge distillation technology to convert the intermediate layer feature map f output by the first feature model in step S2 t (x) and the training set image X in step S1 train Input the second feature model for training;

[0020] S31: Construct the first loss sub-function of the second feature model; convert the training set image X in step S1 train Input the first feature model and use the feature label q generated by the first feature model soft (x) to design the training process of the second feature model; the output probability distribution q of the second feature model s (x) and the feature label q of the first feature model soft The loss between (x) as the first loss sub-function for training the second feature model is:

[0021]

[0022] Among them, loss soft is the first loss sub-function between the softmax function output of the second feature model and the feature label of the first feature model; H() is the function of cross entropy; q t (x) is the probability distribution of the feature label of the first feature model; q s (x) is the output probability distribution of the second feature model;

[0023] S32: construct a second loss sub-function of the second feature model; in the process of training the second feature model, the intermediate layer feature map f of the second feature model is s (x) and S2 are generated by the first feature model and variational autoencoder, and the intermediate layer feature error map only contains fault features ERROR intermediate Find the difference, as the second loss sub-function for training the second feature model:

[0024] loss feature =MSE(f s (x), ERROR intermediate );

[0025] Among them, lossfeature A second loss sub-function for obtaining fault features for the middle layer of the second feature model; f s (x) is the middle layer feature map of the second feature model;

[0026] S33: construct a third loss sub-function of the second feature model; in the process of training the second feature model, the second feature model outputs a probability distribution q s (x) with one-hot encoded label p s The third loss sub-function between (x) is:

[0027]

[0028] Among them, loss h is the third loss sub-function between the softmax function output of the second feature model and the label of the real data set; p s (x) is a one-hot encoded label;

[0029] S34: Integrate the above loss sub-functions to obtain the loss function Loss of the second feature model s for:

[0030] Loss s =α*loss h +β*loss soft +γ*loss feature ;

[0031] Among them, Loss s is the loss function of the second feature model; a is the first loss function loss h Weight coefficient; β is the second loss function loss soft Weight coefficient; γ is the third loss function loss feature Weight coefficient;

[0032] S4: Deploy the second feature model obtained in step S3 in the edge device on the automobile production line, collect and process the busbar images on the automobile production line, use the second feature model to diagnose carbon deposit faults, and issue an alarm in a timely manner.

[0033] Preferably, the busbar picture in step S1 is specifically a picture of the guide rail and bracket fixing system of the automobile production line; the guide rail of the automobile production line is the core component of the busbar and is responsible for conducting current; the collector obtains current by sliding in contact with the guide rail through a carbon brush or a metal contact, and the carbon brush on the collector drops carbon powder into the busbar rail during operation to form carbon marks, which appear as black carbon deposits in the busbar picture.

[0034] Preferably, the busbar image in step S11 is preprocessed, specifically:

[0035] S111: The collected busbar images include carbon deposition fault images and normal images, and the busbar image data set is divided into four levels according to the degree of carbon deposition fault: no carbon deposition fault, carbon deposition fault level 1, carbon deposition fault level 2, and carbon deposition fault level 3; the busbar with no carbon deposition fault has a carbon deposition coverage ratio of less than 5%; the carbon deposition coverage area of ​​the carbon deposition fault level 1 reaches 5%-30%, excluding the 30% point, and has no effect on electrical and mechanical properties; the carbon deposition coverage area of ​​the carbon deposition fault level 2 reaches 30%-60%, which significantly increases the contact resistance of the busbar; the carbon deposition coverage area of ​​the carbon deposition fault level 3 is greater than 60%, the conductive performance of the busbar decreases, and the contact resistance increases;

[0036] S112: Preprocess the collected busbar images. First, crop the busbar images. Second, grayscale the busbar images. Convert the color busbar images into grayscale busbar images. Finally, use the preprocessed busbar images for training.

[0037] Preferably, in step S22, the training set image X train Input the variational autoencoder and extract the intermediate layer feature map f output by the variational autoencoder decoder VAE (x), specifically:

[0038] S221: The training set image X train The input is fed into the variational autoencoder, which decomposes the feature vector in the hidden layer into a mean vector and a variance vector, generating a hidden variable z containing the background information of the busbar image. train Follows an approximate distribution: in, is a parameter to be determined; x is the approximate distribution image output; is the approximate distribution function;

[0039] S222: To determine the approximate distribution function The variational autoencoder uses the joint probability distribution as Input training set image X train associated with the hidden variable z; the distribution of the hidden variable z is set to a standard normal distribution; then the approximate distribution function for:

[0040]

[0041] in, is the approximate distribution function; Using a joint probability distribution for the variational autoencoder; p(X train ) is the probability distribution;

[0042] S223: Due to the difficulty in obtaining an approximate distribution function Introducing the normal distribution p ρ (z,X train ) to determine the approximate distribution function Specifically:

[0043]

[0044] Among them, p ρ (z,X train ) is a normal distribution;

[0045] The normal distribution p ρ (z,X train ) is calculated by neural network, and the input training set image X train An autoencoder mapped to a hidden variable z;

[0046] S224: In order to make the normal distribution p ρ (z,X train ) approximates the approximate distribution function The variational autoencoder uses KL divergence to evaluate the difference between the two distributions, thereby constraining the output normal distribution q of the variational autoencoder ρ (z|X train ) to determine the approximate distribution function Extract the busbar image background information as follows:

[0047]

[0048] Among them, D KL () is q p (z|x) and The KL divergence deviation between them; E() is the mathematical expectation.

[0049] Preferably, the kernel convolution and spatial bilinear interpolation are used to match the number of channels and the spatial resolution for processing in step S23, specifically:

[0050] S231: Calculate the difference between the two feature maps and determine the scaling factors in the horizontal and vertical directions;

[0051] S232: for each target feature pixel (a, b) in the target feature map, map it back to the feature coordinates (a′, b′) in the source feature map according to the scaling factor;

[0052] S233: Find the four nearest neighboring pixel points of the feature coordinates (a′, b′), calculate their integer coordinates, use the bilinear interpolation method, calculate the second target feature pixel value by weighted average, assign it to the target feature pixel (a, b) in the target feature map, and complete the matching of the feature map.

[0053] Preferably, the feature label in step S32 is generated by a softmax function, specifically:

[0054]

[0055] Among them, q i is the distribution of the output layer of the i-th network; z i is the logical value of the SoftMax function layer, is the distillation parameter of the loss function; T is the knowledge distillation temperature parameter; i is the network output layer number; j is the total number of network output layers.

[0056] Preferably, the edge device in step S4 is specifically an embedded device capable of performing neural network reasoning.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] (1) The present invention combines causal inference with knowledge distillation. Causal inference is used to enable the model to acquire the ability to extract fault causal relationships, and knowledge distillation is used to transfer the extracted causal relationship capabilities to the training model, so that the training model can accurately capture the characteristics of carbon deposition on the busbar without being disturbed by background features, thereby improving the robustness, interpretability and generalization ability of the training model.

[0059] (2) The present invention provides a knowledge distillation framework that combines a variational autoencoder with a training model, so that the feature-based knowledge distillation model can smoothly extract the fault features of the intermediate layer of the training model, and match it with the intermediate layer of the training model through a bilinear interpolation method, so that the training model can obtain not only information from the training set samples but also the fault feature learning ability of the deobfuscated training model, so that the training model can maintain high diagnostic accuracy while greatly compressing the scale.

[0060] (3) The present invention is applied to the carbon deposition failure of busbars in automobile parts production systems, so that edge devices in the automobile production system can also deploy high-precision fault diagnosis models without generating excessive computing power consumption. When the production line environment, such as lighting and background, changes, the model can also maintain a high level of fault diagnosis accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1It is a flow chart of the busbar power supply fault diagnosis method based on causal inference and knowledge distillation of the present invention;

[0062] Figure 2 The framework structure diagram of the carbon deposit fault diagnosis method for automobile production line of the present invention;

[0063] Figure 3 It is a schematic diagram of the busbar of the present invention;

[0064] Figures 4a-4d They are respectively schematic diagrams of carbon deposition fault levels of busbars according to the present invention;

[0065] Figure 5 It is a schematic diagram of the first characteristic model structure of the present invention;

[0066] Figure 6 It is a schematic diagram of the structure of the second characteristic model of the present invention;

[0067] Figures 7a-7f They are respectively the input variational autoencoder (VAE) of the present invention and the original background image of the first feature model;

[0068] Figures 8a-8f respectively generating a background image for the variational autoencoder (VAE) of the present invention;

[0069] Figures 9a-9f Heat maps of feature maps are generated for the variational autoencoder (VAE) of the present invention and the first feature model, respectively. DETAILED DESCRIPTION

[0070] The exemplary embodiments, features and aspects of the present invention will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0071] The embodiment of the present invention provides a busbar power supply fault diagnosis method based on causal inference and knowledge distillation, such as Figure 1 As shown, a first feature model based on the VGG16 deep neural network is constructed, and the busbar pictures in the automobile production line are used for training the first feature model; the variational autoencoder and the first feature model are used to de-obfuscate the busbar pictures in the automobile production line to generate an intermediate layer feature map; the model is deployed in the edge device on the automobile production line to perform carbon deposition fault diagnosis.

[0072] like Figure 2The figure shows the framework structure of the method for diagnosing carbon deposit faults in automobile production lines established in an embodiment of the present invention. First, the first feature model is trained using the original image data set, and the soft label of the first feature model is transferred to the second feature model. Then, the VAE is trained with a normal image data set to reconstruct the background. The reconstruction error between the first feature model and the decoder of the VAE is calculated to generate a feature map. The generated feature map is transferred to the second feature model and guides the training of the middle layer of the second feature model. Figure 2 It can be observed that the training input of the second feature model includes the hard labels of the original data set, the soft labels of the first feature model, and the feature map generated by the reconstruction error.

[0073] Step S1: construct a first feature model based on the VGG16 deep neural network, and use busbar images in the automobile production line for training the first feature model.

[0074] The busbar picture specifically shows the guide rails and bracket fixing system of the automobile production line. The guide rails of the automobile production line are the core components of the busbar and are responsible for conducting current. The current collector obtains current by sliding into contact with the guide rails through carbon brushes or metal contacts, and the carbon brushes on the current collector will drop carbon powder into the busbar rails during operation, forming carbon marks, which appear black in the busbar picture. The darker the area in the rails, the more serious the carbon deposition fault.

[0075] The following will further explain the fault diagnosis process of the present invention in detail in combination with the busbar monitoring image data collected from a certain automobile production system. Figure 3 The figure shows the busbar schematic diagram of the automobile production system. The specific implementation steps are as follows:

[0076] Step S11: collect busbar images in the automobile production system, and pre-process the busbar images to eliminate image noise; divide the busbar images into four fault levels according to the coverage area, carbon deposit thickness and glossiness of the busbar images; divide the busbar images into training set images X train And the test set image X test The busbar image preprocessing operations are as follows:

[0077] Step S111: The collected busbar images include carbon deposition fault images and normal images. The embodiment of the present invention collects monitoring images of busbars in a certain automobile production system, a total of 6,000 images are collected, and the busbar image data set is divided into four levels according to the degree of carbon deposition fault based on the production data: Figure 4a The fault with no carbon deposits shows that the busbar is not covered with carbon deposits, and the proportion of carbon deposits on the busbar is less than 5%. Figure 4bThe carbon deposition fault is shown in the first level. The carbon deposition phenomenon is slight, and only a small amount of carbon deposition appears in the local area of ​​the busbar. The thickness of the carbon deposition is thin, and the coverage area is small. The impact on the electrical and mechanical properties is minimal. The carbon deposition coverage area accounts for 5%-30%, excluding the 30% point. Figure 4c The carbon deposits in the secondary stage of carbon deposit fault are shown as follows: the carbon deposits expand in multiple areas, forming a relatively obvious covering layer. The carbon deposit layer is thick and there is local accumulation. The carbon deposit covering area accounts for 30%-60%, which significantly increases the contact resistance of the busbar. Figure 4d The carbon deposit coverage area of ​​the third level carbon deposit fault is wide, and the carbon deposit coverage area accounts for more than 60%. The carbon deposit layer is extremely thick, the conductivity of the busbar is greatly reduced, and the contact resistance is greatly increased.

[0078] Step S112: preprocess the collected busbar images. First, crop the busbar images to obtain images with a pixel size of 224*224. Secondly, grayscale the busbar images to convert the color busbar images into grayscale busbar images. Finally, use the preprocessed busbar images for training.

[0079] The 6000 images are cropped into 224*224 size images, and after preprocessing, they are randomly divided into training set images X train And the test set image X test .

[0080] Step S12: construct a first feature model, which is constructed based on the VGG16 deep neural network model and uses the divided training set image X train The VGG16 deep neural network model is used as the prototype. The model consists of 13 convolutional layers and 2 fully connected layers. In order to speed up network training, the last fully connected layer is replaced by a global average pooling layer. Its structure is as follows Figure 5 As shown. And use the divided training set image X train Train it.

[0081] Step S2: Use a variational autoencoder (VAE) and the first feature model in step S1 to perform a deobfuscation operation on the busbar image in the automobile production line to generate an intermediate layer feature map containing carbon deposit fault characteristics; the first feature model includes three sub-modules: the intermediate layer feature map generated by the first sub-module of the first feature model contains the edge of the carbon deposits and the boundary features between the busbar and the carbon deposits; the second sub-module of the first feature model can add the shape contour of the carbon deposits to the intermediate layer feature map; the intermediate layer feature map generated by the third sub-module of the first feature model contains the distribution characteristics of carbon deposits on the busbar.

[0082] Step S21: Take the training set image X trainInput into the trained first feature model and extract the intermediate layer feature map f output by the first feature model t (x).

[0083] Step S22: The training set image X train Input the variational autoencoder (VAE) and extract the intermediate layer feature map f output by the variational autoencoder (VAE) decoder VAE (x), specifically:

[0084] Step S221: The training set image X train The input is fed into the variational autoencoder (VAE), which decomposes the feature vector in the hidden layer into a mean vector and a variance vector, generating a hidden variable z containing the background information of the busbar image. train Follows an approximate distribution: in, is a parameter to be determined; x is the approximate distribution image output; is an approximate distribution function.

[0085] Step S222: To determine the approximate distribution function The variational autoencoder (VAE) uses a joint probability distribution of Input training set image X train associated with the hidden variable z; the distribution of the hidden variable z is set to a standard normal distribution; then the approximate distribution function for:

[0086]

[0087] in, is the approximate distribution function; Using the joint probability distribution for the variational autoencoder (VAE); p(X train ) is the probability distribution.

[0088] Step S223: Since it is difficult to obtain an approximate distribution function Introducing the normal distribution p ρ (z,X train ) to determine the approximate distribution function Specifically:

[0089]

[0090] Among them, p ρ (z,X train ) is a normal distribution.

[0091] Normal distribution p ρ (z,X train) is calculated by neural network, and the input training set image X train An autoencoder mapped to a hidden variable z.

[0092] Step S224: In order to make the normal distribution p ρ (z,X train ) approximates the approximate distribution function The variational autoencoder (VAE) uses KL divergence to evaluate the difference between the two distributions, thereby constraining the output normal distribution q of the variational autoencoder (VAE) ρ (z|X train ) to determine the approximate distribution function Extract the busbar image background information as follows:

[0093]

[0094] Among them, D KL () is q ρ (z|x) and The KL divergence deviation between them; E() is the mathematical expectation.

[0095] Step S23: Generate a feature map containing only background features using a variational autoencoder (VAE) and calculate the intermediate layer feature map f of the first feature model t (x) and the intermediate layer feature map f of the variational autoencoder (VAE) decoder VAE (x) to obtain a carbon deposition feature map containing only carbon deposition fault features; and use kernel convolution and spatial bilinear interpolation to match the number of channels and spatial resolution for processing, specifically:

[0096] Step S231: Calculate the difference between the two feature maps and determine the scaling factors in the horizontal and vertical directions.

[0097] Step S232: For each target feature pixel (a, b) in the target feature map, map it back to the feature coordinates (a′, b′) in the source feature map according to the scaling factor.

[0098] Step S233: Find the four nearest neighboring pixel points of the feature coordinates (a′, b′), calculate their integer coordinates, use the bilinear interpolation method, calculate the second target feature pixel value by weighted average, assign it to the target feature pixel (a, b) in the target feature map, and complete the feature map matching; obtain the intermediate layer feature error map as follows:

[0099] ERROR intermediate =MSE(f t (x),Ψ VAE (f VAE (x)));

[0100] Among them, ERROR intermediate is the feature error map of the middle layer; MSE() is the mean square error function; f t (x) is the middle layer feature map of the first feature model; f VAE (x) is the feature map of the middle layer of the variational autoencoder; Ψ VAE () is a bilinear interpolation function used to match the feature map dimension of the first feature model and the variational autoencoder; x is the input independent variable of the mean square error function.

[0101] Step S3: Construct a second feature model based on knowledge distillation technology, and use knowledge distillation technology to convert the intermediate layer feature map f output by the first feature model in step S2 t (x) and the training set image X in step S1 train Input the second feature model for training. The second feature model based on knowledge distillation technology consists of 4 convolutional layers, 4 pooling layers and 1 global average pooling layer, such as Figure 6 The second characteristic model structure schematic diagram of the present invention is shown; the parameter is 1554948, while the size of the first characteristic model is 14716740. The scale of the first characteristic model network is compressed by about 9.46 times.

[0102] Step S31: construct the first loss sub-function of the second feature model; convert the training set image X in step S1 train Input the first feature model and use the feature label q generated by the first feature model soft (x) to design the training process of the second feature model; the output probability distribution q of the second feature model s (x) and the feature label q of the first feature model soft The loss between (x) as the first loss sub-function for training the second feature model is:

[0103]

[0104] Among them, loss soft is the first loss sub-function between the softmax function output of the second feature model and the feature label of the first feature model; H() is the function of cross entropy; q t (x) is the probability distribution of the feature label of the first feature model; q s (x) is the output probability distribution of the second feature model.

[0105] The feature labels are generated by the softmax function, specifically:

[0106]

[0107] Among them, q i is the distribution of the output layer of the i-th network; zi is the logical value of the SoftMax function layer, is the distillation parameter of the loss function; T is the knowledge distillation temperature parameter; i is the network output layer number; j is the total number of network output layers.

[0108] Step S32: construct a second loss sub-function of the second feature model; in the process of training the second feature model, the intermediate layer feature map f of the second feature model is s (x) and S2, the error map of the intermediate layer features containing only fault features generated by the first feature model and variational autoencoder (VAE) ERROR intermediate The difference between , as the second loss sub-function for training the second feature model is:

[0109] loss feature＝ MSE(f s (x), ERROR intermediate );

[0110] Among them, loss feature A second loss sub-function for obtaining fault features for the middle layer of the second feature model; f s (x) is the intermediate layer feature map of the second feature model.

[0111] Step S33: construct the third loss sub-function of the second feature model; in the process of training the second feature model, the second feature model outputs the probability distribution q s (x) with one-hot encoded label p s The third loss sub-function between (x) is:

[0112]

[0113] Among them, loss h is the third loss sub-function between the softmax function output of the second feature model and the label of the real data set; p s (x) is the one-hot encoded label.

[0114] Step S34: Integrate the above loss sub-functions to obtain the loss function Loss of the second feature model s for:

[0115] Loss s =a*loss h +β*loss soft +γ*loss feature :

[0116] Among them, Loss s is the loss function of the second feature model; α is the first loss function loss h Weight coefficient; β is the second loss function losssoft Weight coefficient; γ is the third loss function loss feature Weight coefficient.

[0117] The loss function of the second feature model is Loss s The training parameters are: the batch size is set to 32, and the stochastic gradient descent algorithm (SGD) is used as the optimizer with a learning rate of 0.00001 and 50 iterations.

[0118] Step S4: deploy the second feature model obtained in step S3 in the edge device on the automobile production line, collect and process the busbar images on the automobile production line, use the second feature model to diagnose carbon deposition faults, and issue an alarm in time. The edge device is specifically: an embedded device capable of neural network reasoning.

[0119] like Figures 7a-7f They are respectively the input variational autoencoder (VAE) of the present invention and the original background image of the first feature model; a total of 6 groups of actual data images collected on-site are used to input the model for training; the 6 groups of images are used to represent faults of different carbon deposition levels.

[0120] like Figures 8a-8f According to Figures 7a-7f The 6 sets of data images are processed by the variational autoencoder (VAE) to generate 6 sets of background images.

[0121] The training accuracy of the second feature model trained based on the de-obfuscation knowledge distillation method is compared with the baseline model (the same structure model without this method) and the first feature model as shown in Table 1. It can be found that the accuracy of the second feature model using this method is better than the accuracy without this method.

[0122] Table 1 Comparison results of feature model output accuracy

[0123]

[0124] like Figures 9a-9f The figure shows the heat map of the feature map generated by the variational autoencoder (VAE) and the first feature model of the present invention; the intermediate layer feature map generated by the variational autoencoder (VAE) and the first feature model and transmitted to the second feature model respectively corresponds to the 6 groups of data maps in Figure 8. As can be seen from the figure, the generated feature map can capture the fault characteristics of carbon deposition on the busbar. After the intermediate layer feature map is transmitted to the second feature model, the second feature model can better learn the ability to capture the fault characteristics of carbon deposition, thereby improving the accuracy and interpretability of the second feature model, which means that the model learns the causal characteristics related to the fault mode, rather than the background characteristics.

[0125] The beneficial effects of the present invention are as follows: the present invention provides a busbar power supply fault diagnosis method based on causal inference and knowledge distillation, which uses causal inference to enable the model to obtain the ability to extract fault causal relationships, and uses knowledge distillation to transfer the extracted causal relationship capabilities to the second feature model, so that the second feature model can accurately capture the characteristics of carbon deposition on the busbar without being disturbed by background features, so that the second feature model can maintain high diagnostic accuracy while greatly compressing its scale. This method is applied to busbar carbon deposition faults in automotive parts production systems, so that edge devices in automotive production systems can also deploy high-precision fault diagnosis models without generating excessive computing power consumption. When the production line environment, such as lighting and background, changes, the model can also maintain a high fault diagnosis accuracy.

[0126] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A busbar power supply fault diagnosis method based on causal inference and knowledge distillation, characterized in that: It includes: S1: Build a first feature model based on a deep neural network, and use busbar images in the automobile production line to train the first feature model; S11: Collect busbar images from the automobile production system and pre-process the busbar images to eliminate image noise; divide the busbar images into four fault levels based on the coverage area, thickness and glossiness of carbon deposits in the busbar images; and construct a training set of images X train And the test set image X test ; S12: Construct a first feature model. The first feature model is constructed based on a deep neural network model and uses the divided training set image X train Train them; S2: De-obfuscate the busbar image using a variational autoencoder and the first feature model in step S1 to generate an intermediate layer feature map; the first feature model includes three submodules: the intermediate layer feature map generated by the first submodule of the first feature model includes the edge of the carbon deposits and the boundary features between the busbar and the carbon deposits; The second submodule of the first feature model can add a middle layer feature map including a shape contour of carbon deposits; The intermediate layer characteristic map generated by the third submodule of the first characteristic model includes the distribution characteristics of carbon deposits on the busbar; S21: The training set image X train Input into the trained first feature model and extract the intermediate layer feature map f output by the first feature model t (x); S22: The training set image X train Input the variational autoencoder and extract the intermediate layer feature map f output by the variational autoencoder decoder VAE (x); S23: Generate a feature map containing only background features using a variational autoencoder, specifically including: linear shape features of the busbar background, light intensity features of the working environment, and color features of the busbar surface; calculate the intermediate layer feature map f of the first feature model in step S21 t (x) and the intermediate layer feature map f of the variational autoencoder decoder in step S22 VAE (x) to obtain the carbon deposition feature map containing only the carbon deposition fault features; and then use kernel convolution and spatial bilinear interpolation to match the number of channels and spatial resolution for processing, and obtain the intermediate layer feature error map as follows: ERROR intermediate =MSE(f t (x),τ VAE (f VAE (x))) Among them, ERROR intermediate is the feature error map of the middle layer; MSE() is the mean square error function; f t (x) is the middle layer feature map of the first feature model; f VAE (x) is the feature map of the middle layer of the variational autoencoder; Ψ VAE () is a bilinear interpolation function used to match the feature map dimension of the first feature model and the variational autoencoder; x is the input independent variable of the mean square error function; S3: Construct a second feature model based on knowledge distillation technology, and use knowledge distillation technology to convert the intermediate layer feature map f output by the first feature model in step S2 t (x) and the training set image X in step S1 train Input the second feature model for training; S31: Construct the first loss sub-function of the second feature model; convert the training set image X in step S1 train Input the first feature model and use the feature label q generated by the first feature model soft (x) to design the training process of the second feature model; the output probability distribution q of the second feature model s (x) and the feature label q of the first feature model soft The loss between (x) as the first loss sub-function for training the second feature model is: Among them, loss soft is the first loss sub-function between the softmax function output of the second feature model and the feature label of the first feature model; H() is the function of cross entropy; q t (x) is the probability distribution of the feature label of the first feature model; q s (x) is the output probability distribution of the second feature model; S32: construct a second loss sub-function of the second feature model; in the process of training the second feature model, the intermediate layer feature map f of the second feature model is s (x) and S2 are generated by the first feature model and variational autoencoder, and the intermediate layer feature error map only contains fault features ERROR intermediate Find the difference, as the second loss sub-function for training the second feature model: loss feature =MSE(f s (x),ERROR intermediate ) Among them, loss feature A second loss sub-function for obtaining fault features for the middle layer of the second feature model; f s (x) is the middle layer feature map of the second feature model; S33: construct a third loss sub-function of the second feature model; in the process of training the second feature model, the second feature model outputs a probability distribution q s (x) with one-hot encoded label p s The third loss sub-function between (x) is: Among them, loss h is the third loss sub-function between the softmax function output of the second feature model and the label of the real data set; p s (x) is a one-hot encoded label; S34: Integrate the above loss sub-functions to obtain the loss function Loss of the second feature model s for: Los s =α*loss h +β*loss soft +γ*loss feature Among them, Loss s is the loss function of the second feature model; α is the first loss function loss h Weight coefficient; β is the second loss function loss soft Weight coefficient; γ is the third loss function loss feature Weight coefficient; S4: Deploy the second feature model obtained in step S3 in the edge device on the automobile production line, collect and process the busbar images on the automobile production line, use the second feature model to diagnose carbon deposit faults, and issue an alarm in a timely manner.

2. The busbar power supply fault diagnosis method based on causal inference and knowledge distillation according to claim 1 is characterized in that: The busbar picture in step S1 is specifically a picture of the guide rail and bracket fixing system of the automobile production line; the guide rail of the automobile production line is the core component of the busbar and is responsible for conducting current; the collector obtains current by sliding in contact with the guide rail through a carbon brush or a metal contact, and the carbon brush on the collector drops carbon powder into the busbar rail during operation to form carbon marks, which are black carbon deposits in the busbar picture.

3. The busbar power supply fault diagnosis method based on causal inference and knowledge distillation according to claim 1 is characterized in that: The busbar image in step S11 is preprocessed, specifically: S111: The collected busbar images include carbon deposition fault images and normal images, and the busbar image data set is divided into four levels according to the degree of carbon deposition fault: no carbon deposition fault, carbon deposition fault level 1, carbon deposition fault level 2, and carbon deposition fault level 3; the busbar with no carbon deposition fault has a carbon deposition coverage ratio of less than 5%; The first-level carbon deposition fault has a coverage area of ​​5%-30%, excluding the 30% point, and has no effect on electrical and mechanical properties; the second-level carbon deposition fault has a coverage area of ​​30%-60%, which significantly increases the contact resistance of the busbar; The third-level carbon deposit coverage area of ​​the carbon deposit fault is greater than 60%, the conductivity of the busbar decreases, and the contact resistance increases; S112: Preprocess the collected busbar images. First, crop the busbar images. Second, grayscale the busbar images. Convert the color busbar images into grayscale busbar images. Finally, use the preprocessed busbar images for training.

4. The busbar power supply fault diagnosis method based on causal inference and knowledge distillation according to claim 1 is characterized in that: In step S22, the training set image X train Input the variational autoencoder and extract the intermediate layer feature map f output by the variational autoencoder decoder VAE (x), specifically: S221: The training set image X train The input is fed into the variational autoencoder, which decomposes the feature vector in the hidden layer into a mean vector and a variance vector, generating a hidden variable z containing the background information of the busbar image. train Follows an approximate distribution: in, is a parameter to be determined; x is the approximate distribution image output; is the approximate distribution function; S222: To determine the approximate distribution function The variational autoencoder uses the joint probability distribution as Input training set image X train associated with the hidden variable z; the distribution of the hidden variable z is set to a standard normal distribution; then the approximate distribution function for: in, is the approximate distribution function; Using a joint probability distribution for the variational autoencoder; p(X train ) is the probability distribution; S223: Due to the difficulty in obtaining an approximate distribution function Introducing the normal distribution p ρ (z,X train ) to determine the approximate distribution function Specifically: Among them, p ρ (z,X train ) is a normal distribution; The normal distribution p ρ (z,X train ) is calculated by neural network, and the input training set image X train An autoencoder mapped to a hidden variable z; S224: In order to make the normal distribution p ρ (z,X train ) approximates the approximate distribution function The variational autoencoder uses KL divergence to evaluate the difference between the two distributions, thereby constraining the output normal distribution q of the variational autoencoder ρ (z|X train ) to determine the approximate distribution function Extract the busbar image background information as follows: Among them, D KL ( ) is q ρ (z|x) and The KL divergence deviation between them; E() is the mathematical expectation.

5. The busbar power supply fault diagnosis method based on causal inference and knowledge distillation according to claim 1 is characterized in that: In step S23, kernel convolution and spatial bilinear interpolation are used to match the number of channels and spatial resolution for processing, specifically: S231: Calculate the difference between the two feature maps and determine the scaling factors in the horizontal and vertical directions; S232: For each target feature pixel (a, b) in the target feature map, map it back to the feature coordinates (a′, b′) in the source feature map according to the scaling factor; S233: Find the four nearest neighboring pixel points of the feature coordinates (a′, b′), calculate their integer coordinates, use the bilinear interpolation method, calculate the second target feature pixel value by weighted average, assign it to the target feature pixel (a, b) in the target feature map, and complete the matching of the feature map.

6. The busbar power supply fault diagnosis method based on causal inference and knowledge distillation according to claim 1 is characterized in that: The feature labels in step S32 are generated by the softmax function, specifically: Among them, q i is the distribution of the output layer of the i-th network; z i is the logical value of the SoftMax function layer, is the distillation parameter of the loss function; T is the knowledge distillation temperature parameter; i is the network output layer number; j is the total number of network output layers.

7. The busbar power supply fault diagnosis method based on causal inference and knowledge distillation according to claim 1 is characterized in that: The edge device in step S4 is specifically an embedded device capable of performing neural network reasoning.

Images