Lightweight fault diagnosis model compression method based on channel pruning and quantization fusion

Abstract

The invention discloses a lightweight fault diagnosis model compression method based on channel pruning and quantization fusion, relates to the technical field of data processing, and comprises the following steps: training and preparing a fault diagnosis model; l1 regularization is introduced in the model training process, and the weight of a convolutional layer is constrained; after the training is finished, screening out a preset percentage convolution kernel with the lowest contribution degree according to the L1 norm of the convolution kernel, and obtaining a pruned fault diagnosis model; a 8-bit fixed-point quantification method is adopted for the pruned fault diagnosis model, floating-point parameters in the fault diagnosis model are converted into 8-bit fixed-point numbers to be expressed, and the pruned and quantized fault diagnosis model is obtained; and carrying out fine tuning on the fault diagnosis model after pruning and quantification, and carrying out iterative training on the fault diagnosis model by using the training set. According to the method, the fault diagnosis model is pruned by introducing L1 regularization, a low-contribution convolution kernel can be removed, the model is simplified, and in addition, the accuracy of the model can be ensured in the re-simplification process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a lightweight fault diagnosis model compression method based on channel pruning and quantization fusion. BACKGROUND

[0002] From the perspective of industrial production, with the transformation and upgrading of manufacturing industry to intelligence and automation, the complexity and precision of industrial equipment are continuously improving. A large number of advanced equipment are put into use, which greatly improves the production efficiency, but once the equipment fails, it may cause the entire production line to stop and cause huge economic losses. Traditional fault diagnosis methods cannot meet the demand for real-time and accurate diagnosis of a large number of equipment, and modern diagnosis methods relying on complex deep learning models face the dilemma of large model size and high computing resource demand. For example, in an automobile manufacturing plant, the automatic production line involves a large number of mechanical arms, sensors and other equipment, and the initial size of the fault diagnosis model can reach hundreds of MB, which runs slowly and is costly on edge devices. SUMMARY

[0003] The purpose of the present application is to provide a lightweight fault diagnosis model compression method based on channel pruning and quantization fusion to solve the problems in the background art.

[0004] In order to achieve the above purpose, the present application provides the following technical scheme: a lightweight fault diagnosis model compression method based on channel pruning and quantization fusion, comprising the following steps:

[0005] Training and preparing the fault diagnosis model;

[0006] Introducing L1 regularization in the model training process to constrain the weights of the convolutional layer; after the training is completed, according to the L1 norm size of the convolution kernel, the lowest contribution percentage of the convolution kernel is selected to obtain the pruned fault diagnosis model;

[0007] An 8-bit fixed-point quantization method is used for the pruned fault diagnosis model to convert the floating-point parameters in the fault diagnosis model into 8-bit fixed-point numbers, and a pruned and quantized fault diagnosis model is obtained;

[0008] Fine-tuning the pruned and quantized fault diagnosis model, using the training set to iteratively train the fault diagnosis model until it adapts to the compressed structure and parameter representation, and obtaining an optimized fault diagnosis model;

[0009] Testing the optimized fault diagnosis model using the test set to verify the accuracy of the optimized fault diagnosis model for fault diagnosis; checking whether the fault diagnosis model size is compressed to 15MB, if the requirement is met, the compression of the fault diagnosis model is completed.

[0010] In a preferred embodiment, the step of training and preparing the fault diagnosis model comprises:

[0011] determining a fault diagnosis scene, collecting corresponding training data for the fault diagnosis scene, wherein the training data comprises fault data corresponding to a fault type;

[0012] preprocessing the training data, labeling the training data according to the fault degree of the fault type, and respectively labeling fault labels and normal labels;

[0013] dividing the labeled training data to obtain a training set, a validation set and a test set;

[0014] training the convolutional neural network through the training set to obtain the fault diagnosis model.

[0015] In a preferred embodiment, the step of introducing L1 regularization in the model training process to constrain the weights of the convolutional layer, and after the training is completed, according to the L1 norm size of the convolution kernel, filtering out a preset percentage of convolution kernels with the lowest contribution degree, comprises:

[0016] setting training parameters, wherein the training parameters include the number of training rounds, an optimizer and its learning rate;

[0017] In the training loop, the training set is input into the fault diagnosis model to obtain the output, a loss function containing an L1 regularization term is calculated, and back propagation and parameter updating are performed through the optimizer;

[0018] Every preset number of training rounds, the performance of the fault diagnosis model is evaluated using the validation set, and the loss value and accuracy are recorded;

[0019] After the training is completed, all convolutional layers of the fault diagnosis model performance are traversed, and the L1 norm of the convolution kernel weight of each convolutional layer is calculated;

[0020] A preset percentage of convolution kernels are filtered out to obtain a pruned fault diagnosis model.

[0021] In a preferred embodiment, the step S24 of traversing all convolutional layers of the fault diagnosis model performance after the training is completed to calculate the L1 norm of the convolution kernel weight of each convolutional layer comprises:

[0022] After the training is completed, the L1 norm of all convolutional kernels of the fault diagnosis model performance is calculated, and the calculation formula of the L1 norm of a single convolutional kernel is:

[0023] wherein L1 per is the L1 norm of the coutth convolutional kernel, Q cout,cin,h,wrepresents the convolution kernel weight value at the position of the coutth output channel, the cint input channel, height h, and width w;

[0024] A preset percentage is determined, the calculated plurality of convolution kernel L1 norms are sorted, and an L1 norm threshold corresponding to the position of the preset percentage is found.

[0025] In a preferred embodiment, the pruned fault diagnosis model is quantized using an 8-bit fixed-point quantization method to convert floating-point parameters in the fault diagnosis model to 8-bit fixed-point numbers, obtaining a pruned and quantized fault diagnosis model, including:

[0026] Load the pruned fault diagnosis model parameters and structure based on PyTorch;

[0027] Select a preset portion of data from the training set as a calibration data set based on static quantization;

[0028] Load the calibration data set into memory using a data loader.

[0029] In a preferred embodiment, the pruned and quantized fault diagnosis model is fine-tuned, and the training set is used to iteratively train the fault diagnosis model until it adapts to the compressed structure and parameter representation, obtaining an optimized fault diagnosis model, including:

[0030] Divide the training data into batches, input the training data into the pruned and quantized fault diagnosis model by batch, obtain output data, and obtain evaluation indicators based on a loss function;

[0031] Input the training data into the fault diagnosis model without pruning and quantization by batch, obtain original output data, and obtain evaluation indicators based on a loss function;

[0032] Compare the evaluation indicators of the output data with the evaluation indicators of the original output data to obtain the difference between the pruned and quantized fault diagnosis model and the fault diagnosis model without pruning and quantization, and the pruned and quantized fault diagnosis model corresponding to the difference that meets the preset condition is used as the optimized fault diagnosis model.

[0033] In a preferred embodiment, the optimized fault diagnosis model is tested using a test set to verify the accuracy of the optimized fault diagnosis model for fault diagnosis, and to check whether the fault diagnosis model is compressed to 15MB. If the requirement is met, the compression of the fault diagnosis model is completed, including:

[0034] The data packet is set, a plurality of data intervals are divided according to the data packet, and the data volume corresponding to the data interval is marked, a plurality of data points are set in the data packet, one data point corresponds to connecting one data interval, the plurality of data points are connected with each other, and the data points are moved in the data packet;

[0035] The pruned fault diagnosis model is loaded in the data packet, the pruned fault diagnosis model is attached through the data points, and a data point set is obtained;

[0036] The volume of the fault diagnosis model is monitored through the data point set whether it is compressed to 15MB, and if the requirement is met, the compression of the fault diagnosis model is completed.

[0037] In a preferred embodiment, the step of monitoring the volume of the fault diagnosis model through the data point set whether it is compressed to 15MB, and if the requirement is met, the compression of the fault diagnosis model is completed, comprising:

[0038] The loading position and loading range of the fault diagnosis model in the data packet are determined, and the data points corresponding to the data interval occupied by the fault diagnosis model are marked with the data occupation amount of the fault diagnosis model;

[0039] The data points are associated with the fault diagnosis model, and the plurality of data points associated with the fault diagnosis model are connected;

[0040] The data occupation amount is interacted and integrated through the connected data points to obtain the data volume occupied by the fault diagnosis model, and if the volume of the fault diagnosis model is compressed to meet 15MB, the compression of the fault diagnosis model is completed.

[0041] In the above technical solution, the technical effects and advantages provided by the present application are as follows:

[0042] 1. The present application prunes the fault diagnosis model by introducing L1 regularization, which can remove low contribution convolution kernels, simplify the model, and further ensure the accuracy of the model during the simplification process.

[0043] 2. The 8bit fixed-point quantization method is used for calibration in the quantization process, so as to ensure that the quantization error is controlled within 0.5%, and to ensure that the model still maintains high diagnostic accuracy after compression.

[0044] 3. The present application can dynamically and accurately monitor the amount of the last fault diagnosis model compression through the data packet, ensure the compression requirement of the fault diagnosis model, and has good model performance response ability. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed in the embodiments will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art based on these drawings belong to the protection scope of the present application.

[0046] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0047] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor belong to the protection scope of the present application.

[0048] Embodiment 1, please refer to Figure 1 The light-weight fault diagnosis model compression method based on channel pruning and quantization fusion described in the present embodiment includes the following steps:

[0049] S1, training and preparation of the fault diagnosis model;

[0050] S2, introducing L1 regularization in the model training process to constrain the weights of the convolutional layer; after the training is completed, according to the L1 norm size of the convolution kernel, the lowest contribution percentage of the convolution kernel is selected (by removing these low-contribution convolution kernels, the model structure is preliminarily simplified, and the calculation amount and parameter amount of the model are reduced), and a pruned fault diagnosis model is obtained;

[0051] S3, using an 8-bit fixed-point quantization method for the pruned fault diagnosis model, converting the floating-point parameters in the fault diagnosis model into 8-bit fixed-point numbers, and obtaining a pruned and quantized fault diagnosis model (calibration is performed during quantization to ensure that the quantization error is controlled within 0.5%, so as to ensure that the model can still maintain high diagnostic accuracy after compression);

[0052] S4, fine-tuning the pruned and quantized fault diagnosis model, using the training set to iteratively train the fault diagnosis model until the structure and parameter representation after compression are adapted, and obtaining an optimized fault diagnosis model (the model performance is evaluated using the validation set, and if the performance is not up to standard, the pruning ratio or quantization strategy can be adjusted appropriately);

[0053] S5, test the optimized fault diagnosis model using the test set to verify the accuracy of the optimized fault diagnosis model for fault diagnosis; check whether the size of the fault diagnosis model is compressed to 15 MB, and if the requirement is met, the compression of the fault diagnosis model is completed;

[0054] As described in steps S1-S5 above, from the perspective of industrial production, with the transformation and upgrading of manufacturing industry to intelligence and automation, the complexity and precision of industrial equipment are continuously improving. A large number of advanced equipment are put into use, which greatly improves the production efficiency, but once the equipment fails, it may cause the entire production line to stop and cause huge economic losses. According to relevant data, some large manufacturing enterprises may suffer an average annual loss of tens of millions of yuan or even more due to equipment failure. Traditional fault diagnosis methods cannot meet the demand for real-time and accurate diagnosis of a large number of equipment, and modern diagnosis methods relying on complex deep learning models face the dilemma of large model size and high demand for computing resources. For example, in an automobile manufacturing plant, the initial size of the fault diagnosis model of the automatic production line involving a large number of mechanical arms, sensors and other equipment can reach hundreds of MB, which runs slowly and costs high on edge devices. The light-weight fault diagnosis model compression method can reduce the resource consumption of the model, so that the fault diagnosis system can run efficiently in resource-constrained industrial field devices, timely detect and warn equipment failure, ensure production continuity, and reduce economic losses. In the present application, L1 regularization is introduced to prune the fault diagnosis model, which can remove low-contribution convolution kernels and simplify the model. In addition, the accuracy of the model can be guaranteed during the simplification process. The 8-bit fixed-point quantization method is used for calibration during the quantization process to ensure that the quantization error is controlled within 0.5% to ensure that the model can maintain high diagnostic accuracy after compression. Finally, the data packet can dynamically and accurately monitor the quantity of the compressed fault diagnosis model, ensuring the compression requirements of the fault diagnosis model and having good model performance response ability.

[0055] In one embodiment, the step S1 of training and preparing the fault diagnosis model comprises:

[0056] S11, determining a fault diagnosis scene, and collecting corresponding training data for the fault diagnosis scene, wherein the training data includes fault data corresponding to a fault type;

[0057] S12, preprocessing the training data, and labeling the training data according to the fault degree of the fault type, respectively as a fault label and a normal label;

[0058] S13, dividing the labeled training data to obtain a training set, a validation set and a test set;

[0059] S14, training a convolutional neural network through the training set to obtain a fault diagnosis model;

[0060] As described in steps S11-S14 above, according to the specific scene of the fault diagnosis task (such as industrial equipment fault diagnosis, electronic component fault diagnosis, etc.), the required data types are determined, including sensor data, operation logs, historical fault records, etc., and key indicators such as the time span of the data and the sampling frequency are determined. Device collection: by connecting industrial equipment, instruments and meters, etc., real-time collection of operation data is performed using sensors, such as vibration, temperature, current, voltage, etc. Database retrieval: historical fault data and corresponding normal operation data are retrieved from enterprise historical databases and device management systems. Public data set: search for publicly available fault diagnosis data sets in related fields, such as the IEEE PHM data set, the CWRU bearing fault data set, etc., and download the data that meets the requirements. Data cleaning: check data integrity, remove missing values and outliers, and fill in missing data using interpolation methods (such as linear interpolation, polynomial interpolation) or model-based (such as random forest regression) methods. Data labeling: accurately label the data according to fault type, fault degree, etc., to ensure that each data sample corresponds to a clear fault label or normal state label. Data normalization: use normalization methods (such as Min-Max normalization, Z-Score normalization) to standardize the data, and unify the data to the same scale range, to facilitate subsequent model training. Divide the training data set into training set and test set, and divide the labeled data set according to the determined proportion. During the division process, pay attention to keep the label distribution of each subset balanced, and avoid data bias affecting model training and evaluation due to division. According to the characteristics and data types of the fault diagnosis task, select an appropriate original fault diagnosis model, such as a convolutional neural network (CNN).

[0061] In one embodiment, the step S2 of introducing L1 regularization in the model training process to constrain the weights of the convolutional layer, and after the training is completed, according to the L1 norm size of the convolution kernel, the lowest contribution percentage of the convolution kernel is selected.

[0062] S21, set the training parameters, wherein the training parameters include the number of training rounds, the optimizer and its learning rate;

[0063] S22, in the training loop, the training set is input into the fault diagnosis model to obtain the output, the loss function containing the L1 regularization term is calculated, and the back propagation and parameter update are performed through the optimizer;

[0064] S23, every preset number of training rounds, the validation set is used to evaluate the performance of the fault diagnosis model, and the loss value and accuracy are recorded;

[0065] S24, after the training is completed, all the convolutional layers of the fault diagnosis model performance are traversed, and the L1 norm of the convolution kernel weight of each convolutional layer is calculated.

[0066] S25, a preset percentage of convolution kernels is set, the preset percentage of convolution kernels with the lowest contribution degree is filtered out, and a pruned fault diagnosis model is obtained;

[0067] In one embodiment, after the training is completed, the L1 norm of the convolution kernel weight of each convolution layer of the fault diagnosis model performance is calculated in step S24, including:

[0068] S241, after the training is completed, the L1 norm of all convolution kernels of the fault diagnosis model performance is calculated, wherein the calculation formula of the L1 norm of a single convolution kernel is:

[0069] Wherein, L1 per is the L1 norm of the coutth convolution kernel, Q cout,cin,h,w represents the convolution kernel weight value of the coutth output channel, the cinth input channel, the height h and the width w position, in is the upper limit value of the convolution kernel weight of the cinth input channel; ker is the upper limit value of the height of the convolution kernel weight, and wi is the upper limit value of the width of the convolution kernel weight;

[0070] S242, a preset percentage (such as 30%) is determined, the calculated multiple convolution kernel L1 norms are sorted, and the L1 norm threshold value corresponding to the preset percentage position is found.

[0071] As described in steps S21-S25 above, the deep learning framework is confirmed, and the previously divided training set and validation set data are loaded into the memory or data loader through the data loading code. Taking PyTorch as an example, the DataLoader class is used to encapsulate the dataset into an iterable object, which facilitates reading data in batches during training, and the appropriate batch_size (such as 32, 64, etc.) is set. Based on the requirements of the fault diagnosis task, a fault diagnosis model containing convolutional layers is built using the deep learning framework. For example, in PyTorch, the model structure is defined by inheriting the nn.Module class, and the input channel number, output channel number, convolution kernel size, step size, etc. of each convolutional layer are explicitly specified; in the loss function calculation part of the model training, an L1 regularization term is added. Taking PyTorch as an example, assuming that the original loss function is criterion, define the L1 regularization strength hyperparameter, traverse all the convolutional layers of the model, obtain the weight parameters of the convolutional layers, add the L1 regularization term to the original loss function, and determine the number of training epochs (epochs), the optimizer (such as Adam, SGD), and the learning rate. In the training loop, input data and labels are obtained from the data loader in batches, the data is passed into the model to obtain the output, the loss function including the L1 regularization term is calculated, and the backpropagation and parameter update are performed through the optimizer. After training for a certain number of rounds (such as every 10 rounds), the model performance is evaluated using the validation set, and the loss value and accuracy, etc. are recorded to monitor the model training state. If overfitting or underfitting occurs, the hyperparameters are adjusted in time. After training is completed, the L1 norm of each convolutional kernel weight is calculated for each convolutional layer. Taking PyTorch as an example, the L1 norm of each convolutional kernel is calculated, and the calculation results are saved to a list.

[0072] According to the preset percentage (such as 30%), the calculated convolutional kernel L1 norm is sorted, and here the L1 norm threshold corresponding to the percentage position is found according to the training participation of the convolutional kernel to the model. Traverse all the convolutional layers, remove the convolutional kernels whose L1 norm is lower than the threshold and their corresponding input and output channel connections from the model, and update the model structure. For example, in PyTorch, the removal of convolutional kernels is realized by modifying the parameters of the nn.Conv2d layer, and the model computation graph is reconstructed. The filtered model is saved for subsequent quantization or other operations, and key information such as the filtering ratio and the number of removed convolutional kernels is recorded during the pruning process.

[0073] In one embodiment, the pruned fault diagnosis model adopts an 8-bit fixed-point quantization method to convert the floating-point parameters in the fault diagnosis model to 8-bit fixed-point numbers, obtaining a pruned and quantized fault diagnosis model in step S3, comprising:

[0074] S31, load the pruned fault diagnosis model parameters and structure based on PyTorch;

[0075] S32, select a preset portion of data from the training set as a calibration data set based on static quantization (the calibration data set should cover various fault types and normal operating states, and the data distribution should be consistent with the data distribution in the actual application scenario);

[0076] S33, load the calibration data set into memory using a data loader (for subsequent use in calculating quantization parameters);

[0077] As described in steps S31-S33 above, the torch.quantization.convert function is called to convert the fault diagnosis model to 8-bit fixed-point representation (this function calculates the quantization parameters based on the statistical information collected by the observer module, and converts the floating-point parameters in the model to 8-bit fixed-point numbers).

[0078] Select an appropriate quantization tool library according to the deep learning framework used. If GPU acceleration is required for the quantization process, ensure that CUDA and cuDNN are correctly installed and version compatible. At the same time, check the system's memory resources, as the quantization operation may consume a lot of memory, ensuring that the system has enough memory space to handle the conversion of model parameters.

[0079] Use the model loading function provided by the deep learning framework to import the previously saved pruned fault diagnosis model into the current environment. For example, use the torch.load() function to load the model parameters and structure in PyTorch, ensuring that the model can be normally instantiated and run. According to the characteristics of the model and the requirements of the fault diagnosis task, select an appropriate quantization strategy. Common quantization strategies include dynamic quantization and static quantization: static quantization is used here. Static quantization: requires the use of calibration data to pre-calculate quantization parameters, which can generally achieve higher accuracy. For fault diagnosis models, to ensure diagnosis accuracy, static quantization is preferred. In static quantization, specify the module type to be quantized (such as convolutional layers, fully connected layers, etc.) and the data type to be quantized (8-bit fixed-point numbers).

[0080] If static quantization is used, a portion of the original data set needs to be selected as the calibration data set. The calibration data set should cover various fault types and normal operating states, and the data distribution should be consistent with the data distribution in the actual application scenario. Load the calibration data set into memory using the data loader for subsequent calculation of quantization parameters. In PyTorch, for static quantization, the torch.quantization.prepare(model, inplace=True) function needs to be called to prepare the model. This function will insert some necessary observer modules into the model to collect statistical information for calculating quantization parameters. The calibration data set is input into the prepared model for forward propagation, and the observer modules in the model will record statistical information such as the dynamic range of the data. In PyTorch, the calibration data can be batched into the model through a simple loop: After calibration is complete, the torch.quantization.convert(model, inplace=True) function is called to convert the model to 8-bit fixed-point representation. This function will calculate the quantization parameters based on the statistical information collected by the observer modules, and convert the floating-point parameters in the model to 8-bit fixed-point numbers. The performance of the quantized model is evaluated using the test set, and the accuracy, recall, F1 value, and other fault diagnosis-related indicators of the model are calculated. The performance of the quantized model is compared with the original uncompressed model and the pruned unquantized model to ensure that the quantization error is controlled within 0.5%. If the performance decreases too much, try adjusting the quantization strategy, calibration data set, or re-pruning operation. After confirming that the performance of the quantized model meets the requirements, use the model saving function of the deep learning framework to save the model. Save the fault diagnosis model after quantization and pruning to the specified path for subsequent use in actual fault diagnosis tasks.

[0081] In one embodiment, the step S4 of fine-tuning the fault diagnosis model after pruning and quantization, using the training set to iteratively train the fault diagnosis model until it adapts to the compressed structure and parameter representation, obtaining an optimized fault diagnosis model, includes:

[0082] S41, divide the training data into batches, input the training data into the fault diagnosis model after pruning and quantization by batch, obtain output data, and obtain evaluation indicators based on the loss function;

[0083] S42, input the training data into the fault diagnosis model without pruning and quantization by batch, obtain the original output data, and obtain evaluation indicators based on the loss function;

[0084] S43, compare the evaluation indicators of the output data with the evaluation indicators of the original output data, obtain the difference between the pruned and quantized fault diagnosis model and the fault diagnosis model without pruning and quantization, and take the pruned and quantized fault diagnosis model corresponding to the difference meeting the preset condition as the optimized fault diagnosis model.

[0085] As described in steps S41-S43 above, ensure that the version of the deep learning framework (PyTorch or TensorFlow) and related dependent libraries is consistent during quantization and pruning, and avoid model loading or training abnormalities due to version differences. For example, if PyTorch 1.12 version is used to complete quantization and pruning, PyTorch should also be ensured to be this version during fine-tuning. Check whether the data processing library (such as numpy, pandas) and the evaluation indicator calculation library (such as scikit-learn) are installed and the version is adapted. Load data from the divided training set, if the data has format or preprocessing method adjustment after quantization and pruning, ensure that the data processing method during fine-tuning is the same. Use the data loader (such as PyTorch DataLoader) to encapsulate the training data into batches, set appropriate batch_size (such as 32, 64), so as to input the model in batches during training. Use the model loading function of the deep learning framework to load the previously saved pruned and quantized fault diagnosis model into the current environment. Take PyTorch as an example, load the model parameters through torch.load('quantized_pruned_model.pth'), and set the model to training mode model.train(). According to the characteristics of the model and the training requirements, select appropriate optimizers such as Adam, SGD, etc. Take the Adam optimizer as an example, initialize the optimizer in PyTorch using optimizer=torch.optim.Adam(model.parameters(), lr=0.001), where lr is the learning rate, which can be adjusted according to the actual training situation.

[0086] Define the loss function: Select the corresponding loss function according to the type of fault diagnosis task (such as classification task or regression task). If it is a classification task, you can use the cross-entropy loss function criterion = torch.nn.CrossEntropyLoss(); if it is a regression task, you can use the mean square error loss function criterion = torch.nn.MSELoss(). Determine the number of training rounds (epochs), which can generally be set to 10-30 rounds, and adjust according to the model convergence. At the same time, you can set the learning rate decay strategy, such as multiplying the learning rate by 0.1 every 5 rounds, to adjust the model parameters more finely in the later training period, and use the learning rate scheduler (torch.optim.lr_scheduler.StepLR in PyTorch) to implement this function. In the training loop, get the input data and labels from the data loader in batches, pass the data into the model to get the output, and calculate the loss function value. During training, evaluate the model using the validation set every certain number of rounds (such as every 1 round), calculate the accuracy, recall, F1 value, and other indicators, and observe the model performance trend. If the model performance on the validation set appears to be overfitting (such as the validation loss starts to rise), you can stop training early or adjust the regularization parameters. After training, use the test set to fully evaluate the optimized fault diagnosis model, calculate various evaluation indicators, and compare the performance of the original uncompressed model, the pruned and quantized model without fine-tuning, and the performance of the fine-tuned model, to verify whether the fine-tuning operation effectively improves the performance of the model in the compressed state.

[0087] In one embodiment, the use of the test set to test the optimized fault diagnosis model verifies the accuracy of the optimized fault diagnosis model for fault diagnosis; checks whether the fault diagnosis model volume is compressed to 15MB, and if the requirement is met, the step S5 of compressing the fault diagnosis model is completed, including:

[0088] S51, set the data packet, divide a plurality of data intervals corresponding to the data packet and mark the data amount corresponding to the data interval, set a plurality of data points in the data packet, one data point corresponds to connecting one data interval, a plurality of data points are connected to each other, and the data points are moved in the data packet;

[0089] S52, load the pruned fault diagnosis model in the data packet, attach the pruned fault diagnosis model through the data points, and obtain a data point set;

[0090] S53, monitor whether the fault diagnosis model volume is compressed to 15MB through the data point set, and if the requirement is met, the compression of the fault diagnosis model is completed.

[0091] In one embodiment, the step S53 of completing the compression of the fault diagnosis model includes:

[0092] S531, determining the load position and load range of the fault diagnosis model in the data packet, and marking the data points corresponding to the data interval occupied by the fault diagnosis model with the data occupancy of the fault diagnosis model;

[0093] S532, associating the data points with the fault diagnosis model, and connecting the multiple data points associated with the fault diagnosis model;

[0094] S533, interacting and integrating the data occupancy of the connected data points to obtain the data amount occupied by the fault diagnosis model, and if the volume of the fault diagnosis model is compressed to meet 15MB, the compression of the fault diagnosis model is completed.

[0095] As described in the steps S51-S53 above, the data packet is a data storage space (storage), and multiple data intervals are divided in the data storage space, one data interval corresponds to one data point, the data point is a virtual machine, and the multiple data points can communicate with each other. Whether the communication can be confirmed depends on the position of the data interval occupied by the fault diagnosis model in the data packet. After the fault diagnosis model is compressed, it can be stored in the data packet. The data amount of the fault diagnosis model is monitored in real time and dynamically by the data points. The fault diagnosis model always occupies a mobile space range in the data packet. In order to reflect the accuracy and dynamics of the monitoring, one data interval is managed by one data point. When the fault diagnosis model occupies all or part of the data interval, the corresponding data point can be associated with the fault diagnosis model. The data points associated with the fault diagnosis model can be used as the condition for communication and interaction. In this way, the data amount occupied by the fault diagnosis model is obtained by interacting and integrating the data occupancy of the connected data points. If the volume of the fault diagnosis model is compressed to meet 15MB, the compression of the fault diagnosis model is completed. The compression amount of the fault diagnosis model can be dynamically and accurately monitored, and the normal use of the fault diagnosis model is not affected.

[0096] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A lightweight fault diagnosis model compression method based on channel pruning and quantization fusion, characterized by: The following steps are involved: Train and prepare the fault diagnosis model; L1 regularization is introduced during model training to constrain the weights of the convolutional layer; After the training is completed, the preset percentage of convolution kernels with the lowest contribution is screened out according to the L1 norm of the convolution kernel to obtain the pruned fault diagnosis model; The pruned fault diagnosis model is quantized using an 8-bit fixed-point method to convert floating-point parameters in the fault diagnosis model into 8-bit fixed-point representations, thus obtaining the pruned and quantized fault diagnosis model. Fine-tune the pruned and quantized fault diagnosis model, and iteratively train the fault diagnosis model using the training set until it adapts to the compressed structure and parameter representation, thereby obtaining an optimized fault diagnosis model. Use the test set to test the optimized fault diagnosis model to verify the accuracy of the optimized fault diagnosis model in fault diagnosis; check whether the size of the fault diagnosis model is compressed to 15MB. If it meets the requirements, complete the compression of the fault diagnosis model.

2. The lightweight fault diagnosis model compression method based on channel pruning and quantization fusion according to claim 1 is characterized in that: The steps of training and preparing the fault diagnosis model include: Determine a fault diagnosis scenario and collect corresponding training data for the fault diagnosis scenario, wherein the training data includes fault data corresponding to the fault type; Preprocess the training data and label them according to the fault type and fault severity, respectively, as fault labels and normal labels; The labeled training data is divided into training set, validation set and test set; The convolutional neural network is trained using the training set to obtain a fault diagnosis model.

3. The lightweight fault diagnosis model compression method based on channel pruning and quantization fusion according to claim 1 is characterized in that: The L1 regularization is introduced during the model training process to constrain the weights of the convolutional layer; After training, the steps of selecting the convolution kernels with the lowest contribution percentage according to the L1 norm of the convolution kernels include: Set training parameters, including the number of training rounds, optimizer and its learning rate; In the training loop, the training set is passed to the fault diagnosis model to obtain the output, the loss function including the L1 regularization term is calculated, and backpropagation and parameter updates are performed through the optimizer; After each training round, the fault diagnosis model performance is evaluated using the validation set, and the loss value and accuracy are recorded; After the training is completed, all convolutional layers of the fault diagnosis model performance are traversed, and the L1 norm of the convolution kernel weights of each convolutional layer is calculated; The preset percentage convolution kernel is used to screen out the preset percentage convolution kernel with the lowest contribution, and the pruned fault diagnosis model is obtained.

4. The lightweight fault diagnosis model compression method based on channel pruning and quantization fusion according to claim 3 is characterized in that: After the training is completed, all convolution layers of the fault diagnosis model performance are traversed, and the L1 norm of the convolution kernel weight of each convolution layer is calculated in step S24, including: After training, the L1 norm of all convolution kernels of the fault diagnosis model performance is calculated. The calculation formula of the L1 norm of a single convolution kernel is: Among them, L1 per is the L1 norm of the cout convolution kernel, Q cout,cin,h,w Represents the convolution kernel weight value at the cout-th output channel, cin-th input channel, height h, and width w position; Determine a preset percentage, sort the calculated multiple convolution kernel L1 norms, and find the L1 norm threshold corresponding to the preset percentage position.

5. The lightweight fault diagnosis model compression method based on channel pruning and quantization fusion according to claim 1 is characterized in that: The step of converting floating-point parameters in the fault diagnosis model into 8-bit fixed-point representations by using an 8-bit fixed-point quantization method on the pruned fault diagnosis model to obtain the pruned and quantized fault diagnosis model comprises: Load the pruned fault diagnosis model parameters and structure based on PyTorch; Based on static quantization, a preset portion of data is selected from the training set as a calibration data set; Use the Data Loader to load the calibration dataset into memory.

6. The lightweight fault diagnosis model compression method based on channel pruning and quantization fusion according to claim 1 is characterized in that: The steps of fine-tuning the pruned and quantized fault diagnosis model and iteratively training the fault diagnosis model using the training set until the model adapts to the compressed structure and parameter representation to obtain an optimized fault diagnosis model include: Divide the training data into batches, input the training data into the pruned and quantized fault diagnosis model in batches, obtain output data, and obtain evaluation indicators based on the loss function; The training data is input into the fault diagnosis model without pruning and quantization in batches to obtain the original output data, and the evaluation index is obtained based on the loss function; The evaluation indicators of the output data are compared with the evaluation indicators of the original output data to obtain the difference between the pruned and quantized fault diagnosis model and the fault diagnosis model without pruning and quantization. The pruned and quantized fault diagnosis model corresponding to the difference that meets the preset conditions is used as the optimized fault diagnosis model.

7. The lightweight fault diagnosis model compression method based on channel pruning and quantization fusion according to claim 1 is characterized in that: The optimized fault diagnosis model is tested using the test set to verify the accuracy of the optimized fault diagnosis model in fault diagnosis; Check whether the size of the fault diagnosis model is compressed to 15MB. If the size meets the requirement, complete the steps of compressing the fault diagnosis model, including: Setting a data packet, dividing the data packet into multiple data intervals and marking the data amount corresponding to the data intervals, setting multiple data points in the data packet, each data point corresponding to a corresponding data interval, multiple data points are connected to each other, and the data points are moved in the data packet; The pruned fault diagnosis model is loaded into a data packet, and the pruned fault diagnosis model is attached through data points to obtain a data point set; Monitor whether the fault diagnosis model size is compressed to 15MB through the data point set. If the requirement is met, the compression of the fault diagnosis model is completed.

8. The lightweight fault diagnosis model compression method based on channel pruning and quantization fusion according to claim 7 is characterized in that: The step of monitoring whether the volume of the fault diagnosis model is compressed to 15MB by the data point set and completing the compression of the fault diagnosis model if the requirement is met includes: Determine the load position and load range of the fault diagnosis model in the data packet, and mark the data occupied by the fault diagnosis model with the corresponding data points in the data interval occupied by the fault diagnosis model; Associating data points with the fault diagnosis model, and connecting multiple data points associated with the fault diagnosis model; The data volume occupied by the fault diagnosis model is obtained by interacting and integrating the data occupied by the connected data points. If the volume of the fault diagnosis model is compressed to meet 15MB, the compression of the fault diagnosis model is completed.

Images