Lithium battery surface defect detection and model construction method based on multi-task learning

Abstract

The present invention provides a lithium battery surface defect detection based on multi-task learning and a model construction method thereof, which belongs to the technical field of lithium battery defect detection based on computer vision. A domain generalization neural network is constructed based on the lithium battery surface image and the steel surface defect image, and a multi-scale feature extraction module, a feature fusion module based on residual attention, and the first deep convolutional layer in the trained dual-task learning deep convolutional neural network are deployed on the background computer; an industrial high-precision CMOS industrial camera is used to collect the surface image of the lithium battery on the production line and transmit it to the background computer; the lithium battery surface image is sequentially passed through the multi-scale feature extraction module, the feature fusion module based on residual attention, and the deep convolutional layer to obtain the lithium battery surface defect detection result. The present invention solves the problem of poor generalization ability and data scarcity of existing defect detection models, and at the same time enhances the model's perception of tiny defects on the surface of lithium batteries and steel surfaces.

Description

Technical Field

[0001] The present invention belongs to the technical field of lithium battery surface defect detection based on computer vision, and in particular relates to a lithium battery surface defect detection based on multi-task learning and a model building method thereof. Background Art

[0002] Lithium batteries are widely used in portable electronic products, new energy vehicles, mobile power sources and other fields due to their advantages such as high energy density, high specific capacity, low self-discharge rate, long life and environmental protection. They are one of the mainstream energy sources for future development. However, in the production and use of lithium batteries, surface defects such as scratches, pits and stains will inevitably appear. These surface defects will seriously affect the performance and stability of the battery, and may cause safety problems such as fire or explosion. Therefore, detecting surface defects of lithium batteries is crucial to ensure battery performance and safety. However, traditional manual inspection is easily affected by subjective factors, with high false detection rate and missed detection rate, and requires a lot of manpower and time. With the development of computer technology, machine vision technology has gradually been applied to surface defect detection of lithium batteries.

[0003] The Chinese invention patent with application number CN202311740663.2 discloses a lightweight lithium battery surface defect detection method with enhanced image feature fusion, including the following steps: Step 1: Collect the original image of the lithium battery surface; Step 2: Classify and annotate the original image and create a data set; Step 3: Use the lightweight Mobilenetv2 as the backbone network, and improve the neck network and head network to construct a defect detection model; Step 4: Train the defect detection model to obtain the weight of the best prediction model; Step 5: Use the weight of the best prediction model to load the model and predict the results.

[0004] The Chinese invention patent with application number CN202010417852.6 discloses a lithium battery surface defect detection method based on a generative adversarial network, comprising the following steps: Step 1: Use a high-precision CMOS industrial camera to capture the surface image of the lithium battery; Step 2: Use the normal surface of the lithium battery surface image described in step 1 to train the generative adversarial network; Step 3: Use the generator test image of the trained generative adversarial network described in step 2 to reconstruct the image, and use the reconstructed image and the test image to perform a difference operation to obtain the area of ​​interest; Step 4: Difference the lithium battery surface image described in step 1 by binarization, and perform connected domain size screening to exclude false detection areas.

[0005] The above techniques have the following limitations:

[0006] The surface area of ​​lithium batteries is small and the defect shapes vary greatly. The above technology cannot learn the difference characteristics of the shape of lithium battery surface defects, resulting in poor generalization ability of the model. The recognition accuracy of lithium battery surface defects of the same type but with large shape differences is low;

[0007] The above technology requires a large amount of lithium battery surface image data when training the deep learning model. However, in the lithium battery production process, there are fewer defective lithium battery products. Therefore, it is difficult to obtain a large amount of defective lithium battery surface image data, which wastes manpower and material resources. Summary of the invention

[0008] In view of the above problems, the first aspect of the present invention proposes a method for constructing a lithium battery surface defect detection model based on multi-task learning, comprising the following process:

[0009] S1, collecting lithium battery surface images, including lithium battery surface defect-free images, lithium battery surface melting defect images, lithium battery surface scratch defect images, lithium battery surface crack defect images, and lithium battery surface groove defect images, and collecting steel surface defect images, including steel surface defect-free images, steel surface melting defect images, steel surface scratch defect images, steel surface crack defect images, and steel surface groove defect images;

[0010] S2, preprocessing the lithium battery surface image and the steel surface image; the preprocessing includes denoising and manual image labeling. LabelImg is used to label the defects in the image with rectangular boxes, including the defect location information, i.e. the horizontal and vertical coordinates of the center point of the rectangular box corresponding to the marked defect, as well as the height and width information, and the defect type;

[0011] S3, constructing a lithium battery surface defect detection model, including a multi-scale feature extraction module, a feature fusion module, and a dual-task learning deep convolutional neural network module connected in sequence;

[0012] The multi-scale feature extraction module extracts multi-scale features of lithium battery surface image data and steel surface image data respectively; the feature fusion module fuses the multi-scale features of lithium battery surface image and steel surface image respectively based on residual attention to obtain a fusion feature map of lithium battery surface image and a fusion feature map of steel surface image;

[0013] The multi-task learning deep convolutional neural network module includes two parallel deep convolutional layers, the first deep convolutional layer is used to solve the lithium battery surface defect detection task, and the second deep convolutional layer is used to solve the steel surface defect detection task, and different tasks complement each other's information; the lithium battery surface image fusion feature map is input into the first deep convolutional neural network, and the steel surface image fusion feature map is input into the second deep convolutional neural network. When the multi-task learning deep convolutional neural network is trained, the knowledge between the first deep convolutional neural network and the second deep convolutional neural network is transferred, so that the model can more accurately identify lithium battery surface defects similar to steel defects, thereby improving the accuracy of lithium battery surface defect detection;

[0014] S4, training the constructed lithium battery surface defect detection model to obtain the final model.

[0015] Preferably, the lithium battery surface image and the steel surface image use a denoising module to remove exposure noise to improve the quality of the image, and the denoising module includes an input layer, a first hole convolution layer, a second hole convolution layer, a third hole convolution layer, a fourth hole convolution layer, a first feature stitching layer, a second feature stitching layer, and an image reconstruction layer; wherein the input layer is composed of a convolution layer and a ReLU activation function, which is responsible for extracting the feature map of the lithium battery surface image and the steel surface image; the hole convolution layer is responsible for increasing the receptive field of the lithium battery surface image feature map and the steel surface image feature map, and a larger receptive field helps the model capture a wider range of contextual information in the image and helps to extract high-level defect features; the first hole convolution layer and the second hole convolution layer are parallel structures, and the first hole convolution layer and the second hole convolution layer are parallel structures. The three atrous convolution layers and the fourth atrous convolution layer are in parallel. The first feature splicing layer is responsible for vector splicing the output results of the first atrous convolution layer and the second atrous convolution layer. The second feature splicing layer is responsible for vector splicing the output results of the third atrous convolution layer and the fourth atrous convolution layer. The output of the first feature splicing layer is the input of the third atrous convolution layer and the fourth atrous convolution layer. This design allows the network to learn features of multiple scales at the same time, increasing the richness of features. The image reconstruction layer is composed of a convolution layer and a ReLU activation function. The output of the second feature splicing layer is used to reconstruct the lithium battery surface image feature map and the steel surface image. The lithium battery surface image and the steel surface image are respectively input into the denoising module to obtain the denoised lithium battery surface image. And the denoised steel surface image .

[0016] Preferably, the multi-scale feature extraction module includes five convolutional layers and one adaptive pooling layer, wherein the convolution kernel scales of the first convolutional layer and the second convolutional layer are , the convolution kernel scales of the third and fourth convolution layers are , the convolution kernel scale of the fifth convolutional layer is ; The first and second convolutional layers are mainly used for dimensional transformation of lithium battery surface defect feature maps and steel surface defect feature maps, which can reduce the number of channels of the feature maps, reduce the amount of calculation, and increase the nonlinear expression ability of the model. The third and fourth convolutional layers can further extract more detailed features, which help the model identify tiny defects on the surface of lithium batteries and steel. The fifth convolutional layer can extract higher-level semantic features, which helps the model identify macro defects or overall features on the surface of lithium batteries and steel. Through convolution kernels of different scales, multi-scale features from local to global can be extracted, which improves the model's ability to recognize lithium battery surface defects and steel surface defects. The adaptive pooling layer is mainly used to reduce the dimensions of lithium battery surface defect feature maps and steel surface defect feature maps, reduce the number of parameters, alleviate overfitting problems, and improve the generalization ability of the model.

[0017] The multi-scale feature extraction module is used to capture the features of defects of different scales in the lithium battery surface image and the steel surface image, retain the local details and global information of the defects, and convert the lithium battery surface image into Input the multi-scale feature extraction module to obtain the multi-scale features of the lithium battery surface image , the steel surface image Input multi-scale feature extraction module Multi-scale features of steel surface image .

[0018] Preferably, the feature fusion module consists of a multi-head attention module, a residual module and a normalization module, which are used to capture the sequence correlation between different channels of the feature map; Enhance model capabilities: The multi-head attention module executes multiple attention mechanisms in parallel, so that the model can focus on different angles. and The residual module enables the network to learn the residual between input and output instead of directly learning the mapping from input to output, thus alleviating the problem of gradient explosion.

[0019] Will The calculation formula for inputting the multi-head attention module is as follows:

[0020] ;

[0021] ;

[0022] in For the number of heads, For the The query linear projection matrix of each head, For the The key linear projection matrix of each head, For the The linear projection matrix of the head value, For each head dimension, is the output weight matrix, is the softmax activation function, is the vector concatenation operator;

[0023] Will The calculation formulas of the multi-head attention module are input as follows:

[0024] ;

[0025] ;

[0026] in For the number of heads, For the The query linear projection matrix of each head, For the The key linear projection matrix of each head, For the The linear projection matrix of the head value, For each head dimension, is the output weight matrix, is the softmax activation function, is the vector concatenation operator;

[0027] Next, and The residual module is respectively input, and the residual module consists of two linear layers, an activation function layer and a normalization layer. The specific formula is as follows:

[0028] ;

[0029] ;

[0030] in It is a feedforward neural network consisting of two linear layers and an activation function. For the mean operator and To find the standard deviation operator, , are learnable affine transformation parameters.

[0031] Preferably, the multi-task learning deep convolutional neural network module includes two parallel deep convolutional layers, the first deep convolutional layer is used to solve the lithium battery surface defect detection task, and the second deep convolutional layer is used to solve the steel surface defect detection task; different tasks can complement each other's information to improve the model's ability to identify defects. In lithium battery surface defect detection, by simultaneously learning the characteristics of steel surface defects, the model can more accurately identify lithium battery surface defects similar to steel defects, thereby improving the accuracy of lithium battery surface defect detection. ; The deep convolutional layer includes the sixth convolutional layer, the first mobile flip bottleneck convolutional layer MBConv1, the second mobile flip bottleneck convolutional layer MBConv2, the third mobile flip bottleneck convolutional layer MBConv3, the seventh convolutional layer, the average pooling layer, the first fully connected layer and the second fully connected layer; the convolution kernel scale of the sixth convolutional layer is , the convolution kernel scale of the seventh convolutional layer is , the MBConv1 contains a convolution kernel with a scale of The convolution and a convolution kernel scale is The MBConv2 contains a convolution kernel with a scale of The convolution and a convolution kernel scale is The MBConv3 contains a convolution kernel with a scale of The convolution and a convolution kernel scale is MBConv1, MBConv2, and MBConv3 use a reverse bottleneck design, which first expands the number of channels through convolution, then extracts features through depthwise separable convolution, and finally reduces the number of channels through convolution. This design can reduce the number of model parameters and computation, and effectively capture the spatial information of the input feature map.

[0032] Preferably, during training, the first deep convolutional layer and the second deep convolutional layer are trained using the following multi-task loss function: Update the parameters:

[0033] ;

[0034] in is the classification loss function for the lithium battery surface defect detection task, is the classification loss function for the steel surface defect detection task, and is the weight, The specific expression is:

[0035] ;

[0036] in, Represents a lithium battery surface image sample The probability of being predicted correctly, is the sample size of lithium battery surface image, represents the modulation factor, through which the difficult samples are focused. Its increase can reduce the weight of easy samples and increase the loss ratio of samples with small defects that are difficult to identify;

[0037] The specific expression is:

[0038] ;

[0039] in, Represents a steel surface image sample The probability of being predicted correctly, is the sample size of steel surface image.

[0040] Preferably, the , , which is convenient for solving the task of lithium battery surface defect detection.

[0041] The second aspect of the present invention also provides a lithium battery surface defect detection method based on multi-task learning, which deploys the lithium battery surface defect detection model constructed by the construction method described in the first aspect to a background computer and includes the following process:

[0042] Use industrial high-precision CMOS industrial cameras to collect surface images of lithium batteries on the production line and transmit them to the background computer;

[0043] The lithium battery surface image passes through the denoising module, multi-scale feature extraction module, residual attention-based feature fusion module and the first deep convolution layer in sequence to obtain the lithium battery surface defect detection results.

[0044] The third aspect of the present invention also provides a lithium battery surface defect detection device based on multi-task learning, the device includes at least one processor and at least one memory, the processor and the memory are coupled; the memory stores a computer execution program of the lithium battery surface defect detection model constructed by the construction method described in the first aspect; when the processor executes the computer execution program stored in the memory, the processor executes a lithium battery surface defect detection method based on multi-task learning.

[0045] The fourth aspect of the present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program or instruction of a lithium battery surface defect detection model constructed by the construction method described in the first aspect, and when the program or instruction is executed by a processor, the processor executes a lithium battery surface defect detection method based on multi-task learning.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention aims to propose a lithium battery surface defect detection method based on multi-scale multi-task learning to solve the problem that the prior art has poor generalization ability and requires a large amount of data for model training.

[0048] 1. Solve the problem of data scarcity: Lithium battery surface defect image data is relatively scarce. The method of the present invention can enhance the performance of the model by sharing data between different tasks, thereby alleviating the problem of scarce lithium battery surface defect image data to a certain extent. It solves the problem that the existing technology requires a large amount of lithium battery surface defect image data for model training, and improves the generalization ability of the model.

[0049] 2. This method uses a multi-scale convolutional network to fully capture the detailed features of lithium battery surface defects and steel surface defects at different scales, thereby enhancing the model's ability to perceive tiny defects on the surface of lithium batteries and steel. At the same time, the multi-scale convolutional layer effectively retains the surface defect information of lithium batteries and steel at different scales, avoiding the loss of details and blurring caused by a single-scale convolutional layer, and obtaining a richer and more accurate representation of the surface defects of lithium batteries and steel.

[0050] 3. The multi-scale feature fusion module enhances the network's ability to understand tiny defects on the surface of lithium batteries and steel materials, and effectively captures the microscopic details and overall morphological information of defects on the surface of lithium batteries and steel materials. In addition, the residual module and normalization module further improve the representativeness and robustness of the features, enhance the model's ability to identify different types of surface defects, and significantly improve the accuracy of recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The figure is an overall flow chart of the lithium battery surface defect detection method of the present invention.

[0052] Figure 2 This is a schematic diagram of the denoising module structure of the present invention.

[0053] Figure 3 Schematic diagram of the structure of the multi-scale feature extraction module of the present invention.

[0054] Figure 4 Schematic diagram of the feature fusion module structure based on residual attention in the present invention.

[0055] Figure 5 This is a schematic diagram of the dual-task learning deep convolutional neural network structure.

[0056] Figure 6 Schematic diagram of the detection results of scratch defects on the steel surface.

[0057] Figure 7 Schematic diagram of the scratch defect detection results on the lithium battery surface.

[0058] Figure 8 This is a simplified structural diagram of the lithium battery surface defect detection equipment in Example 2 of the present invention. DETAILED DESCRIPTION

[0059] The invention will be further described below in conjunction with specific embodiments.

[0060] Embodiment 1:

[0061] The present invention proposes a lithium battery surface defect detection method based on multi-scale multi-task learning. The overall process is as follows: Figure 1 As shown:

[0062] First, a data set was created. A high-precision CMOS industrial camera was used to collect images of lithium battery surfaces, including images of lithium battery surfaces without defects, images of lithium battery surfaces with melting defects, images of lithium battery surfaces with scratch defects, images of lithium battery surfaces with crack defects, and images of lithium battery surfaces with groove defects. The ratio of the number of the five types of images was 1:1:1:1:1. Images were manually annotated according to image types. Then, steel surface defect images were collected on the public data set NEU-CLS, including images of steel surfaces without defects, images of steel surfaces with melting defects, images of steel surfaces with scratch defects, images of steel surfaces with crack defects, and images of steel surfaces with groove defects. The ratio of the number of the five types of images was 1:1:1:1:1. The scale of steel images is ten times that of lithium battery images.

[0063] Construct a multi-scale feature extraction module to extract multi-scale features of lithium battery surface image data and steel surface image data respectively;

[0064] A feature fusion module based on residual attention is constructed to fuse the multi-scale features of the lithium battery surface image and the multi-scale features of the steel surface image, respectively, to obtain the fused feature map of the lithium battery surface image and the fused feature map of the steel surface image;

[0065] Construct a dual-task learning deep convolutional neural network, input the fusion feature map of the lithium battery surface image into the first deep convolutional neural network, input the fusion feature map of the steel surface image into the second deep convolutional neural network, and train the dual-task learning deep convolutional neural network;

[0066] The surface images of lithium batteries are detected based on the denoising module, multi-scale feature extraction module, residual attention-based feature fusion module and the trained dual-task learning deep convolutional neural network.

[0067] Regarding the denoising module Figure 2As shown in the figure, the design includes an input layer, a first dilated convolution layer, a second dilated convolution layer, a third dilated convolution layer, a fourth dilated convolution layer, a first feature splicing layer, a second feature splicing layer and an image reconstruction layer. This design allows the network to simultaneously learn features of multiple scales and increase the richness of features.

[0068] Regarding the multi-scale feature extraction module Figure 3 As shown, it includes five convolutional layers and one adaptive pooling layer, where the convolution kernel scales of the first and second convolutional layers are , the convolution kernel scales of the third and fourth convolution layers are , the convolution kernel scale of the fifth convolutional layer is ; The function of this module is to extract the features of lithium battery surface images and steel surface images at different levels from local to global, and enhance the perception of image edges and contrast;

[0069] The adaptive pooling layer can fully capture the features of defects of different scales in the lithium battery surface image and the steel surface image, retain the local details and global information of the defects, and transform the lithium battery surface image into Input the multi-scale feature extraction module to obtain the multi-scale features of the lithium battery surface image , the steel surface image Input multi-scale feature extraction module Multi-scale features of steel surface image .

[0070] Regarding the feature fusion module based on residual attention, such as Figure 4 As shown, it consists of a multi-head attention module, a residual module and a normalization module, which are used to capture the correlation between different channels of the feature map;

[0071] Will The calculation formula for inputting the multi-head attention module is as follows:

[0072] ;

[0073] ;

[0074] in For the number of heads, For the The query linear projection matrix of each head, For the The key linear projection matrix of each head, For the The linear projection matrix of the head value, For each head dimension, is the output weight matrix, is the softmax activation function, is the vector concatenation operator;

[0075] Will The calculation formulas of the multi-head attention module are input as follows:

[0076] ;

[0077] ;

[0078] in For the number of heads, For the The query linear projection matrix of each head, For the The key linear projection matrix of each head, For the The linear projection matrix of the head value, For each head dimension, is the output weight matrix, is the softmax activation function, is the vector concatenation operator;

[0079] Next, and The residual module is respectively input, and the residual module consists of two linear layers, an activation function layer and a normalization layer. The specific formula is as follows:

[0080] ;

[0081] ;

[0082] in It is a feedforward neural network consisting of two linear layers and an activation function. For the mean operator and To find the standard deviation operator, , are learnable affine transformation parameters.

[0083] Regarding multi-task learning of deep convolutional neural networks, such as Figure 5 As shown, it contains two parallel deep convolutional layers. The first deep convolutional layer solves the lithium battery surface defect detection task, and the second deep convolutional layer solves the steel surface defect detection task. The deep convolutional layer includes the sixth convolutional layer, the first mobile flip bottleneck convolutional layer (MBConv1), the second mobile flip bottleneck convolutional layer (MBConv2), the third mobile flip bottleneck convolutional layer (MBConv3), the seventh convolutional layer, the average pooling layer, the fully connected layer 1 and the fully connected layer 2; the convolution kernel scale of the sixth convolutional layer is , the convolution kernel scale of the seventh convolutional layer is , the MBConv1 contains a convolution kernel with a scale of The convolution and a convolution kernel scale is The MBConv2 contains a convolution kernel with a scale of The convolution and a convolution kernel scale is The MBConv3 contains a convolution kernel with a scale of The convolution and a convolution kernel scale is Depthwise separable convolution;

[0084] During training, deep convolutional layer 1 and deep convolutional layer 2 are trained using the following multi-task loss function Update the parameters:

[0085] ;

[0086] in is the classification loss function for the lithium battery surface defect detection task, is the classification loss function for the steel surface defect detection task, and The present invention is more concerned with solving the task of lithium battery surface defect detection. , ; The specific expression is:

[0087] ;

[0088] in, Represents a lithium battery surface image sample The probability of being predicted correctly, is the sample size of lithium battery surface image, represents the modulation factor, through which the hard-to-separate samples are focused. Its increase can reduce the weight of easy-to-separate samples and increase the loss ratio of hard-to-identify small defect samples. Through the loss function, the first deep convolutional layer and the second deep convolutional layer can use data from different tasks to enhance their own performance, thereby reducing the dependence on single-task training data.

[0089] The specific expression is:

[0090] ;

[0091] in, Represents a steel surface image sample The probability of being predicted correctly, is the sample size of steel surface image.

[0092] The lithium battery surface image is detected based on the multi-scale feature extraction module, the residual attention-based feature fusion module and the trained dual-task learning deep convolutional neural network. The process includes:

[0093] Deploy the multi-scale feature extraction module, the residual attention-based feature fusion module, and the first deep convolutional layer of the trained dual-task learning deep convolutional neural network to the background computer;

[0094] Use industrial high-precision CMOS industrial cameras to collect surface images of lithium batteries on the production line and transmit them to the background computer;

[0095] The surface image of the lithium battery is sequentially processed through the denoising module, the multi-scale feature extraction module, the feature fusion module based on residual attention and the first deep convolution layer to obtain the surface defect detection result of the lithium battery. Among them, the trained multi-scale feature extraction module extracts the surface image features of the lithium battery; the surface image features of the lithium battery are input into the trained feature fusion module to obtain the fusion features; the fusion features are input into the trained first deep convolution layer to obtain the surface image detection result of the lithium battery, and the rectangular box on the output image will select the surface defects in the image.

[0096] This example provides results that can be demonstrated as Figure 6 and Figure 7 As shown, Figure 6 The detection result of scratch defects on the steel surface. Figure 7 The test results of scratch defects on the surface of lithium batteries. Figure 7 It can be seen that the present invention can better detect defects on the surface of lithium batteries. This embodiment also provides a comparison of results with well-known models of deep convolutional neural networks, VGG19 and YOLOV8, and the evaluation indicators are mAP50 and mAP50:95. The results are shown in Table 1. The model proposed in the present invention has achieved the maximum value in both mAP50 and mAP50:95, indicating that the model has a higher accuracy. The method of the present invention is superior to the deep convolutional neural network, which shows that the multi-scale feature extraction module, feature fusion module and multi-task learning method proposed in the present invention are effective in the problem of lithium battery surface defect detection.

[0097] Table 1 Quantitative analysis of test data sets in different models

[0098] .

[0099] Embodiment 2:

[0100] like Figure 8As shown, the present invention also provides a lithium battery surface defect detection device based on multi-task learning, the device includes at least one processor and at least one memory, and also includes a communication interface and an internal bus; a computer execution program is stored in the memory; a computer execution program of a lithium battery surface defect detection model constructed by the construction method described in Example 1 is stored in the memory; when the processor executes the computer execution program stored in the memory, the processor can execute a lithium battery surface defect detection method based on multi-task learning. Wherein the internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus. Wherein the memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a U disk, a mobile hard disk, a read-only memory, a disk or an optical disk.

[0101] The device may be provided as a terminal, a server or other forms of devices.

[0102] Figure 8 is a block diagram of a device shown as an example. The device may include one or more of the following components: a processing component, a memory, a power component, a multimedia component, an audio component, an input / output (I / O) interface, a sensor component, and a communication component. The processing component generally controls the overall operation of the electronic device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component may include one or more processors to execute instructions to complete all or part of the steps of the above method. In addition, the processing component may include one or more modules to facilitate interaction between the processing component and other components. For example, the processing component may include a multimedia module to facilitate interaction between the multimedia component and the processing component.

[0103] The memory is configured to store various types of data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0104] The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0105] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0106] Embodiment 3:

[0107] The present invention also provides a computer-readable storage medium, which stores a computer execution program of the lithium battery surface defect detection model constructed by the construction method described in Example 1. When the computer execution program is executed by a processor, the processor can execute a lithium battery surface defect detection method based on multi-task learning.

[0108] Specifically, a system, device or equipment equipped with a readable storage medium may be provided, on which a software program code for implementing the functions of any of the above-mentioned embodiments is stored, and a computer or processor of the system, device or equipment reads and executes the instructions stored in the readable storage medium. In this case, the program code read from the readable medium itself can implement the functions of any of the above-mentioned embodiments, so the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0109] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk (such as CD-ROM, CD-R, CD-RW, DVD-20 ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tape, etc. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0110] It should be understood that the storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in a terminal or server.

[0111] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0112] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0113] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0114] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for constructing a lithium battery surface defect detection model based on multi-task learning, characterized in that: The process includes: S1, collecting lithium battery surface images, including lithium battery surface defect-free images, lithium battery surface melting defect images, lithium battery surface scratch defect images, lithium battery surface crack defect images, and lithium battery surface groove defect images, and collecting steel surface defect images, including steel surface defect-free images, steel surface melting defect images, steel surface scratch defect images, steel surface crack defect images, and steel surface groove defect images; S2, preprocessing the lithium battery surface image and the steel surface image; the preprocessing includes denoising and labeling; S3, constructing a lithium battery surface defect detection model, including a multi-scale feature extraction module, a feature fusion module, and a multi-task learning deep convolutional neural network module connected in sequence; The multi-scale feature extraction module extracts multi-scale features of the lithium battery surface image data and the steel surface image data respectively to obtain feature maps containing different resolutions; The feature fusion module fuses the multi-scale features of the lithium battery surface image and the multi-scale features of the steel surface image based on residual attention, and obtains the fusion feature map of the lithium battery surface image and the fusion feature map of the steel surface image; the multi-task learning deep convolutional neural network module includes two parallel deep convolutional layers, the first deep convolutional layer is used to solve the lithium battery surface defect detection task, and the second deep convolutional layer is used to solve the steel surface defect detection task, and different tasks complement each other; S4, training the constructed lithium battery surface defect detection model to obtain the final model.

2. The method for constructing a lithium battery surface defect detection model based on multi-task learning according to claim 1, characterized in that: The lithium battery surface image and the steel surface image use a denoising module to remove exposure noise to improve the quality of the image. The denoising module includes an input layer, a first hole convolution layer, a second hole convolution layer, a third hole convolution layer, a fourth hole convolution layer, a first feature stitching layer, a second feature stitching layer, and an image reconstruction layer; wherein the input layer is composed of a convolution layer and a ReLU activation function, and is used to extract feature maps of the lithium battery surface image and the steel surface image; the hole convolution layer is used to increase the receptive field of the lithium battery surface image feature map and the steel surface image feature map, wherein the first hole convolution layer and the second hole convolution layer are parallel structures, and the third hole convolution layer and the fourth hole convolution layer are parallel structures. It is a parallel structure. The first feature splicing layer is responsible for vector splicing the output results of the first and second dilated convolution layers. The second feature splicing layer is responsible for vector splicing the output results of the third and fourth dilated convolution layers. The output of the first feature splicing layer is the input of the third and fourth dilated convolution layers. The image reconstruction layer is composed of a convolution layer and a ReLU activation function. The output of the second feature splicing layer is used to reconstruct the lithium battery surface image feature map and the steel surface image. The lithium battery surface image and the steel surface image are input into the denoising module respectively to obtain the denoised lithium battery surface image ILIT and the denoised steel surface image ISTE.

3. The method for constructing a lithium battery surface defect detection model based on multi-task learning according to claim 1, characterized in that: The multi-scale feature extraction module includes five convolution layers and one adaptive pooling layer, wherein the convolution kernel scales of the first and second convolution layers are 1×1, the convolution kernel scales of the third and fourth convolution layers are 3×3, and the convolution kernel scale of the fifth convolution layer is 5×5; wherein the first and second convolution layers are used for dimensional transformation of the lithium battery surface defect feature map and the steel surface defect feature map; the third and fourth convolution layers are used for further extracting more detailed features so that the model can identify the tiny defects on the surface of the lithium battery and the surface of the steel; the fifth convolution layer is used for extracting higher-level semantic features so that the model can identify the macroscopic defects or overall features on the surface of the lithium battery and the surface of the steel; The adaptive pooling layer is used to capture the features of defects of different scales in the lithium battery surface image and the steel surface image, retain the local details and global information of the defects, and input the lithium battery surface image ILIT into the multi-scale feature extraction module to obtain the multi-scale feature F of the lithium battery surface image. LIT , the steel surface image ISTE is input into the multi-scale feature extraction module steel surface image multi-scale feature F STE .

4. The method for constructing a lithium battery surface defect detection model based on multi-task learning according to claim 1, characterized in that: The feature fusion module consists of a multi-head attention module, a residual module and a normalization module, and is used to capture the correlation between different channels of the feature map; The multi-head attention module executes multiple attention mechanisms in parallel, allowing the model to focus on F from different perspectives. LIT and F STE The residual module enables the model to learn the residual between input and output, rather than directly learning the mapping from input to output, thus alleviating the problem of gradient explosion. F LIT The calculation formula for inputting the multi-head attention module is as follows: Where N is the number of heads, W is i Q，LIT is the query linear projection matrix of the i-th head, W i K，LIT is the key linear projection matrix of the i-th head, W i K，LIT is the linear projection matrix of the value of the i-th head, d is the dimension of each head, W O，LIT is the output weight matrix, σ is the softmax activation function, and Concat(·) is the vector concatenation operator; F STE The calculation formula for inputting the multi-head attention module is as follows: Where N is the number of heads, W is i Q，STE is the query linear projection matrix of the i-th head, W i K，STE is the key linear projection matrix of the i-th head, W i K，STE is the linear projection matrix of the value of the i-th head, d is the dimension of each head, W O，STE is the output weight matrix, σ is the softmax activation function, and Concat(·) is the vector concatenation operator; Next, and The residual module is respectively input, and the residual module consists of two linear layers, an activation function layer and a normalization layer. The specific formula is as follows: Where LLS(·) is a feed-forward neural network consisting of two linear layers and an activation function, μ(·) is the mean operator and y(·) is the standard deviation operator, ∈=0.0000001, and γ is a learnable affine transformation parameter.

5. The method for constructing a lithium battery surface defect detection model based on multi-task learning according to claim 1, characterized in that: The multi-task learning deep convolutional neural network module includes two parallel deep convolutional layers, the first deep convolutional layer is used to solve the lithium battery surface defect detection task, and the second deep convolutional layer is used to solve the steel surface defect detection task; different tasks complement each other's information to improve the model's ability to recognize defects; the deep convolutional layer includes the sixth convolutional layer, the first mobile flip bottleneck convolutional layer MBConv1, the second mobile flip bottleneck convolutional layer MBConv2, the third mobile flip bottleneck convolutional layer MBConv3, the seventh convolutional layer, the average pooling layer, the a fully connected layer and a second fully connected layer; the convolution kernel scale of the sixth convolution layer is 3×3, the convolution kernel scale of the seventh convolution layer is 1×1, the MBConv1 includes two convolutions with a convolution kernel scale of 1×1 and one depth-separable convolution with a convolution kernel scale of 3×3, the MBConv2 includes two convolutions with a convolution kernel scale of 1×1 and one depth-separable convolution with a convolution kernel scale of 5×5, and the MBConv3 includes two convolutions with a convolution kernel scale of 1×1 and one depth-separable convolution with a convolution kernel scale of 3×3.

6. The method for constructing a lithium battery surface defect detection model based on multi-task learning according to claim 5, characterized in that: During the training process, the first and second depth convolutional layers are parameter updated by the following multi-task loss function L: L=αL1+βL2 Where L1 is the classification loss function of the lithium battery surface defect detection task, L2 is the classification loss function of the steel surface defect detection task, α and β are weights, and the specific expression of L1 is: in, represents the probability that the lithium battery surface image sample i is predicted correctly, U is the sample size of the lithium battery surface image, and ζ represents the modulation factor, which is used to focus on the difficult samples. Its increase can reduce the weight of the easy samples and increase the loss ratio of the samples with small defects that are difficult to identify; The specific expression of L2 is: in, represents the probability that the steel surface image sample i is predicted correctly, and Q is the sample size of the steel surface image.

7. The method for constructing a lithium battery surface defect detection model based on multi-task learning according to claim 6, characterized in that: The α=0.8 and β=0.2 facilitate the solution of the lithium battery surface defect detection task.

8. A lithium battery surface defect detection method based on multi-task learning, characterized in that: The lithium battery surface defect detection model constructed by the construction method according to any one of claims 1 to 7 is deployed on a background computer, and includes the following processes: Use industrial high-precision CMOS industrial cameras to collect surface images of lithium batteries on the production line and transmit them to the background computer; The lithium battery surface image passes through the denoising module, multi-scale feature extraction module, residual attention-based feature fusion module and the first deep convolution layer in sequence to obtain the lithium battery surface defect detection results.

9. A lithium battery surface defect detection device based on multi-task learning, characterized in that: The device includes at least one processor and at least one memory, the processor and the memory are coupled; the memory stores a computer execution program of a lithium battery surface defect detection model constructed by the construction method as described in any one of claims 1 to 7; when the processor executes the computer execution program stored in the memory, the processor executes a lithium battery surface defect detection method based on multi-task learning.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instruction of a lithium battery surface defect detection model constructed by the construction method as described in any one of claims 1 to 7. When the program or instruction is executed by a processor, the processor executes a lithium battery surface defect detection method based on multi-task learning.

Images