New energy electrode plate defect detection method and system based on 3D laser point cloud image

Abstract

The invention discloses a new energy electrode plate defect detection method and system based on a 3D laser point cloud image. The system comprises a bimodal feature fusion module, a backbone module, a neck module, a detection head and an evaluation module. Wherein the bimodal feature fusion module comprises an image fusion module, a space and channel attention module and a DSL module, and the image fusion module is used for performing first feature extraction and fusion on image features; and the space and channel attention module is used for standardizing the image after the first fusion, and performing superposition fusion on the standardized image and the original image. Compared with the prior art, the multi-dimensional image is formed by performing extraction fusion on the image features for multiple times, so that defect inspection can be performed on the electrode plate better.

Description

Technical Field

[0001] The present invention relates to the field of electrode plate detection, and in particular to a new energy electrode plate defect detection method and system based on 3D laser point cloud images. Background Art

[0002] In recent years, the field of new energy has developed very rapidly. China has become one of the countries with the widest application of new energy technologies in the world, among which hydrogen energy is recognized as a clean energy. my country is accelerating the layout and development plan of hydrogen energy, and has made progress in many aspects of the field of hydrogen energy. Hydrogen fuel cells are a power generation device that converts the chemical energy of hydrogen into electrical energy. They have the advantages of zero carbon emissions, no pollution, low operating temperature, and low noise. They have always been considered to be the ultimate solution to using hydrogen energy to solve the future energy crisis of mankind.

[0003] The current mainstream method for producing hydrogen is electrolysis of water, and the electrolyzer used is an important tool for producing hydrogen by electrolysis of water. Small defects in the electrode plate will affect the performance of the hydrogen fuel cell. Therefore, it is very important to provide a set of accurate and complete electrode plate defect detection methods. However, the current mainstream defect detection is still focused on plane images. In actual production, there are many types of defects, and some concave and convex three-dimensional defects with height differences are easy to miss if they are only based on plane images. Summary of the invention

[0004] The purpose of the present invention is to address the problem that the existing electrode plate detection is mainly plane image, but the defect detection is incomplete, which makes the electrode plate prone to problems during use. To address this shortcoming, a new energy electrode plate defect detection method and system based on 3D laser point cloud images are proposed.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A new energy electrode plate defect detection method based on 3D laser point cloud images, including a two-dimensional linear motor motion platform, a 3D laser line scanning device and an industrial computer, including the following steps:

[0007] S1. Scan the electrode plate in multiple directions by a 3D laser line scanning device to form a 3D point cloud image and data set, and pre-process the data set; the 3D point cloud image and data set include an RGB image and a depth image;

[0008] S2, dividing the 3D point cloud image and the data set into corresponding training sets and test sets respectively;

[0009] S3, input the training set into the YOLOv5 network for training to obtain a defect detection model;

[0010] S4. Test and correct the accuracy of the defect detection model through the test set;

[0011] S5. Evaluate the defect detection model based on the evaluation method;

[0012] S6, inputting the 3D point cloud image and data set of the electrode plate collected by the 3D laser line scanning device into the modified defect detection model for defect analysis;

[0013] S7. After the defect detection model is detected, the detection result is output.

[0014] As a further preference of the present invention, the pretreatment comprises:

[0015] Calibrate the main plane of the 3D point cloud image, separate the depth information of the 3D point cloud image based on the Z axis, and form an RGB image and a depth image;

[0016] The data set is subjected to invalid point removal, scanning vibration compensation and noise compensation.

[0017] As a further preferred embodiment of the present invention, S3 includes:

[0018] S31, extracting features from the RGB image and the depth image respectively to obtain RGB features and depth features;

[0019] S32, RGB features and depth features are convolved to compress the number of channels.

[0020] S33, through dilated convolution, the receptive field of the network is expanded;

[0021] S34, restore the number of channels through convolution, and after batch normalization and linear correction unit activation function, add the residual edge to form a residual structure to obtain new RGB features and depth features;

[0022] S35, the new RGB features and depth features are stacked and input into the attention module.

[0023]

[0024]

[0025] Among them, ReLU represents the activation function, F′ RGB and F′ D Represents the features after convolution and CBS structure, F RGB and F D Represents the original features, represents the concatenation of two modal features along the channel dimension, x i Represents the original input value of the i-th sample on a certain feature dimension, μ BRepresents the mean of the current mini-batch data (calculated by dimension), is the variance of the current mini-batch data, ε is a very small positive value, y i is the standardized value of the i-th sample; the new RGB features and depth features are fused to form the fusion feature F fusion .

[0026] S36, normalize the fusion features based on the spatial attention mechanism;

[0027] S37, based on the attention mechanism and the spatial attention mechanism, the standardized fusion features are superimposed and fused with the 3D point cloud image;

[0028] S38, performing feature extraction on the superimposed fusion features and the depth image based on depth convolution and point-by-point convolution respectively;

[0029] S39, the features extracted in S38 are fused through the SE attention mechanism to form the third fusion feature;

[0030] S310 and DBL modules extract and enhance the three-dimensional fusion features.

[0031] S311. The features extracted from the three features are fused by downsampling through the asymptotic feature pyramid network to form a defect detection model.

[0032] As a further preferred embodiment of the present invention, the defect detection model is output through a repeated convolution module.

[0033] As a further preferred embodiment of the present invention, S5 includes:

[0034] according to

[0035]

[0036] Among them, TP is the number of positive samples correctly identified, FP is the number of negative samples misclassified, FN is the number of true positive samples that are not correctly detected by the model, the F1 indicator represents the harmonic mean, and s is the total number of categories; the precision Precision, recall rate Recall, average precision AP achieved at different recall levels, and average precision mAP of different categories are calculated.

[0037] A new energy electrode plate defect detection system based on 3D laser point cloud images, including a dual-modal feature fusion module, a backbone module, a neck module, a detection head and an evaluation module;

[0038] The dual-modal feature fusion module includes an image fusion module, a space and channel attention module, and a DSL module. The image fusion module is used to extract and fuse the image features for the first time; the space and channel attention module is used to standardize the image after the first fusion, and superimpose and fuse the standardized image with the original image;

[0039] The backbone module includes a ConvNeXtV2SE module and a channel-aware aggregation module. The ConvNeXtV2SE module includes an improved convolutional network and an attention module SE; the improved convolutional network performs deep feature extraction on the superimposed fused image; the attention module SE performs a second fusion on the features after the deep feature extraction, and the ConvNeXtV2SE module performs a third feature extraction on the image after the second fusion;

[0040] The neck module includes a DBL module, a feature asymptotic feature pyramid module and a RepConv module. The DBL module is used to enhance the feature expression capability; the asymptotic feature pyramid module is used to fuse the enhanced features in the three feature extractions; and the repeated convolution module is used to increase the reusability of important features.

[0041] The evaluation module is used to evaluate the accuracy and recall of the detection system.

[0042] The present invention proposes a new energy electrode plate defect detection method and system based on 3D laser point cloud images, which has the following beneficial effects compared with the prior art:

[0043] 1. The present invention forms a multi-dimensional image by extracting and fusing image features multiple times, which can better inspect the defects of the electrode plate;

[0044] 2. The DBL module is used to further process features after the three-layer output of the backbone, producing a more comprehensive feature fusion in the neck. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the process of the present invention;

[0046] Figure 2 is a schematic diagram of the device of the present invention;

[0047] Figure 3 It is a schematic diagram of the structure of the system model of the present invention;

[0048] Figure 4 is a schematic diagram of the feature fusion module;

[0049] Figure 5 is a schematic diagram of the spatial and channel attention module;

[0050] Figure 6This is the schematic diagram of the ConvNeXtV2SE module;

[0051] Figure 7 It is a schematic diagram of the CAA module;

[0052] Figure 8 It is a schematic diagram of the structure of the SE attention mechanism;

[0053] Fig. 9 It is a schematic diagram of repeated convolution module;

[0054] Fig.10 It is a schematic diagram of the RAFPN architecture;

[0055] Fig.11 This is the schematic diagram of the DBL module. DETAILED DESCRIPTION

[0056] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Embodiment 1: Combination Figure 1-Figure 2 A new energy electrode plate defect detection device based on 3D laser point cloud images includes a two-dimensional linear motor motion platform, a 3D laser line scanning device and an industrial computer.

[0058] When in use, the fuel cell plate to be tested is placed on a two-dimensional linear motor motion platform. The linear motor is controlled to run at a constant speed in one direction, and the 3D laser is triggered to scan at the same time. A point cloud is generated for the scanned part, and the platform's movement accuracy is 2μm.

[0059] In order to meet the collection of 3D point cloud images, the LJ-X8060 line laser measuring instrument under Keyence is used. The measuring instrument can perform point cloud scanning on the electrode plate. The image sensor it uses ensures consistency during the scanning process. In addition, the X8060 can ensure that the measurement accuracy is maintained while quickly moving the sample. The principle is to diffuse the laser into a line laser through a cylindrical objective lens and then project it on the surface of the target object to form diffuse reflection. After the reflected light is imaged on the CMOS, the displacement and shape are measured by detecting changes in position and shape. The X8060 uses a large-aperture light-receiving lens to enable the CMOS to receive more reflected light, thereby improving the scanning accuracy. In the present invention, the point cloud data in the width direction of the object is measured by the X-axis, and the point cloud data in the height direction is measured by the Z-axis. The parameter data of the LJ-X8060 are shown in Table 1 below. The structural line laser measuring instrument meets the accuracy requirements required for this experiment.

[0060] Table 1

[0061]

[0062]

[0063] Embodiment 2: A defect detection method for new energy electrode plates based on 3D laser point cloud images. The present invention first uses the Keyence LJ-X8060 line laser to scan the electrode plate from different parts and form a data set, and divides the data set into regions; preprocesses the divided 3D point cloud image data set, including removing invalid points, scanning vibration compensation and noise compensation; calibrates the main plane of the 3D point cloud image, separates the depth information of the image with the Z axis, and forms an RGB image and a depth image; inputs the processed RGB and depth paired images into the designed YOLOv5 network; utilizes the RGB and depth image feature-level fusion module based on the spatial and channel attention mechanism in the network; trains the network in the prepared data set, and inputs the bimodal images into the network in pairs; extracts features from the RGB image and depth image of the electrode plate to obtain a feature map; and completes the defect detection of the electrode plate through the feature processing structure of the YOLOv5 network.

[0064] The specific steps include:

[0065] S1. Use 3D laser line scanning equipment to scan the electrode plate in multiple directions to form a 3D point cloud image and data set, and pre-process the data set.

[0066] Calibrate the main plane of the 3D point cloud image, separate the depth information of the 3D point cloud image based on the Z axis, and form an RGB image and a depth image;

[0067] The data set is subjected to invalid point removal, scanning vibration compensation and noise compensation.

[0068] S2. Divide the 3D point cloud image and data set into corresponding training set and test set respectively.

[0069] S3. Input the training set into the YOLOv5 network for training to obtain a defect detection model.

[0070] Feature fusion is an effective way to combine different information from various sensors. The current mainstream fusion methods are pixel-level, feature-level and decision-level fusion, which can be deployed at different depths of the network. This invention adopts a feature-level fusion module (RGBDF) of RGB image and depth image based on spatial and channel attention mechanism, which mainly solves the problem that the RGB image of the electrode plate lacks height information.

[0071] The structure of the image fusion module is as follows: Figure 3 As shown, the fusion steps are as follows:

[0072] The feature fusion module passes in multimodal features in a paired manner. The RGB features and depth features are compressed to 1 / 4 of the original number of channels through a 1×1 convolution, and then a 3×3 dilated convolution is performed to expand the network's receptive field, thereby obtaining more dense data. After that, the number of channels is restored through convolution and the Batch Normal standardization and Rectified Linear Unit (ReLU) activation function are performed, and the residual edge is added to form a residual structure, which solves the gradient vanishing and gradient exploding problems while deepening the network depth. Then, the obtained RGB features and depth features are stacked and input into the attention module. Finally, the number of channels is restored through 1×1 convolution, Batch Normal standardization and Sigmoid activation function, so that the fused feature size is the same as the input RGB features and depth features. Its operating formula is as follows:

[0073]

[0074] Among them, F fusion represents the features after the fusion of the depth image and the RGB image, ReLU represents the activation function, and F′ RGB and F′ D represents the features extracted by the CBS structure composed of convolution (Conv), batch normalization (BN) and Sigmoid weighted linear unit, F RGB and F D Represents the original features, It represents the concatenation of two modal features along the channel dimension. The second formula is the Batch Norma formula, where x i Represents the original input value of the i-th sample on a certain feature dimension, μ B Represents the mean of the current mini-batch data (calculated by dimension), is the variance of the current mini-batch data, ε is a very small positive value, y i is the standardized value of the i-th sample. The third formula is the Sigmoid activation function.

[0075] The spatial and channel attention modules are as follows Figure 5 As shown in the figure, the attention module uses spatial attention and channel attention. The spatial attention first compresses the number of channels through a 1×1 convolution on the feature map, and then passes it through two 3×3 dilated convolutions. After that, the number of channels is restored through convolution and batch normalization. The channel attention mechanism uses the SqueezeExcitation module, which first multiplies the original feature map with the result of the spatial attention module. Then, in order to avoid losing the features of large defects after the network depth is too deep, the original image is added after multiplying the original feature to ensure that the output size is the same as the original input size.

[0076] As the building block of the feature extraction layer, the C3 module directly connects the feature map processed by the bottleneck layer to the original input feature map. However, this feature fusion method may not fully capture the different feature maps of the defect, resulting in insufficient extracted features. In addition, the bottleneck layer structure of the C3 module consists of a stack of convolutional layers, which may hinder the model from processing multi-scale defect targets. Therefore, the Channel-Aware Aggregation (CAA) module is used instead of the C3 module.

[0077] First, the backbone network does not extract enough features, resulting in low accuracy in small target defect detection of parts. The present invention uses ConvNeXt V2 Block to better understand the high-level structure of the data and extract more meaningful feature representations, thereby improving the performance of the model in image detection tasks. In short, it consists of deep convolution and point-by-point convolution. The main advantage of deep convolution is that it reduces the number of parameters and the amount of computation required, thereby improving model efficiency. On the other hand, point-by-point convolution can retain and integrate the feature maps of deep convolution and improve the model's ability to capture interactions between different channels.

[0078] At the same time, in order to further enhance the channel representation ability of the network, the SE attention mechanism is integrated with ConvNeXt V2Block (improved convolutional network) to form Figure 6 The ConvNeXtV2SE module shown, Figure 7 The SE module performs two main operations: squeeze and excitation, as shown in the following formula. F stands for squeeze. In this step, the feature map is globally averaged and pooled to generate a 1×1×c vector, where u is the feature map and u c (i, j) is the value at position (i, j) and the input u of the cth channel convolution layer. The feature map is then pooled into a global average and the weight W is used to generate equation 2, where W is obtained through learning. The generated weight information can adaptively weigh the features in each channel, enabling the network to better adapt to complex tasks and scenarios and improve feature representation capabilities. The structure of the SE attention mechanism is shown in Figure 6 As shown. The RGB and depth image fusion network is used to perform feature-level fusion of the RGB and depth images after 3D point cloud image processing in the backbone network. The fused image continues to extract features. The backbone network also includes a channel-aware aggregation module CAA, where the expression of the attention mechanism is:

[0079]

[0080] s=F ex (z, W)=σ(g(z, W))=σ(W2δ(W1z)),

[0081] Where H represents the height of the feature map, W represents the width of the feature map, Z represents the generated feature map, δ represents the ReLU activation function, W1 and W2 represent two fully connected layers, and finally σ represents the Sigmoid function.

[0082] Secondly, the first convolutional layer CBS1 is used in the present invention to process the input x and divide the result into two parts with equal number of channels. These two parts are then taken as list y. The last group of elements y2 in list y are processed sequentially by the ConvNeXtV2SE module. Each ConvNeXtV2SE block consists of a 7×7 deep convolutional layer, a layer norm layer, and two linear layers (pwconv1 and pwconv2). The ConvNeXtV2SE block also contains global response normalization (GRN) and squeeze excitation (SE) layers for enhanced feature extraction and feature importance determination, and the output of each module is extended to list y. Finally, all elements in list y (including the original segments and elements processed by the ConvNeXtV2SE module) are connected together, and the connection result is then passed to the second convolutional layer CBS2 to produce the final output.

[0083] The perceptual aggregation module CAA consists of two main convolutional layers (CBS1 and CBS2). The size of the CBS1 convolutional layer is 1×1×(2c), and the ConvNeXtV2SE block has a depthwise convolutional layer of size 7×7×(c). The linear layers (pwconv1 and pwconv2) of the ConvNeXtV2SE block have sizes of 1×1×(4c) and 1×1×c, and the size of the CBS2 convolutional layer is 1×1×c.

[0084] In the method of the present invention, AFPN (Asymptotic Feature Pyramid Network) is used instead of PANet. The AFPN module accepts feature maps of three different levels with different spatial dimensions and numbers of channels. The three input feature maps are upsampled or downsampled to match their spatial dimensions with the spatial dimensions of the current level. If upsampling is performed, the feature map is upsampled to the target size using a bilinear interpolation method. If downsampling is performed, a 1×1 convolution is applied to perform the operation. Finally, a feature fusion operation is performed on the three adjusted feature maps to produce a fused feature map. The fusion of large, medium and small layers of features is used to improve the model's ability to understand and detect objects and scenes of different scales. The use of three-layer fusion reduces the computational complexity while maintaining high performance. AFPN can fuse feature maps of different layers at the same time, thereby reducing the risk of information loss and improving the detection of small targets. In order to further reduce the computational complexity of the model, the method of the present invention uses a repeated convolution module RepConv to process the output of feature fusion. The complex convolution module RepConv can increase the reusability of important features and mitigate the impact of changes in feature resolution, thereby further improving the detection performance of the model at different scales and reducing the number of parameters, as follows Fig. 9 As shown. The present invention uses As output, As input. Where C1=C2, H1×H2, W1=W2. RepConv consists of three parts, namely BN layer, 1×1 convolution + BN layer, and 3×3 convolution + BN layer, where the BN layer is a batch normalization layer. The feature map is summed element by element after passing through the three branches to obtain the final feature map. This structure can significantly enhance the multi-scale representation capability of the model to solve the multi-scale problem of defects in complex industrial environments. In the inference stage, the above three-branch structure can be equivalent to a 3×3 convolution structure with batch normalization parameters, and the equivalent process is shown in the following formula. The three-part structure used during training can be expressed as follows:

[0085]

[0086] in, σ3, γ3 and β3 are parameters in the batch normalization operation.

[0087] By integrating AFPN with RepConv, the RAFPN architecture is formed. This effectively improves the accuracy of defect target detection of different scales. Fig.10 shown.

[0088] In actual detection, most of the defective objects are small objects with overlapping, occlusion and multi-scale problems. In order to enable the model to process features more fully before RAFPN feature fusion and enrich the fusion process to locate more areas conducive to object detection, a DBL module is added after the three-layer feature output of the backbone feature extraction network, such as Fig.11 shown.

[0089] S4. Test and correct the accuracy of the defect detection model through the test set.

[0090] S5. Evaluate the defect detection model based on the evaluation method.

[0091] In order to evaluate the overall performance of the model, several evaluation indicators are used in the method of the present invention, namely precision (P), recall (R), average precision (AP), mean average precision (mAP), FPS, giga floating point operations per second (GFLOPS), parameters and model volume. AP represents the area enclosed by the precision recall curve and can be calculated using the following formula, where P, R, F1 and the calculation expression are as follows:

[0092]

[0093]

[0094] In the evaluation method provided, True Positives (TP) refers to the number of correctly identified positive samples, while False Positives (FP) refers to the number of misclassified negative samples. False Negatives (FN) refers to the number of true positive samples that are not correctly detected by the model. The F1 index represents the harmonic mean, which combines Precision and Recall and is calculated by their harmonic mean.

[0095] AP represents the average precision achieved at different recall levels. mAP represents the average AP of different categories, and its formula is as follows:

[0096]

[0097] Among them, s represents the total number of categories. In addition, the model's ability to detect frames per second is expressed in FPS, and the higher the value, the faster the detection speed. GFLOPS represents the number of floating-point calculations performed by the model per second. The larger the GFLOPS value, the higher the computational complexity. These parameters represent the complete number of trainable features in the model, including the individual weights of each layer and the number of network layers. The model volume refers to the total size of the weight file, which is determined by the model's data type and the total number of trainable parameters. The unit of the model volume is expressed in MB.

[0098] S6. Input the 3D point cloud image and data set of the electrode plate collected by the 3D laser line scanning device into the modified defect detection model for defect analysis.

[0099] S7. After the defect detection model is detected, the detection result is output.

[0100] Embodiment 3: A new energy electrode plate defect detection system based on 3D laser point cloud images, comprising a dual-modal feature fusion module, a backbone module, a neck module, a detection head and an evaluation module;

[0101] Among them, the bimodal feature fusion module includes an image fusion module, a spatial and channel attention module and a DSL module. The image fusion module is used to perform the first feature extraction and fusion of image features; the spatial and channel attention module is used to standardize the image after the first fusion, and superimpose and fuse the standardized image with the original image.

[0102] The backbone module includes a ConvNeXtV2SE module and a perception aggregation module CAA. The ConvNeXtV2SE module includes an improved convolutional network and an attention module SE; the improved convolutional network performs deep feature extraction on the superimposed and fused image; the attention module SE performs a second fusion on the features after the deep feature extraction, and the ConvNeXtV2SE module performs a third feature extraction on the image after the second fusion.

[0103] The neck module consists of a DBL module, an asymptotic feature pyramid module, and a RepConv module, which are composed of a deep over-parameterized convolutional layer (DO-DConv), batch normalization (BatchNormalization), and a leaky linear rectifier activation function (LeakRelu); the DBL module is added after the three-layer feature output of the backbone feature extraction network to enhance the feature expression capability. The enhanced feature input features are the asymptotic feature pyramid module and the RepConv module. The asymptotic feature pyramid module is used to fuse the enhanced features from the three feature extractions; the RepConv module is used to increase the reusability of important features; in order to enable the model to process features more fully before the RAFPN feature fusion, and enrich the fusion process to locate more areas that are conducive to target detection.

[0104] The evaluation module is used to evaluate the accuracy and recall of the detection system.

[0105] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A new energy electrode plate defect detection method based on 3D laser point cloud images, comprising a two-dimensional linear motor motion platform, a 3D laser line scanning device and an industrial computer, characterized in that: The following steps are involved: S1. Scan the electrode plate in multiple directions by a 3D laser line scanning device to form a 3D point cloud image and data set, and pre-process the data set; the 3D point cloud image and data set include an RGB image and a depth image; S2, dividing the 3D point cloud image and the data set into corresponding training sets and test sets respectively; S3, input the training set into the YOLOv5 network for training to obtain a defect detection model; S4. Test and correct the accuracy of the defect detection model through the test set; S5. Evaluate the defect detection model based on the evaluation method; S6, inputting the 3D point cloud image and data set of the electrode plate collected by the 3D laser line scanning device into the modified defect detection model for defect analysis; S7. After the defect detection model is detected, the detection result is output.

2. The new energy electrode plate defect detection method based on 3D laser point cloud image according to claim 1 is characterized in that: The pre-processing comprises: Calibrate the main plane of the 3D point cloud image, separate the depth information of the 3D point cloud image based on the Z axis, and form an RGB image and a depth image; The data set is subjected to invalid point removal, scanning vibration compensation and noise compensation.

3. The new energy electrode plate defect detection method based on 3D laser point cloud image according to claim 1 is characterized in that: The S3 includes: S31, extracting features from the RGB image and the depth image respectively to obtain RGB features and depth features; S32, RGB features and depth features are convolved to compress the number of channels. S33, through dilated convolution, the receptive field of the network is expanded; S34, restore the number of channels through convolution, and after batch normalization and linear correction unit activation function, add the residual edge to form a residual structure to obtain new RGB features and depth features; S35, the new RGB features and depth features are stacked and input into the attention module. Among them, ReLU represents the activation function, F R ' GB and F D ′ represents the features after convolution and CBS structure, F RGB and F D Represents the original features, represents the concatenation of two modal features along the channel dimension, x i Represents the original input value of the i-th sample on a certain feature dimension, μ B Represents the mean of the current mini-batch data, calculated by dimension, is the variance of the current mini-batch data, ε is a very small positive value, y i is the standardized value of the i-th sample; the new RGB features and depth features are fused to form the fusion feature F fusion . S36, normalize the fusion features based on the spatial attention mechanism; S37, based on the attention mechanism and the spatial attention mechanism, the standardized fusion features are superimposed and fused with the 3D point cloud image; S38, performing feature extraction on the superimposed fusion features and the depth image based on depth convolution and point-by-point convolution respectively; S39, the features extracted in S38 are fused through the SE attention mechanism to form the third fusion feature; S310, extracting features from the third fusion features, S311. The features extracted from the three features are fused by downsampling through the asymptotic feature pyramid network to form a defect detection model.

4. The new energy electrode plate defect detection method based on 3D laser point cloud image according to claim 1 is characterized in that: The defect detection model is output through repeated convolution modules.

5. The new energy electrode plate defect detection method based on 3D laser point cloud image according to claim 1 is characterized in that: The S5 includes: according to Among them, TP is the number of positive samples correctly identified, FP is the number of negative samples misclassified, FN is the number of true positive samples that are not correctly detected by the model, the F1 indicator represents the harmonic mean, and S is the total number of categories; the precision Precision, recall rate Recall, average precision AP achieved at different recall levels, and average precision mAP of different categories are calculated.

6. A new energy electrode plate defect detection system based on 3D laser point cloud images according to any one of claims 1 to 5, characterized in that: It includes bimodal feature fusion module, backbone module, neck module, detection head and evaluation module; The dual-modal feature fusion module includes an image fusion module and a space and channel attention module. The image fusion module is used to extract and fuse the image features for the first time. The space and channel attention module is used to standardize the image after the first fusion and superimpose and fuse the standardized image with the original image. The backbone module includes a ConvNeXtV2SE module and a channel-aware aggregation module. The ConvNeXtV2SE module includes an improved convolutional network and an attention module SE; the improved convolutional network performs deep feature extraction on the superimposed fused image; the attention module SE performs a second fusion on the features after the deep feature extraction, and the ConvNeXtV2SE module performs a third feature extraction on the image after the second fusion; The neck module includes an asymptotic feature pyramid module and a repeated convolution module. The asymptotic feature pyramid module is used to fuse the features from the three feature extractions; the repeated convolution module is used to increase the reusability of important features. The neck module includes a DBL module, a feature asymptotic feature pyramid module and a RepConv module. The DBL module is used to enhance the feature expression capability; the asymptotic feature pyramid module is used to fuse the enhanced features in the three feature extractions; and the repeated convolution module is used to increase the reusability of important features. The evaluation module is used to evaluate the accuracy and recall of the detection system.

Images