Self-iteration membrane electrode assembly defect detection method, device and equipment

Abstract

The invention provides a self-iteration membrane electrode assembly defect detection method, device and equipment. The method provided by the invention comprises the following steps: taking a pre-trained initial defect detection model as a current detection model, and performing defect positioning and defect type identification on a surface image of a to-be-detected membrane electrode assembly by utilizing the current detection model to obtain a detection result of the to-be-detected membrane electrode assembly; determining the detection precision of the current detection model according to the plurality of detection results; when the detection precision is smaller than a preset precision value, defect position labeling and defect type labeling are carried out on the stored surface images of the membrane electrode assembly to be detected, and the plurality of labeled surface images are determined as iterative training samples; and combining the iterative training sample and an initial training sample used for training the initial defect detection model, training the current detection model to obtain an iterative defect detection model, and carrying out defect detection. The self-iteration membrane electrode assembly defect detection method provided by the invention is used for improving the detection precision of defect detection.

Description

Technical Field

[0001] The present application relates to the technical field of defect detection, and in particular to a self-iterative membrane electrode assembly defect detection method, device and equipment. Background Art

[0002] In the hydrogen energy industry chain, the performance of membrane electrode assembly (MEA) as the core component of proton exchange membrane hydrogen fuel cells and all-vanadium flow batteries directly determines the reliability, stability and durability of the entire battery system. However, in the actual manufacturing process, typical defects such as wrinkles, dirt, and bubbles often occur in membrane electrode assemblies, which can seriously affect the performance of the battery and even cause the battery to fail or be scrapped. Therefore, efficient defect detection of membrane electrode assemblies has become an important link in ensuring battery quality.

[0003] At present, the defect detection of membrane electrode assemblies mainly adopts manual visual inspection method. Manual visual inspection method relies on the subjective experience and visual judgment of the inspectors, which not only has low detection efficiency, but also the detection results are greatly affected by human factors, making it difficult to ensure the accuracy of detection. Summary of the invention

[0004] In view of this, the present application provides a self-iterative membrane electrode assembly defect detection method, device and equipment to improve the detection accuracy of defect detection.

[0005] Specifically, the present application is implemented through the following technical solutions: In a first aspect, the present application provides a self-iterative membrane electrode assembly defect detection method, the method comprising: The pre-trained initial defect detection model is used as the current detection model, and the surface image of the membrane electrode assembly to be tested is used to locate defects and identify defect types by using the current detection model to obtain the detection result of the membrane electrode assembly to be tested; wherein, when the surface image of the membrane electrode assembly to be tested is used to locate defects and identify defect types by using the current detection model, the surface image of the membrane electrode assembly to be tested is saved; Determining the detection accuracy of the current detection model according to multiple detection results of multiple membrane electrode assemblies to be tested; When the detection accuracy is less than a preset accuracy value, the stored surface image of the membrane electrode assembly to be tested is annotated with defect positions and defect types, and the annotated plurality of surface images are determined as iterative training samples; Merging the iterative training samples with the initial training samples used to train the initial defect detection model to obtain a training sample set; Retraining the current detection model using the training sample set to obtain an iterative defect detection model; wherein, when training the current detection model and the initial defect detection model, the loss function for calculating the category loss is an improved SlideLoss loss function; the value of the IOU threshold in the weight function of the improved SlideLoss loss function in the current iteration round is related to the average value of the intersection-and-union ratio corresponding to the current iteration round and the average value of the intersection-and-union ratio corresponding to the historical iteration rounds before the current iteration round; The iterative defect detection model is used as a current detection model, and defect detection is performed using the current detection model.

[0006] A second aspect of the present application provides a self-iterative membrane electrode assembly defect detection device, the device comprising a detection module, a determination module, a labeling module, a merging module and a training module; wherein: The detection module is used to use the pre-trained initial defect detection model as the current detection model, and use the current detection model to locate defects and identify defect types on the surface image of the membrane electrode assembly to be tested, so as to obtain the detection result of the membrane electrode assembly to be tested; wherein, when the surface image of the membrane electrode assembly to be tested is located and the defect type is identified using the current detection model, the surface image of the membrane electrode assembly to be tested is saved; The determination module is used to determine the detection accuracy of the current detection model according to multiple detection results of multiple membrane electrode assemblies to be tested; The labeling module is used to label the defect positions and defect types of the stored surface images of the membrane electrode assembly to be tested when the detection accuracy is less than a preset accuracy value, and determine the labeled multiple surface images as iterative training samples; The merging module is used to merge the iterative training samples and the initial training samples used to train the initial defect detection model to obtain a training sample set; The training module is used to retrain the current detection model using the training sample set to obtain an iterative defect detection model; wherein, when training the current detection model and the initial defect detection model, the loss function for calculating the category loss is an improved SlideLoss loss function; the value of the IOU threshold in the weight function of the improved SlideLoss loss function in the current iteration round is related to the average value of the intersection-and-union ratio corresponding to the current iteration round and the average value of the intersection-and-union ratio corresponding to the historical iteration rounds before the current iteration round; The detection module is further configured to use the iterative defect detection model as a current detection model and perform defect detection using the current detection model.

[0007] The third aspect of the present application provides a self-iterative membrane electrode assembly defect detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the methods provided in the first aspect of the present application are implemented.

[0008] The self-iterative membrane electrode assembly defect detection method, device and equipment provided by the present application uses a pre-trained initial defect detection model as a current detection model, and uses the current detection model to locate defects and identify defect types on the surface image of the membrane electrode assembly to be tested, thereby obtaining a detection result of the membrane electrode assembly to be tested, and then determines the detection accuracy of the current detection model based on multiple detection results of multiple membrane electrode assemblies to be tested, and then, when the detection accuracy is less than a preset accuracy value, marks the defect location and defect type on the stored surface image of the membrane electrode assembly to be tested, and determines the multiple marked surface images as iterative training samples, and then merges the iterative training samples and the initial training samples used to train the initial defect detection model to obtain a training sample set. Then, the training sample set is used to retrain the current detection model to obtain the iterative defect detection model. Finally, the iterative defect detection model is used as the current detection model, and the current detection model is used to perform defect detection. In this way, on the one hand, when the detection accuracy is lower than the preset accuracy value, the model training data set is continuously improved, and the model is iteratively trained so that the model can be continuously improved and adapt to more complex situations in actual scenarios, thereby improving the accuracy and robustness of defect detection. On the other hand, the improved SlideLoss loss function is used to dynamically adjust the IOU threshold in the weight function according to the average value of the intersection-over-union ratio of the current and historical iteration rounds, so that the model can more accurately handle unbalanced samples and complex defect types, thereby further improving the classification and positioning performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flow chart of Embodiment 1 of the self-iterative membrane electrode assembly defect detection method provided by the present application; Figure 2 A schematic diagram of a defect detection process shown in an exemplary embodiment of the present application; Figure 3 An initial defect detection model is shown as an exemplary embodiment of the present application; Figure 4 A schematic diagram of a Bottleneck module shown as an exemplary embodiment of the present application; Figure 5 A schematic diagram of a dilated convolution process according to an exemplary embodiment of the present application; Figure 6 A flow chart of Embodiment 2 of the self-iterative membrane electrode assembly defect detection method provided in the present application; Figure 7A flowchart of Embodiment 3 of the self-iterative membrane electrode assembly defect detection method provided in the present application; Figure 8 A hardware structure diagram of a self-iterative membrane electrode assembly defect detection device in which the self-iterative membrane electrode assembly defect detection device of the present application is located; Fig. 9 This is a schematic structural diagram of Embodiment 1 of the self-iterative membrane electrode assembly defect detection device provided in the present application. DETAILED DESCRIPTION

[0010] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0011] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0012] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0013] Specific embodiments are given below to introduce the technical solution of the present application in detail.

[0014] Figure 1 This is a flow chart of the first embodiment of the self-iterative membrane electrode assembly defect detection method provided by the present application. Please refer to Figure 1 , the method provided in this embodiment may include: S101. Use a pre-trained initial defect detection model as a current detection model, and utilize the current detection model to locate defects and identify defect types on the surface image of the membrane electrode assembly to be tested, so as to obtain a detection result of the membrane electrode assembly to be tested; wherein, when utilizing the current detection model to locate defects and identify defect types on the surface image of the membrane electrode assembly to be tested, the surface image of the membrane electrode assembly to be tested is saved.

[0015] Specifically, an initial surface image of the membrane electrode assembly can be obtained; then the initial surface image is manually annotated and data preprocessed to obtain an initial training data set, and the initial defect detection model is trained using the initial training data set.

[0016] In specific implementation, the initial surface image of the membrane electrode assembly can be cropped into a standard image; then the standard image is manually annotated based on the defect detection task requirements to obtain defect information including defect type and defect location; and then the annotated standard image is randomly divided into a standard initial training set and a standard initial verification set in a ratio of 8:2, that is, 80% of the standard initial images to be detected are taken as the standard initial training set, and 20% are taken as the standard initial verification set; and then the standard initial training set and the standard initial verification set are data enhanced to obtain an initial training data set.

[0017] It should be noted that the closed-loop start threshold and closed-loop stop threshold can be set. When the model accuracy index is lower than or equal to the closed-loop start threshold, the iterative training data set is merged with the initial training data set, and the YOLOv11 model is input for iterative training to obtain the iterative defect detection model; when the model accuracy index is higher than or equal to the closed-loop stop threshold, the iterative training is stopped. In this way, a defect detection model with real-time update capability can be obtained to make up for the problem of insufficient generalization capability of the initial defect detection model. At the same time, setting the closed-loop start threshold and closed-loop stop threshold and combining iterative automatic annotation of the image to be detected can realize fully automatic self-iteration of the defect detection model. Therefore, as the number of iterations increases, the iterative defect detection model can fit all defect types and defect states encountered in actual production, and has stronger generalization capability and robustness.

[0018] It should be noted that, in order to further improve the display effect of the acquired initial surface image, the brightness and contrast of the initial surface image may be adjusted, and image denoising and image sharpening processing may also be performed to improve the display effect of defects in the image.

[0019] In specific implementation, image denoising uses the Gaussian filtering method, which can smooth the image and reduce the impact of environmental noise on image quality. Compared with mean filtering, Gaussian filtering can retain more image details. The formula of the Gaussian filter is as follows: ; Among them, x and y are the horizontal and vertical coordinates of the pixel in the image, indicating the distance between the pixel and the center point of the Gaussian filter kernel, σ is the standard deviation of the Gaussian distribution, which determines the strength of the noise removal ability, and exp represents the exponential function. In specific implementation, you can choose a 3*3 Gaussian filter kernel and set σ=1.

[0020] Furthermore, the Laplace operator is used for image sharpening to emphasize the mutation details (i.e., defect details) in the image. For a two-dimensional image figure(x, y), the Laplace operator is defined as: ; The image sharpening formula based on the sharpening Laplace operator is: ; Among them, output is the output image, figure is the original image, if the center value of the Laplace convolution kernel is negative, then c=-1, if the center value of the Laplace convolution kernel is positive, then c=1.

[0021] Furthermore, when manually annotating and preprocessing the data of the initial image to be inspected, since the defect detection task requires obtaining the type of defects in the image and the location of the defects, when manually annotating the initial image to be inspected, it is also necessary to annotate the type of defects and the location of the defects.

[0022] In specific implementation, each defect in each initial image to be detected should correspond to 5 annotation information: <object-class> <x> <y> <width> <height>,in, <object-class>Indicates the defect category, which can be an integer number starting from 0, indicating the 0th, 1st, 2nd, ..., defect. There is no order of defect categories here, and the present invention does not limit this; <x>and <y>They represent the x-coordinate of the center of the annotation box divided by the total width of the image (also called the relative x-coordinate of the center of the annotation box) and the y-coordinate of the center of the annotation box (also called the relative y-coordinate of the center of the annotation box) divided by the total height of the image. <width>and <height>They respectively represent the width of the annotation box divided by the total width of the image (also called the relative width of the annotation box) and the height of the annotation box divided by the total height of the image (also called the relative height of the annotation box). Here, the annotation box refers to the minimum enclosing rectangular box of the defect in the image.

[0023] In specific implementation, for example, in one embodiment, it is assumed that an initial image to be detected contains three defects, and these three defects belong to three categories respectively. Table 1 is an example of a labeling file after the image is manually labeled: Table 1 Examples of annotated files after manual annotation 0 0.53 0.23 0.11 0.98 1 0.25 0.43 0.26 0.27 2 0.92 0.78 0.47 0.71 Among them, 0, 1, and 2 in the first column represent the category numbers of the three defects, and the second to fifth columns represent the relative x-coordinates of the centers of the annotation boxes of the three defects, the relative y-coordinates of the centers of the annotation boxes, the relative widths of the annotation boxes, and the relative heights of the annotation boxes, respectively.

[0024] In specific implementation, for example, in one embodiment, data preprocessing may include the following operations: (1) cropping the initial surface image into a square standard image to facilitate the subsequent convolution operation when extracting image features; (2) randomly cropping the standard image to generate images with different perspectives and different field of view ranges, which can improve the robustness of the model; (3) randomly flipping the standard image to increase the number of samples in the data set, enhance the model training effect, and prevent the model from overfitting; (4) randomly rotating the standard image to simulate shooting perspectives at different angles, so that the model maintains detection capabilities when facing defect rotation; (5) randomly transforming the standard image to improve the robustness of the model under different lighting conditions and different color spaces; (6) performing geometric transformations on the standard image, such as scaling, translation, perspective transformation, etc., so that the model can understand spatial transformations and changes in object shapes.

[0025] Referring to the previous description, it can be understood that the pre-trained initial defect detection model is a defect detection model that can perform defect location and defect type identification, wherein defect location is the process of obtaining the defect position, and defect type identification is the process of determining the defect type of the defect. In other words, the input of the initial defect detection model is the surface image, and the output is the positioning frame where the defect is located and the defect category corresponding to the defect. By learning the defects of the membrane electrode assembly during the training stage, it can identify the defects of the membrane electrode assembly to be tested.

[0026] It should be noted that the membrane electrode assembly is a sandwich structure consisting of a proton exchange membrane, a catalyst layer and a gas diffusion layer, and is a key component in hydrogen fuel cells that converts chemical energy into electrical energy. In specific implementation, when it is necessary to perform defect detection on the membrane electrode assembly, the image of the membrane electrode assembly to be tested can be collected through sensor equipment such as a camera to obtain a surface image of the membrane electrode assembly to be tested.

[0027] Among them, in order to improve the defect detection effect of the surface image, a blue light source, an infrared light source, a yellow light source or a white light source can be selected according to the production site environment for back lighting, high-angle lighting or low-angle lighting to improve the clarity of the defect information in the surface image, which is convenient for subsequent feature extraction and defect detection. The embodiment of the present invention selects a white light source for high-angle lighting.

[0028] In specific implementation, when inspecting the membrane electrode assembly to be tested, the surface image of the membrane electrode assembly to be tested is input into the current detection model. The current detection model marks the position of the defect in the form of a positioning frame through defect positioning, and determines the type of the defect through defect type recognition.

[0029] Specifically, the positioning frame can be expressed as: <x> <y> <width> <height>,in, <x>and <y>They represent the horizontal coordinate of the center of the positioning frame divided by the total width of the image and the vertical coordinate of the center of the positioning frame divided by the total height of the image; <width>and <height>They represent the width of the positioning frame divided by the total width of the image and the height of the positioning frame divided by the total height of the surface image, respectively. In addition, the types of defects are wrinkles, dirt, and bubbles in bubbles.

[0030] It should be noted that the method provided in this embodiment saves the surface image of the membrane electrode assembly to be tested after the recognition is completed, so as to perform iterative training based on the saved surface image later.

[0031] S102: Determine the detection accuracy of the current detection model according to a plurality of detection results of a plurality of membrane electrode assemblies to be tested.

[0032] Specifically, detection accuracy is an indicator to measure the performance of the current detection model. In specific implementation, the precision rate P can be used as the detection accuracy. When the value of the precision rate P is greater than a preset value, it is determined that the current detection model meets the detection accuracy requirements. In addition, mAP50 or mAP95 can also be used as the detection accuracy. For the specific calculation method of mAP50 or mAP95, refer to the description in the relevant technology, which will not be repeated here.

[0033] S103, when the detection accuracy is less than a preset accuracy value, marking defect positions and defect types on the stored surface image of the membrane electrode assembly to be tested, and determining the marked multiple surface images as iterative training samples.

[0034] Specifically, the preset accuracy value is used to determine whether the performance of the current detection model meets the requirements. It is understandable that when the detection accuracy is less than the preset accuracy value, it is determined that the accuracy of the current detection model does not meet the requirements, and further measures need to be taken to improve the performance of the detection model.

[0035] It should be noted that the specific value of the preset accuracy value is set according to actual needs, and is not limited in this embodiment. In specific implementation, for example, in one embodiment, the preset accuracy value can be set to: the recognition accuracy rate is greater than 95%.

[0036] Furthermore, when the detection accuracy is less than a preset accuracy value, the defect positions and defect types are annotated on the stored surface images of the membrane electrode assembly to be tested, and the annotated multiple surface images are determined as iterative training samples.

[0037] Figure 2 This is a schematic diagram of a defect detection process according to an exemplary embodiment of the present application. Figure 2 Optionally, in a possible implementation, the defect position and defect type annotation of the stored surface image of the membrane electrode assembly to be tested include: (1) For each saved surface image, grayscale transformation processing, Gaussian filtering processing and edge extraction processing are performed on the surface image in sequence to obtain a mask label image corresponding to the surface image; wherein the foreground part in the mask label image represents the defect position.

[0038] In a specific implementation, the RGB values ​​of the surface image are converted into grayscale values ​​to perform grayscale transformation processing. Further, the purpose of removing noise and smoothing the image is achieved through Gaussian filtering. Then, through edge extraction processing, the areas where the grayscale values ​​in the image change significantly are detected. These areas usually correspond to the boundaries of the object or the locations of defects. For example, in a specific implementation, edge extraction can be performed based on the Sobel operator, which detects edges by calculating the gradients of the image in the horizontal and vertical directions.

[0039] Specifically, grayscale transformation is the process of converting a color image into a grayscale image, with the goal of simplifying image data, highlighting structural features, and reducing computational complexity. Common methods include weighted averaging, which is to perform a weighted summation of the RGB channels based on the sensitivity of the human eye to different colors.

[0040] In addition, Gaussian filtering is used to smooth the image and reduce the impact of noise on edge detection. It reduces high-frequency noise in the image while retaining the main edge information by convolving with the Gaussian kernel. In specific implementation, Gaussian filtering can be performed based on the following formula: ; Among them, x is the horizontal coordinate of each pixel in the image, y is the vertical coordinate of each pixel in the image, exp represents the exponential function, and σ is the standard deviation of the Gaussian distribution, which determines the strength of the noise removal ability.

[0041] Furthermore, after the surface image is denoised, the brightness and contrast of the surface image may be adjusted, and the image may be sharpened to improve the defect display effect in the image.

[0042] It should be noted that a binary image is obtained through edge detection. For the convenience of explanation, the image is recorded as a mask label image, in which the foreground part (pixel value is 1) represents the detected defect position, and the background part (pixel value is 0) represents the defect-free area.

[0043] Furthermore, the foreground part of the mask label image can be marked with a specific color, while the background part can be kept in a color different from the foreground, so as to distinguish defects.

[0044] (2) Determine the circumscribed rectangular frame of the foreground part in the mask label image, and annotate the circumscribed rectangular frame as an automatic annotation frame on the surface image to obtain a surface image with an automatic annotation frame.

[0045] Specifically, the bounding rectangular frame refers to the smallest rectangular area that can completely surround the foreground part, and the defects in the image are marked and located by the bounding rectangular frame.

[0046] In specific implementation, in the mask label image, the foreground part represents the position of the defect, and the minimum rectangular frame that can completely surround the foreground part is found through the contour detection function of the image processing library, and the minimum rectangular frame is determined as the circumscribed rectangular frame. For example, in one embodiment, the contour detection function is used to find the contours of all white areas, and finally the minimum rectangular frame that can surround each contour is found by calculation to obtain a surface image with an automatic annotation frame.

[0047] (3) Crop a portion of the image corresponding to the automatic annotation frame from the surface image with the automatic annotation frame, input the portion of the image into the current detection model, and obtain defect type information corresponding to the surface image through the current detection model.

[0048] Specifically, the image processing library OpenCV may be used to determine the coordinates of the upper left corner of the automatic annotation box and the size of the cropping area to complete the cropping operation.

[0049] Furthermore, the cropped partial image is used as input and input into the current detection model. The current detection model processes and analyzes the input partial image and outputs defect type information corresponding to the partial image.

[0050] (4) Determine the center coordinates, width, and height of the automatic annotation frame as defect location information, and determine the defect location information and the defect type information as annotation information of the surface image.

[0051] In the specific implementation, the center coordinates, width and height of the automatic annotation box are extracted as the location information of the defect. With the center coordinates as the center point, it is expanded according to the width and height. The obtained rectangle is the defect location information, and then combined with the defect type information to complete the annotation of the surface image.

[0052] For further information, please refer to Figure 2 ,After the defects are extracted, the defect location information of the defects is obtained so as to ,complete the annotation of the surface image in combination with the defect type information.

[0053] It should be noted that edge extraction can highlight the edge information in the image, help identify the outline of the defect, and thus more accurately locate the defect. The cropped portion of the image is then input into the current detection model. Through analysis and processing of the model, the defect type information corresponding to the surface image can be obtained. In this way, the defect location information is combined with the defect type information to form complete annotation information, which provides comprehensive support for subsequent image analysis and processing.

[0054] Specifically, it reduces the workload and time of manual labeling through automated processing, improves labeling efficiency, and can significantly reduce the cost of manual labeling. In addition, manual labeling may have subjective differences and errors, while automated labeling can ensure consistency and objectivity, maintain stable performance during long-term operation, and reduce inaccurate labeling due to human errors. In addition, through technologies such as grayscale transformation, Gaussian filtering, and edge extraction, the defect location can be more accurately extracted under different lighting, noise and other conditions, thereby improving the accuracy of labeling. Furthermore, after cutting out the defect area with the automatic labeling box and inputting it into the detection model, the specific type of the defect area (such as cracks, bubbles, corrosion, etc.) can be quickly obtained.

[0055] S104: Merge the iterative training samples and the initial training samples used to train the initial defect detection model to obtain a training sample set.

[0056] Specifically, the iterative training samples and the initial training samples are merged into a common data set to obtain a training sample set.

[0057] In the specific implementation, under the premise of considering the balance of samples, the data formats of the two sample sets are unified to ensure that the training sample sets can be correctly processed in the subsequent training process.

[0058] S105. Retrain the current detection model using the training sample set to obtain an iterative defect detection model; wherein, when training the current detection model and training the initial defect detection model, the loss function for calculating the category loss is an improved SlideLoss loss function; the value of the IOU threshold in the weight function of the improved SlideLoss loss function in the current iteration round is related to the average value of the intersection-and-union ratio corresponding to the current iteration round and the average value of the intersection-and-union ratio corresponding to the historical iteration rounds before the current iteration round.

[0059] It should be noted that when training the current detection model and training the initial defect detection model, the loss function for calculating the category loss is the improved SlideLoss loss function; the value of the IOU threshold in the weight function of the improved SlideLoss loss function in the current iteration round is related to the average value of the intersection-and-union ratio corresponding to the current iteration round and the average value of the intersection-and-union ratio corresponding to the historical iteration rounds before the current iteration round.

[0060] Specifically, the SlideLoss loss function is a loss function designed to solve the problem of sample imbalance. In detection tasks, the detection model may be more inclined to learn samples that are easy to classify and ignore samples that are difficult to classify. The SlideLoss loss function gives higher weights to difficult samples, making the model pay more attention to these samples during training, thereby improving the overall detection performance.

[0061] For example, in one embodiment, the SlideLoss loss function is as follows: SlideLoss = ; in, is the weight related to the sample difficulty, is the basic loss for each sample. It should be noted that It can be calculated by the intersection-and-union ratio of the samples. For example, in one embodiment, a lower weight is set for samples with a high intersection-and-union ratio. , set higher weights for samples with low intersection-over-union ratio .

[0062] In the traditional SlideLoss loss function, the weight function is usually fixed and cannot adapt to the needs of different training stages. To this end, in this application, the improved SlideLoss loss function introduces a dynamic adjustment mechanism so that the IoU threshold is adjusted as the training process progresses. Specifically, in each iteration round, the IoU threshold is updated according to the average value of the intersection-over-union ratio corresponding to the current iteration and the average value of the intersection-over-union ratio corresponding to the historical iteration rounds. This dynamic adjustment method enables the model to more effectively balance the impact of easy-to-classify samples and difficult-to-classify samples, thereby improving detection performance.

[0063] The improved SlideLoss loss function dynamically adjusts the IoU threshold, enabling the detection model to focus on difficult samples more effectively during training, thereby improving the overall performance of defect detection.

[0064] Furthermore, the weight function of the improved SlideLoss loss function is as follows: ; in, is the weight of the i-th training sample; is the intersection-and-union ratio between the predicted box and the true box of the i-th training sample; is the IOU threshold.

[0065] It should be noted that through the improved SlideLoss loss function, easy-to-classify samples and difficult-to-classify samples can be distinguished according to the intersection-over-union ratio between the predicted box and the true box (samples with a higher degree of match are easy-to-classify samples, and samples with a lower degree of match are difficult-to-classify samples).

[0066] Furthermore, the improved SlideLoss loss function uses a weighting function to give higher weights to difficult-to-classify samples to solve the loss imbalance problem between difficult and easy samples and increase the model's attention to difficult-to-classify samples.

[0067] It should be noted that in each round of training, the process of determining the IOU threshold of this round includes: (1) Determine the decay rate of the current round according to the first formula based on the preset decay rate threshold, the round value of the current round, and the preset time decay constant; (2) calculating the reference threshold of this round according to the second formula based on the decay rate, the reference threshold of the previous round, and the average value of the intersection-over-union ratio of all training samples in this round; wherein the initial value of the reference threshold is a preset value; (3) Determine the reference threshold of this round as the IOU threshold of this round; Among them, the first formula is: ; The second formula is: ; The decay is the decay rate, is the preset decay rate threshold, d is the round value of the current round, τ is the preset time decay constant, A1 is the reference threshold of this round, is the reference threshold of the previous round, is the average of the intersection-over-union ratios of all training samples in this round.

[0068] It can be understood that the IOU threshold in each round is related to the value in the previous round. Referring to the first formula, it can be understood that as the round d increases, decay gradually approaches the preset decay rate threshold, that is, as the round d increases, the update trend of decay is from 0 to the preset decay rate threshold. Further, referring to the second formula, as the round d increases, the update trend of decay is from 0 to ,Right now The weight coefficient gradually increases from 0 to ,Right now The weight coefficient before gradually increases. In other words, as the round d increases, the weight coefficient of the average value of the intersection-over-union ratio of all training samples in this round gradually decreases. The weight coefficient of the reference threshold in the previous round gradually increases. In this way, the IoU threshold can change smoothly while improving the model's attention to difficult-to-classify samples, reducing the impact of noise information and making the loss calculation more stable.

[0069] It is understandable that as the round d increases, the weight coefficient of the reference threshold of the previous round is gradually increased, which is equivalent to relying more on historical information. This method effectively allows the model to pay more attention to difficult-to-classify samples, avoiding ignoring these samples too early, thereby improving the model's generalization ability for difficult-to-classify samples. In addition, in the initial stage, the model may be more sensitive to the IoU threshold. At this time, using the average value of the intersection-over-union ratio of the current round of training samples can help quickly adapt to the data distribution. However, as the training deepens, gradually increasing the weight of the average value of the intersection-over-union ratio corresponding to the historical iteration rounds can suppress the impact of the current round of data noise and smooth the changes in the IoU threshold. Furthermore, the smooth IOU threshold update can reduce the gradient instability caused by abnormal samples or fluctuations, thereby making the loss calculation more robust and promoting model convergence.

[0070] In addition, through the above-mentioned improvement of SlideLoss loss function, the learning focus of the model will be different at different stages of training: Early stage: more dependent on the intersection-and-union ratio of the current round to quickly capture the distribution characteristics of the data.

[0071] Later stage: more reliance on the intersection-over-union ratio of historical rounds makes the change of IoU threshold more stable, thereby reducing the risk of overfitting.

[0072] This dynamic adjustment mechanism can naturally adapt to the needs of the model at different training stages and avoid the rigidity problem of fixed IOU thresholds.

[0073] In addition, dynamically adjusting the IoU threshold can effectively alleviate the problem of gradient explosion or gradient vanishing. Especially in the later stages of training, the smooth change of the IoU threshold can prevent the model from undergoing drastic parameter updates due to extreme behavior of local samples, thus improving the stability of the training process.

[0074] Furthermore, improving the dynamic adjustment of the IoU threshold during the update process of the SlideLoss loss function means more accurate modeling of the loss distribution. This mechanism can effectively reduce the dominant effect of noise samples on the loss, making the model training more robust and thus performing better in real scenarios.

[0075] It should be noted that the traditional SlideLoss loss function uniformly handles the error between the predicted box and the true box of each training sample when processing the target detection task. The contribution of easy-to-classify samples and difficult-to-classify samples to the loss is the same, which causes the model to pay too much attention to easy-to-classify samples and ignore difficult-to-classify samples, thereby affecting the model's generalization ability and adaptability to complex scenes. The improved SlideLoss loss function introduces a weight function, which dynamically adjusts the weight of each training sample according to the intersection-and-union ratio between the predicted box and the true box. When the IOU of the predicted box and the true box is low (i.e., difficult-to-classify samples), the weight function will give these samples a higher weight, thereby increasing their contribution in the loss function and making the model pay more attention to these difficult-to-classify samples. On the contrary, when the IOU of the predicted box and the true box is high (i.e., easy-to-classify samples), the weight function will give these samples a lower weight, reducing their contribution in the loss function.

[0076] In addition, the improved SlideLoss loss function also adopts a method of dynamically adjusting the IOU threshold, which can gradually adjust the IOU threshold as the training progresses, so that the model can better adapt to the training needs of different stages. In this way, by using the improved SlideLoss loss function, on the one hand, the model's attention to difficult-to-classify samples is increased, thereby improving the model's adaptability to complex scenes. On the other hand, by dynamically adjusting the IOU threshold, the threshold can be automatically adjusted according to different stages of training, so that the model can gradually improve its prediction accuracy during the training process. Since the improved SlideLoss loss function can better handle the imbalance problem between easy-to-classify samples and difficult-to-classify samples, it can improve the overall performance of the model.

[0077] The dynamic IoU threshold adjustment mechanism integrates the information of the current round and the historical round, and has significant advantages in enhancing the focus on difficult-to-classify samples, smoothing threshold changes, dynamically adapting to the needs of the training phase, and improving the robustness of the loss function. This mechanism can not only improve the training efficiency of the model, but also ensure the stability of the loss calculation and parameter update process while improving the model performance, so as to better adapt to the needs of complex tasks.

[0078] Further, in an optional implementation, when training the current detection model and training the initial defect detection model, the second loss function for calculating the positioning loss is: ; ; ; Among them, the is the second loss function; the (x1, y1) is the center coordinate of the prediction box; the ( ) is the center coordinate of the real frame, is the intersection-and-union ratio between the auxiliary bounding box of the prediction box and the auxiliary bounding box of the real box; wherein the auxiliary bounding box of the prediction box is a bounding box centered on the center of the prediction box and smaller than the size of the prediction box, and the auxiliary bounding box of the real box is a bounding box centered on the center of the real box and smaller than the size of the real box; the sizes of the auxiliary bounding boxes of the prediction box and the auxiliary bounding boxes of the real box are negatively correlated with the intersection-and-union ratio of the prediction box and the real box.

[0079] Specifically, Focusing on the shape and size of the bounding box itself to calculate the loss makes the bounding box regression more accurate; Further reduce the competitiveness of easy-to-classify samples and further increase the model's attention to difficult-to-classify samples.

[0080] It should be noted that in ,use Instead of the intersection-over-union ratio between the predicted box and the true box, the intersection-over-union ratio between the auxiliary bounding box of the predicted box and the auxiliary bounding box of the true box is used to replace the intersection-over-union ratio between the predicted box and the true box. In this way, different samples can be distinguished during the regression process, and auxiliary boxes of different scales are used to calculate the loss for different samples, which can speed up the regression process.

[0081] It should be noted that the auxiliary bounding box of the predicted box is a bounding box centered on the center of the predicted box and smaller than the size of the predicted box, and the auxiliary bounding box of the real box is a bounding box centered on the center of the real box and smaller than the size of the real box.

[0082] Furthermore, the sizes of the auxiliary bounding boxes of the predicted box and the auxiliary bounding boxes of the real box are negatively correlated with the intersection-over-union ratio of the predicted box and the real box. That is, the larger the intersection-over-union ratio of the predicted box and the real box, the smaller the auxiliary bounding boxes of the predicted box and the real box, and the smaller the intersection-over-union ratio of the predicted box and the real box, the larger the auxiliary bounding boxes of the predicted box and the real box.

[0083] Specifically, the specific method for determining the bounding box of the prediction box and the auxiliary bounding box of the real box is set according to actual needs and will not be repeated here.

[0084] It should be noted that By focusing on the shape and size of the bounding box instead of the traditional IoU, the optimization process of the bounding box is further refined, making the regression more accurate. In addition, the size of the auxiliary bounding box of the predicted box and the true box is negatively correlated with their intersection-over-union ratio (the larger the IoU, the smaller the auxiliary box; the smaller the IoU, the larger the auxiliary box), so that different samples can be optimized for regression loss at a reasonable scale. The design of the auxiliary bounding box simplifies the model's error optimization for difficult-to-classify samples, thereby improving the convergence speed. Furthermore, The exponential function is used to highlight samples with large deviations between the predicted box and the center of the true box, thereby increasing the model's attention to these difficult-to-classify samples.

[0085] Referring to the above description, it can be understood that Will and The combination of the two parts forms a unified loss function. This combination can give full play to the advantages of both, complement each other, and achieve better performance of the model in the target detection task. The following are the specific benefits of combining the two: 1. Dynamic weighting combined with fine regression to improve the optimization effect of difficult-to-classify samples: The offset between the center point of the predicted box and the true box is converted into a loss weight through an exponential function, and the contribution of each sample to the total loss is dynamically adjusted. In this way, difficult-to-classify samples with large offsets have large weights, making the model pay more attention to these samples.

[0086] The size and shape of the bounding box are regressed finely through the IoU and shape error of the auxiliary box. In this way, even if the sample offset is large, the model can gradually approach the real box through more reasonable shape optimization.

[0087] After the combination, difficult-to-classify samples can not only receive higher weighted attention, but also be more finely optimized in the shape and size of the bounding box, thereby improving the overall model performance.

[0088] 2. Distinguish the importance of samples and accelerate convergence: RWise-IoU reduces the weight of easy-to-classify samples, thereby reducing the impact of these samples on the loss, while During the optimization process, the size of the auxiliary box is dynamically adjusted for samples with different intersection-over-union ratios, making the gradient information of difficult-to-classify samples more effective. In this way, the model prioritizes learning difficult-to-classify samples and avoids paying too much attention to easy-to-classify samples, thereby accelerating convergence.

[0089] 3. Comprehensive center offset and shape error to improve prediction accuracy: RWise-IoU focuses on the center point offset between the predicted box and the real box, which can significantly reduce the positioning error of large offset samples. The auxiliary bounding box is used to further focus on the size and shape consistency of the predicted box and the true box. In this way, after combining the two, the model can accurately adjust the size and shape of the bounding box while optimizing the center point, thereby improving the accuracy of the overall prediction.

[0090] 4. Good adaptability to targets of different scales: After combining the two, the size of the auxiliary box is negatively correlated with the intersection-over-union ratio of the real box. For small targets, the intersection-over-union ratio is usually small, and the auxiliary box is larger, which helps to improve its regression effect; while for large targets, the intersection-over-union ratio is high, the auxiliary box is smaller, and more attention is paid to detail optimization. In this way, whether it is a small target or a large target, the combined loss function can handle it reasonably and has good adaptability.

[0091] 5. Reduce competition between samples and improve model robustness: Using the IoU of the auxiliary box instead of the traditional IoU allows the model to avoid directly comparing the IoU of the full box during the regression process, reducing the competitiveness between samples. Wise-IoU The emphasis on weight allocation for difficult-to-classify samples also further reduces the interference of easy-to-classify samples. In this way, the model is more robust and has stronger adaptability to complex scenes in target detection (such as occlusion, dense targets, etc.).

[0092] S106: Use the iterative defect detection model as a current detection model, and perform defect detection using the current detection model.

[0093] In a specific implementation, after an iterative defect detection model having a detection accuracy greater than or equal to a preset accuracy value is obtained using a training sample set, defect detection is performed using the iterative defect detection model.

[0094] The self-iterative membrane electrode assembly defect detection method provided in the present embodiment uses a pre-trained initial defect detection model as the current detection model, and uses the current detection model to locate defects and identify defect types on the surface image of the membrane electrode assembly to be tested, thereby obtaining the detection result of the membrane electrode assembly to be tested, and then determines the detection accuracy of the current detection model according to multiple detection results of multiple membrane electrode assemblies to be tested, and then when the detection accuracy is less than a preset accuracy value, marks the defect location and defect type on the stored surface image of the membrane electrode assembly to be tested, and determines the multiple marked surface images as iterative training samples, and then merges the iterative training samples and the initial training samples used to train the initial defect detection model to obtain a training sample set, and then The current detection model is retrained using the training sample set to obtain an iterative defect detection model. Finally, the iterative defect detection model is used as the current detection model, and the current detection model is used for defect detection. In this way, on the one hand, when the detection accuracy is lower than the preset accuracy value, the model training data set is continuously improved, and the model is iteratively trained so that the model can be continuously improved and adapt to more complex situations in actual scenarios, thereby improving the accuracy and robustness of defect detection. On the other hand, the improved SlideLoss loss function is used to dynamically adjust the IOU threshold in the weight function according to the average value of the intersection-over-union ratio of the current and historical iteration rounds, so that the model can more accurately handle unbalanced samples and complex defect types, thereby further improving the classification and positioning performance.

[0095] Optionally, in a possible implementation, Figure 3 This is an initial defect detection model shown in an exemplary embodiment of the present application. Figure 3 In a possible implementation, the initial defect detection model is a detection model based on the YOLO model; the backbone network in the initial defect detection model is a CSPNet backbone network that introduces a C3K2 module; the SPPF module in the CSPNet backbone network is an SPPF module that introduces a multi-head attention mechanism; the Bottleneck module in the C3K2 module includes a first convolution module, a second convolution module, a third convolution module and a fusion module; The first convolution module is used to perform convolution processing, normalization processing and activation processing on the input feature map in sequence to obtain a first intermediate feature map; The second convolution module includes a plurality of dilated convolution modules and a normalization processing module arranged behind each dilated convolution module; each dilated convolution module is used to perform a dilated convolution operation on the first intermediate feature map according to the dilated rate and convolution kernel corresponding to the dilated convolution module to obtain a dilated convolution result; the normalization processing module behind each dilated convolution module is used to normalize the initial dilated convolution result to obtain a normalized dilated convolution result; The third convolution module is used to fuse multiple normalized hole convolution results and output a second intermediate feature map; The fusion module is used to fuse the input feature map and the second intermediate feature map to obtain a fused feature map.

[0096] Specifically, the initial defect detection model is a detection model based on the YOLO model; the specific structure of the traditional YOLOv11 model is described in the related art, which will not be repeated here. In this embodiment, based on the traditional YOLOv11 model, the backbone network is a CSPNet backbone network that introduces the C3K2 module.

[0097] In the specific implementation, the transfer learning method can be used. First, the weight parameters of the YOLOv11 pre-trained model officially released by ultralytics are imported, and then the initial training data set is input to adjust the weight parameters of the pre-trained model to obtain an initial defect detection model suitable for membrane electrode assembly defect detection tasks. This can improve the convergence speed of the model and greatly reduce the time cost and computing power cost required for training.

[0098] See also Figure 3 As described in , the C3K2 module in the CSPNet backbone network includes a first type of C3K2 module (C3k=False) and a second type of C3K2 module (C3k=True). For the internal structure of the first type of C3K2 module and the second type of C3K2 module, please refer to the description in the related art, which will not be repeated here. Further, it can be understood that a Bottleneck module is provided in the first type of C3K2 module and the second type of C3K2 module. In this embodiment, the Bottleneck module is improved. After the improvement, the Bottleneck module includes a first convolution module, a second convolution module, a third convolution module and a fusion module.

[0099] In addition, a SPPF module is also provided in the CSPNet backbone network. In this embodiment, the SPPF module is also improved. After the improvement, the SPPF module is an SPPF module that introduces a multi-head attention mechanism.

[0100] The following is a detailed introduction to the Bottleneck module. Specifically, Figure 4 This is a schematic diagram of a Bottleneck module according to an exemplary embodiment of the present application. Figure 4 , the Bottleneck module includes a first convolution module, a second convolution module, a third convolution module and a fusion module. In specific implementation, the input feature map is input into the Bottleneck module, and then the first convolution module performs preliminary convolution processing on the input feature map to extract low-level features and obtain the first intermediate feature map. Then, the second convolution module further extracts the features of the first intermediate feature map to increase the complexity and abstraction of the features and obtain multiple normalized hole convolution results. Then, the third convolution module (the first convolution module can implement specific functions through a 1*1 convolution kernel) is used to fuse multiple normalized hole convolution results to obtain the second intermediate feature map. Finally, the fusion module fuses the initial input feature map and the second intermediate feature map, and outputs the processed fused feature map.

[0101] It is understandable that the second convolution module is provided with a plurality of dilated convolution modules, and the number of dilated convolution modules is set according to actual needs. Furthermore, each dilated convolution module can have convolutions of different sizes and dilation rates of different sizes. In addition, each dilated convolution module performs dilated convolution operations based on its own dilation rate and convolution kernel.

[0102] Figure 5 This is a schematic diagram of a second convolution module shown in an exemplary embodiment of the present application. Figure 5 ,exist Figure 5 In the example shown, the second convolution module includes 5 dilated convolution modules. For the convenience of explanation, the 5 dilated convolution modules are sequentially recorded as the first dilated convolution module, the second dilated convolution module, the third dilated convolution module, the fourth dilated convolution module and the fifth dilated convolution module from left to right. The convolution kernel size of the first dilated convolution module is 9*9, and the dilation rate is 1. The convolution kernel size and dilation rate of other dilated convolution modules are as follows: Figure 5 As shown in , no further description is given here.

[0103] It should be noted that in the dilated convolution operation, the elements between the convolution kernels are inserted into the gaps by the dilation ratio to perform the convolution calculation. The dilation ratio specifies the size of the gap between the convolution kernel elements. The effective receptive field size of the convolution kernel is calculated according to the following formula: Keffective = ; K: the original size of the convolution kernel (e.g. 9*9); d: void ratio.

[0104] Furthermore, the dilated convolution increases the receptive field by introducing a dilation rate, and the dilated convolution does not downsample the feature map while expanding the receptive field, thereby retaining the resolution and ensuring that subtle defect features are not ignored. In addition, multiple dilated convolution modules can extract features of different scales by setting different dilation rates, and have good adaptability to diverse defect forms. Furthermore, the combination of different dilation rates can effectively fuse local information and global information, and improve the ability to recognize complex defects.

[0105] The self-iterative membrane electrode assembly defect detection method provided in this embodiment sets the Bottleneck module to be composed of a first convolution module, a second convolution module, a third convolution module and a fusion module. In this way, by introducing multiple hole convolution modules in the second convolution module, it is possible to not only expand the receptive field, enhance the multi-scale feature extraction capability, and improve the detail retention capability, but also further, by fusing the first intermediate feature map and the second intermediate feature map, it is possible to enhance the feature expression capability, effectively enhance the model's recognition performance for complex, multi-scale defects, and optimize the computational cost.

[0106] Figure 6 This is a flow chart of Embodiment 2 of the self-iterative membrane electrode assembly defect detection method provided by this application. Please refer to Figure 6 Based on the above embodiment, the method provided in this embodiment further includes, before training the initial defect detection model: S601. Construct multiple loss function schemes; wherein each loss function scheme includes a loss function for calculating category loss and a loss function for calculating positioning loss.

[0107] Specifically, the loss function scheme is a combination of loss functions designed for the fault detection task, which is used to guide the training process of the model so that its output is closer to the true value.

[0108] Furthermore, the specific types and specific numbers of loss functions included in the loss function scheme are set according to actual needs, and are not limited in this embodiment. In specific implementation, the loss function scheme may include a loss function for calculating the category loss for the fault category, and a loss function for calculating the positioning loss for the fault location.

[0109] S602: Obtain a public data set related to membrane electrode surface defects, and divide the public data set into a training set and a test set.

[0110] Specifically, the public dataset contains a large amount of relevant data. In other words, a large number of surface images with automatically annotated defects can be obtained through the public dataset.

[0111] In specific implementation, the dataset GC10-DET can be used as a public dataset, and the public dataset GC10-DET can be divided into a training set and a test set according to a preset ratio. Among them, the training set accounts for a larger proportion (the training set can account for 70%-80%) and is used for model training; the test set accounts for a smaller proportion (the test set can account for 20%-30%) and is used for model evaluation and testing.

[0112] S603 . For each loss function scheme, use the loss function scheme as the loss function of the initial defect detection model, train the initial defect detection model using the training set, and obtain an intermediate model corresponding to the loss function scheme.

[0113] In specific implementation, the function for calculating the category loss in a loss function scheme is the SlideLoss loss function combined with the EMA (ExponentialMoving Average, EMA for short), and the loss for calculating the positioning is the ShapeIoU loss function combined with the Wise-IoU and Inner-IoU ideas, and different loss function schemes are used as the loss function of the initial defect detection model. For example, in one embodiment, Table 2 is a table of loss function schemes for different category losses shown in an exemplary embodiment of the present application; Table 3 is a table of loss function schemes for different positioning losses shown in an exemplary embodiment of the present application; multiple loss function schemes can be determined through Tables 2 and 3: Table 2 Loss function schemes for different types of losses Localization loss function Class loss function P Precision R recall mAP50 mAP50-95 CIo BCE 0.68 0.603 0.639 0.337 CIo FocalLoss 0.514 0.562 0.585 0.297 CIo VariFocalLoss 0.636 0.559 0.562 0.281 CIo EMA-SlideLoss 0.679 0.624 0.673 0.353 Table 3 Loss function schemes for different positioning losses Localization loss function Class loss function P Precision R recall mAP50 mAP50-95 Inner-CIoU EMA-SlideLoss 0.664 0.653 0.671 0.333 MDPIoU EMA-SlideLoss 0.766 0.629 0.672 0.341 Inner-MDPIoU EMA-SlideLoss 0.638 0.642 0.662 0.332 Focaler-CIoU EMA-SlideLoss 0.650 0.623 0.637 0.332 Focaler-MDPIoU EMA-SlideLoss 0.735 0.622 0.661 0.341 Wise-MDPIoU EMA-SlideLoss 0.689 0.649 0.682 0.349 Wise-ShapeIoU EMA-SlideLoss 0.717 0.624 0.672 0.337 Wise-Inner-ShapeIoU EMA-SlideLoss 0.733 0.627 0.685 0.350 Wise-Inner-MDPIoU EMA-SlideLoss 0.775 0.613 0.671 0.341 Among them, P (Precision, referred to as P) is the precision, R (Recall, referred to as R) is the recall rate, mAP (Mean Average Precision, referred to as mAP) is the average precision over multiple categories, mAP50 represents the mAP value with an IoU threshold of 50%, and mAP50-95 represents the average of the mAP values ​​with an IoU threshold in the range of 50-95%.

[0114] S604: Determine multiple accuracy indicators of the intermediate models corresponding to each loss function scheme based on the test set.

[0115] Specifically, the specific type of the accuracy indicator is determined according to actual needs, and is not limited in this embodiment.

[0116] In specific implementation, the accuracy index may include precision, recall, average precision on multiple defect categories, etc. For example, in one embodiment, the precision P may be calculated based on the following formula: ; Among them, P is the precision rate, which represents the proportion of samples that are actually positive among all samples predicted by the model as positive examples; TP is the true positive example, that is, the number of samples correctly predicted by the model as positive; FP is the false positive example, that is, the number of samples incorrectly predicted by the model as positive.

[0117] Furthermore, the recall rate R can be calculated based on the following formula: ; Among them, R is the recall rate, which represents the proportion of samples predicted by the model as positive among all samples that are actually positive, TP is the true positive sample, and FN is the false negative sample, that is, the number of samples that the model incorrectly predicts as negative.

[0118] Furthermore, the average precision over multiple categories can be calculated based on the following formula: ; Among them, mAP is the average precision, AP k is the average precision of the kth category, and n is the number of categories.

[0119] S605. According to the multiple accuracy indicators, determine an optimal loss function scheme from the multiple loss function schemes as the loss function of the initial defect detection model.

[0120] In specific implementation, the optimal loss function schemes of multiple loss schemes are calculated according to the values ​​of multiple precision indicators to obtain the loss function of the initial defect detection model. For example, in one embodiment, according to the values ​​of multiple precision indicators, the accuracy of loss function scheme 1 is determined to be 80%, the accuracy of loss function scheme 2 is 85%, and the accuracy of loss function scheme 3 is 90%. At this time, loss function scheme 3 is determined as the loss function of the initial defect detection model.

[0121] It is understandable that by selecting the optimal loss function solution, different loss functions will guide the model to learn different feature representations during the training process, and the optimal loss function can make the model better fit the distribution of real data, thereby improving the detection accuracy of the model. Selecting the optimal loss function solution can also enhance the generalization ability of the model. By comparing the performance of different loss function solutions on accuracy indicators, selecting the optimal loss function solution can also provide explainability for the model's decision.

[0122] Figure 7 This is a flow chart of Embodiment 3 of the self-iterative membrane electrode assembly defect detection method provided by this application. Please refer to Figure 7 Based on the above embodiment, the method provided in this embodiment, before the step of retraining the current detection model using the training sample set, further includes: S701: Classify the training sample set according to defect type, and count the number of samples included in each defect type.

[0123] In a specific implementation, for example, in one embodiment, the training sample set is classified to obtain 1000 training samples with the defect type of wrinkles, 1500 training samples with the defect type of dirt, and 2000 training samples with the defect type of bubbles.

[0124] S702: Determine whether there is any imbalanced distribution of categories in the training sample set according to the number of samples included in each defect type.

[0125] In specific implementation, the numerical range of the number of samples included in each defect type can be determined based on historical training data, and the training sample set that does not meet the numerical range is determined as a case of unbalanced category distribution.

[0126] For example, in one embodiment, the maximum difference in the number of samples of each defect type is determined to be 1000 based on historical training data. When the difference between the training sample with the largest number of samples and the training sample with the smallest number of samples exceeds 1000, it is determined that there is an imbalance in the category distribution.

[0127] For another example, in another embodiment, the lower limit of the number of samples is determined to be 2000 based on historical training data. When there are samples with a number less than 2000, it is determined that there is an imbalance in category distribution.

[0128] S703. When there is an imbalance in the category distribution of the training sample set, for the target defect type, the number of samples included in the defect category is expanded based on data enhancement technology or resampling technology to avoid the imbalance in category distribution; wherein, when the difference between the ratio of the number of samples included in a class of defect types to the total number of samples included in the training sample set and the standard value is greater than a preset threshold, the defect category is the target defect type; the standard value is the ratio of the number of samples included in the training set to the number of categories of the defect type included in the training set.

[0129] Specifically, the target defect type is the defect type corresponding to the training sample that causes the class distribution imbalance. For example, in one embodiment, when the bubble class training sample causes the class distribution imbalance, the target defect type is the bubble class training sample.

[0130] Furthermore, data augmentation technology generates more training samples by transforming or expanding existing data to improve the generalization ability and performance of the model; resampling technology regenerates training sample sets from the original data set through sampling with or without replacement to solve the problem of uneven category distribution.

[0131] Furthermore, the standard value is used to determine whether a defect type is a target defect type. For example, in one embodiment, the number of samples included in the training set is 3000, and the number of defect types included in the training set is 6. In this case, the standard value is 500.

[0132] Furthermore, the specific value of the preset threshold is set according to actual needs, and is not limited in this embodiment. In specific implementation, half of the standard value can be determined as the preset threshold. For example, in one embodiment, the standard value is 500 and the preset threshold is 250.

[0133] In specific implementation, when the number of samples of a training sample is less than the standard value and exceeds the preset threshold, the number of samples included in the defect category is expanded based on data enhancement technology or resampling technology. For example, in one embodiment, combined with the above embodiment, when the number of samples corresponding to defect C is 200, which is 300 less than the standard value and exceeds the preset threshold of 250, the training samples corresponding to defect C are expanded.

[0134] The self-iterative membrane electrode assembly defect detection method provided in this embodiment determines whether there is an imbalance in the category distribution of the training sample set based on the number of samples contained in each defect type. By expanding the defect types with a smaller number of samples, the category distribution of the training sample set is made more balanced, which helps the model learn the characteristics of various defect types. In this way, the problem of imbalance in the category distribution of the training sample set is effectively solved, and the accuracy and robustness of the model are improved.

[0135] S704: Retrain the current detection model using the training sample set to obtain an iterative defect detection model.

[0136] Among them, when training the current detection model and training the initial defect detection model, the loss function for calculating the category loss is the improved SlideLoss loss function; the value of the IOU threshold in the weight function of the improved SlideLoss loss function in the current iteration round is related to the average value of the intersection-and-union ratio corresponding to the current iteration round and the average value of the intersection-and-union ratio corresponding to the historical iteration rounds before the current iteration round.

[0137] Specifically, the specific implementation process of this step can be found in the description of the above embodiment, which will not be repeated here.

[0138] S705: Use the iterative defect detection model as a current detection model, and perform defect detection using the current detection model.

[0139] Specifically, the specific implementation process of this step can be found in the description of the above embodiment, which will not be repeated here.

[0140] The self-iterative membrane electrode assembly defect detection method provided in this embodiment determines that the model needs to be iteratively updated when the detection accuracy of the model is less than the preset accuracy value. At this time, the iterative training samples and the initial training samples are merged, and the model is retrained using this richer training sample to obtain an iterative defect detection model that meets the requirements. At the same time, when training the iterative defect detection model, the improved SlideLoss loss function is used to calculate the category loss. By dynamically adjusting the IOU threshold in the weight function, the model pays more attention to difficult-to-classify samples, thereby improving the classification accuracy of the model. At the same time, the value of the IOU threshold is related to the average value of the intersection-and-union ratio of the current iteration round and the historical iteration round, which helps the model better adapt to changes in data distribution and stabilize the training process. In this way, the reliability and accuracy of membrane electrode assembly defect detection can be guaranteed through the self-iterative detection model.

[0141] Optionally, in a possible implementation, when training the current model and training the initial defect detection model, for each training sample, the training sample is first processed to obtain a processed training sample; wherein the first processing includes at least any one of the following processing: brightness adjustment, contrast adjustment, image denoising processing and image sharpening processing; The processed training samples are cut into standard sizes to obtain standard training samples; The standard training samples are preprocessed to obtain final training samples; wherein the preprocessing includes at least any one of the following processes: random cropping, random flipping transformation, random rotation, random color transformation, and geometric transformation.

[0142] Specifically, the specific processing process included in the first processing is determined according to actual needs, and is not limited in this embodiment. For example, in one embodiment, the first processing of the training sample includes image denoising and image sharpening. For another example, in another embodiment, the first processing of the training sample includes brightness adjustment, contrast adjustment, image denoising and image sharpening.

[0143] Furthermore, the Laplace operator can be used to sharpen the image. In specific implementation, the Laplace operator is defined by the following formula: ; in, is the Laplace operator, figure is the original image, x is the horizontal coordinate of each pixel in the image, y is the vertical coordinate of each pixel in the image, yes.

[0144] Image sharpening is done based on the following formula: ; Among them, output is the output image, figure is the original image, if the center value of the Laplace convolution kernel is negative, then c=-1, if the center value of the Laplace convolution kernel is positive, then c=1.

[0145] Furthermore, the specific size of the standard size is determined according to the processing capability of the model. In specific implementation, the model performance can be determined according to the historical processing data of the model before obtaining the training sample, and then the standard size can be determined.

[0146] Furthermore, the cropped standard training samples are randomly cropped to increase the diversity of the data; random flip transformation is performed to further increase the diversity of the data; random rotation is performed to simulate different shooting angles; random color transformation is performed to simulate different lighting conditions; and geometric transformation is performed to simulate different shooting distances and angles. It should be noted that the above preprocessing operations can be selected and combined according to actual needs.

[0147] The self-iterative membrane electrode assembly defect detection method provided in this embodiment can improve image quality and increase data diversity through initial processing, so that the model can identify defects more accurately and improve detection accuracy and efficiency. In this way, by improving the quality of training data, the generalization ability of the model is improved, the risk of overfitting is reduced, and the model can be better trained.

[0148] Corresponding to the aforementioned embodiment of a self-iterative membrane electrode assembly defect detection method, the present application also provides an embodiment of a self-iterative membrane electrode assembly defect detection device.

[0149] The present application provides an embodiment of a self-iterative membrane electrode assembly defect detection device that can be applied to a self-iterative membrane electrode assembly defect detection device. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the processor of the self-iterative membrane electrode assembly defect detection device in which it is located reads the corresponding computer program instructions in the non-volatile memory into the memory and runs them. From the hardware level, if Figure 8 As shown, it is a hardware structure diagram of the self-iterative membrane electrode assembly defect detection device of the present application, except Figure 8 In addition to the processor, memory, network interface, and non-volatile memory shown, the self-iterative membrane electrode assembly defect detection device in which the device in the embodiment is located may also include other hardware, which will not be described in detail, generally based on the actual function of the self-iterative membrane electrode assembly defect detection device.

[0150] Fig. 9 This is a structural diagram of the first embodiment of the self-iterative membrane electrode assembly defect detection device provided by the present application. Fig. 9 The device provided in this embodiment includes a detection module 901, a determination module 902, a labeling module 903, a merging module 904 and a training module 905; wherein: The detection module 901 is used to use the pre-trained initial defect detection model as the current detection model, and use the current detection model to locate defects and identify defect types on the surface image of the membrane electrode assembly to be tested, so as to obtain the detection result of the membrane electrode assembly to be tested; wherein, when using the current detection model to locate defects and identify defect types on the surface image of the membrane electrode assembly to be tested, the surface image of the membrane electrode assembly to be tested is saved; The determination module 902 is used to determine the detection accuracy of the current detection model according to multiple detection results of multiple membrane electrode assemblies to be tested; The labeling module 903 is used to label the defect positions and defect types of the stored surface images of the membrane electrode assembly to be tested when the detection accuracy is less than a preset accuracy value, and determine the labeled multiple surface images as iterative training samples; The merging module 904 is used to merge the iterative training samples and the initial training samples used to train the initial defect detection model to obtain a training sample set; The training module 905 is used to retrain the current detection model using the training sample set to obtain an iterative defect detection model; wherein, when training the current detection model and the initial defect detection model, the loss function for calculating the category loss is an improved SlideLoss loss function; the value of the IOU threshold in the weight function of the improved SlideLoss loss function in the current iteration round is related to the average value of the intersection-and-union ratio corresponding to the current iteration round and the average value of the intersection-and-union ratio corresponding to the historical iteration rounds before the current iteration round; The detection module 901 is further configured to use the iterative defect detection model as a current detection model and perform defect detection using the current detection model.

[0151] The device of this embodiment can be used to perform Figure 1 The steps, specific implementation principles and implementation processes of the method embodiment shown are similar and will not be repeated here.

[0152] Please continue to refer to Figure 8 The present application also provides a self-iterative membrane electrode assembly defect detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the methods provided in the first aspect of the present application are implemented.

[0153] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of any one of the methods provided in the present application are implemented.

[0154] Optionally, in another possible implementation, the self-iterative membrane electrode assembly defect detection device may further include a storage module, which stores initial images to be detected and iterative images to be detected, initial training data sets and iterative training data sets, initial defect detection models and iterative defect detection models, and computer execution instructions for the two stages of initial training and iterative training.

[0155] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0156] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0157] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.< / height> < / width> < / y> < / x> < / height> < / width> < / y> < / x> < / height> < / width> < / y> < / x> < / height> < / width> < / y> < / x> < / object-class>

Claims

1. A self-iterative membrane electrode assembly defect detection method, characterized in that: The method comprises: The pre-trained initial defect detection model is used as the current detection model, and the surface image of the membrane electrode assembly to be tested is used to locate defects and identify defect types by using the current detection model to obtain the detection result of the membrane electrode assembly to be tested; wherein, when the surface image of the membrane electrode assembly to be tested is used to locate defects and identify defect types by using the current detection model, the surface image of the membrane electrode assembly to be tested is saved; Determining the detection accuracy of the current detection model according to multiple detection results of multiple membrane electrode assemblies to be tested; When the detection accuracy is less than a preset accuracy value, the stored surface image of the membrane electrode assembly to be tested is annotated with defect positions and defect types, and the annotated plurality of surface images are determined as iterative training samples; Merging the iterative training samples with the initial training samples used to train the initial defect detection model to obtain a training sample set; Retraining the current detection model using the training sample set to obtain an iterative defect detection model; wherein, when training the current detection model and the initial defect detection model, the loss function for calculating the category loss is an improved SlideLoss loss function; the value of the IOU threshold in the weight function of the improved SlideLoss loss function in the current iteration round is related to the average value of the intersection-and-union ratio corresponding to the current iteration round and the average value of the intersection-and-union ratio corresponding to the historical iteration rounds before the current iteration round; The iterative defect detection model is used as a current detection model, and defect detection is performed using the current detection model.

2. The method according to claim 1, characterized in that: The weight function of the improved SlideLoss loss function is as follows: ； Among them, the is the weight of the i-th training sample; Said is the intersection-and-union ratio between the predicted box and the true box of the i-th training sample; is the IOU threshold; In each round of training, the process of determining the IOU threshold of this round includes: Determine the decay rate of this round according to the first formula according to the preset decay rate threshold, the round value of the current round, and the preset time decay constant; According to the attenuation rate, the reference threshold of the previous round, and the average value of the intersection-and-union ratios of all training samples in the current round, the reference threshold of the current round is calculated according to the second formula; wherein the initial value of the reference threshold is a preset value; The reference threshold of this round is determined as the IOU threshold of this round; Among them, the first formula is: ； The second formula is: ； The decay is the decay rate, is the preset decay rate threshold, d is the round value of the current round, τ is the preset time decay constant, A1 is the reference threshold of this round, is the reference threshold of the previous round, is the average of the intersection-over-union ratios of all training samples in this round.

3. The method according to claim 1, characterized in that When training the current detection model and training the initial defect detection model, the second loss function for calculating the positioning loss is: ； ； ； Among them, the is the second loss function; the (x, y) is the center coordinate of the prediction box; the ( ) is the center coordinate of the real frame, is the intersection-and-union ratio between the auxiliary bounding box of the prediction box and the auxiliary bounding box of the real box; wherein the auxiliary bounding box of the prediction box is a bounding box centered on the center of the prediction box and smaller than the size of the prediction box, and the auxiliary bounding box of the real box is a bounding box centered on the center of the real box and smaller than the size of the real box; the sizes of the auxiliary bounding boxes of the prediction box and the auxiliary bounding boxes of the real box are negatively correlated with the intersection-and-union ratio of the prediction box and the real box.

4. The method according to claim 1, characterized in that: The initial defect detection model is a detection model based on the YOLO model; the backbone network in the initial defect detection model is a CSPNet backbone network that introduces a C3K2 module; the SPPF module in the CSPNet backbone network is an SPPF module that introduces a multi-head attention mechanism; the Bottleneck module in the C3K2 module includes a first convolution module, a second convolution module, a third convolution module and a fusion module; The first convolution module is used to perform convolution processing, normalization processing and activation processing on the input feature map in sequence to obtain a first intermediate feature map; The second convolution module includes a plurality of dilated convolution modules and a normalization processing module arranged behind each dilated convolution module; each dilated convolution module is used to perform a dilated convolution operation on the first intermediate feature map according to the dilated rate and convolution kernel corresponding to the dilated convolution module to obtain an initial dilated convolution result; the normalization processing module behind each dilated convolution module is used to normalize the initial dilated convolution result to obtain a normalized dilated convolution result; The third convolution module is used to fuse multiple normalized hole convolution results and output a second intermediate feature map; The fusion module is used to fuse the input feature map and the second intermediate feature map to obtain a fused feature map.

5. The method according to claim 1, characterized in that The defect position and defect type of the stored surface image of the membrane electrode assembly to be tested are marked, including: For each saved surface image, grayscale transformation processing, Gaussian filtering processing and edge extraction processing are sequentially performed on the surface image to obtain a mask label image corresponding to the surface image; wherein the foreground part in the mask label image represents the defect position; Determine the circumscribed rectangular frame of the foreground part in the mask label image, and mark the circumscribed rectangular frame as an automatic marking frame on the surface image to obtain a surface image with an automatic marking frame; Crop a portion of the image corresponding to the automatic annotation frame from the surface image with the automatic annotation frame, input the portion of the image into the current detection model, and obtain defect type information corresponding to the surface image through the current detection model; The center coordinates, width, and height of the automatic annotation frame are determined as defect location information, and the defect location information and the defect type information are determined as annotation information of the surface image.

6. The method according to claim 1, characterized in that Before training the initial defect detection model, the method further includes: Construct multiple loss function schemes; each loss function scheme includes a loss function for calculating the category loss and a loss function for calculating the positioning loss; Obtaining a public data set related to membrane electrode surface defects, and dividing the public data set into a training set and a test set; For each loss function scheme, use the loss function scheme as the loss function of the initial defect detection model, train the initial defect detection model using the training set, and obtain an intermediate model corresponding to the loss function scheme; Determine multiple accuracy indicators of the intermediate model corresponding to each loss function scheme based on the test set; According to the multiple accuracy indicators, an optimal loss function scheme is determined from the multiple loss function schemes as the loss function of the initial defect detection model.

7. The method according to claim 1, characterized in that Before retraining the current detection model using the training sample set, the method further includes: Classifying the training sample set according to defect types, and counting the number of samples included in each defect type; According to the number of samples included in each defect type, determine whether there is an imbalance in the distribution of categories in the training sample set; When there is an imbalance in category distribution in the training sample set, for the target defect type, the number of samples included in the defect category is expanded based on data enhancement technology or resampling technology to avoid the imbalance in category distribution; wherein, when the difference between the ratio of the number of samples included in a class of defect types to the total number of samples included in the training sample set and the standard value is greater than a preset threshold, the defect category is the target defect type; the standard value is the ratio of the number of samples included in the training set to the number of categories of the defect type included in the training set.

8. The method according to claim 1, characterized in that The method further comprises: When training the current detection model and the initial defect detection model, for each training sample, the training sample is first processed to obtain a processed training sample; wherein the first processing includes at least any one of the following processing: brightness adjustment, contrast adjustment, image denoising processing and image sharpening processing; The processed training samples are cut into standard sizes to obtain standard training samples; The standard training samples are preprocessed to obtain final training samples; wherein the preprocessing includes at least any one of the following processes: random cropping, random flipping transformation, random rotation, random color transformation, and geometric transformation.

9. A self-iterative membrane electrode assembly defect detection device, characterized in that: The device includes a detection module, a determination module, a labeling module, a merging module and a training module; wherein, The detection module is used to use the pre-trained initial defect detection model as the current detection model, and use the current detection model to locate defects and identify defect types on the surface image of the membrane electrode assembly to be tested, so as to obtain the detection result of the membrane electrode assembly to be tested; wherein, when the surface image of the membrane electrode assembly to be tested is located and the defect type is identified using the current detection model, the surface image of the membrane electrode assembly to be tested is saved; The determination module is used to determine the detection accuracy of the current detection model according to multiple detection results of multiple membrane electrode assemblies to be tested; The labeling module is used to label the defect positions and defect types of the stored surface images of the membrane electrode assembly to be tested when the detection accuracy is less than a preset accuracy value, and determine the labeled multiple surface images as iterative training samples; The merging module is used to merge the iterative training samples and the initial training samples used to train the initial defect detection model to obtain a training sample set; The training module is used to retrain the current detection model using the training sample set to obtain an iterative defect detection model; wherein, when training the current detection model and the initial defect detection model, the loss function for calculating the category loss is an improved SlideLoss loss function; the value of the IOU threshold in the weight function of the improved SlideLoss loss function in the current iteration round is related to the average value of the intersection-and-union ratio corresponding to the current iteration round and the average value of the intersection-and-union ratio corresponding to the historical iteration rounds before the current iteration round; The detection module is further configured to use the iterative defect detection model as a current detection model and perform defect detection using the current detection model.

10. A self-iterative membrane electrode assembly defect detection device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 8 when executing the program.

Images