Method for training defect detection model, defect detection method and device

Abstract

Embodiments of the present application provide a defect detection model training method, a defect detection method and a device. The training method can iteratively train a preset defect detection model based on known sample data to obtain a trained defect detection model. In each iteration process, a fusion feature image can be generated based on a visible light image and an infrared image. Then, a preset defect position detection head can be used to determine candidate defect position coordinates of the fusion feature image, and a preset heat density estimation sub-network can be used to estimate the heat density of the fusion feature image to generate a temperature difference image and a temperature gradient image. Finally, the candidate defect position coordinates, the temperature difference image and the temperature gradient image can be used to generate a defect area, a defect temperature difference and a defect depth corresponding to the candidate defect position coordinates. After defect detection using the trained defect detection model, the three-dimensional spatial distribution of the defect can be accurately described through the defect position coordinates, the defect temperature difference and the defect depth.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a training method for a defect detection model, a defect detection method, and an apparatus. Background Technology

[0002] With the development of battery technology, the application fields of batteries are becoming increasingly wide, and the requirements for battery performance are also becoming increasingly stringent. For example, before batteries leave the factory, it is necessary to conduct defect detection on their physical properties to ensure the consistency of their physical performance.

[0003] Currently, the physical characteristics of batteries are typically assessed through manual visual inspection or sensor testing. However, since the human eye can only observe relatively obvious defects, and sensors can only detect their presence, neither can accurately describe the three-dimensional spatial distribution of defects. Therefore, how to accurately describe the three-dimensional spatial distribution of battery defects has become a pressing technical problem to be solved in this field.

[0004] The above statements are for the purpose of providing background information in relation to this application only and do not necessarily constitute prior art. Summary of the Invention

[0005] In view of the above problems, embodiments of this application provide a training method for a defect detection model, a defect detection method, and an apparatus.

[0006] In a first aspect, embodiments of this application provide a training method for a defect detection model. The method includes: acquiring multiple sample data sets, wherein each sample data set includes a visible light image and an infrared image of a known defect object; iteratively training a preset defect detection model based on the multiple sample data sets to obtain a trained defect detection model; wherein any iteration process includes: generating a fused feature image based on the sample data input in the current iteration process; inputting the fused feature image into a preset defect location detection head to generate candidate defect location coordinates; inputting the fused feature image into a preset thermal density estimation subnetwork to generate a temperature difference image and a temperature gradient image; and generating the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image.

[0007] In the above embodiments, a preset defect detection model can be iteratively trained based on known sample data to obtain a trained defect detection model. During each iteration, a fused feature image can be generated based on the visible light and infrared images of the sample data. Then, a preset defect location detection head can be used to determine the coordinates of candidate defects in the fused feature image, and a preset thermal density estimation sub-network can be used to estimate the thermal density of the fused feature image, generating a temperature difference image and a temperature gradient image. Finally, based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image, the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates can be generated. Subsequently, using the trained defect detection model to detect defects in the object to be detected, such as a battery, the defect location coordinates, defect temperature difference, and defect depth can be obtained. This allows for a precise description of the three-dimensional spatial distribution of battery defects, facilitating accurate subsequent processing of battery defects and improving the efficiency of subsequent battery defect processing.

[0008] In some optional embodiments, each iteration process further includes: determining the defect level corresponding to the candidate defect location coordinates based on the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates.

[0009] In the above embodiments, the defect level corresponding to each candidate defect region can be determined based on the defect area, defect temperature difference, and defect depth of each candidate defect region, which facilitates subsequent processing of defects according to the level and thus improves defect processing efficiency.

[0010] In some optional embodiments, generating the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image includes: generating a mask image corresponding to the candidate defect location coordinates; and generating the defect area, defect temperature difference, and defect depth corresponding to the defect location coordinates based on the mask image, the temperature difference image, and the temperature gradient image.

[0011] In the above embodiments, the defect area, defect temperature difference, and defect depth of each candidate defect region can be calculated using the mask image, temperature difference image, and temperature gradient image of each candidate defect region. By combining the defect location information provided by the mask image with the physical characteristic information provided by the temperature difference image and temperature gradient image, irrelevant hotspots are not included in the calculation, reducing the risk of misidentification. Furthermore, this enables accurate and traceable quantitative evaluation of defects, improving the accuracy of subsequent defect detection.

[0012] In some optional embodiments, generating the fused feature image based on the sample data input in the current iteration process includes: extracting a visible light feature image from the visible light image input in the current iteration based on a preset visible light convolution model, wherein the preset visible light convolution model includes a parallel first convolution branch, a second convolution branch, and an identity mapping branch; extracting an infrared feature image from the infrared image input in the current iteration based on a preset infrared convolution model; and fusing the visible light feature image and the infrared feature image to generate the fused feature image.

[0013] In the above examples, during the training of the defect detection model, visible light feature images can be extracted through three parallel branches. The multi-branch structure integrates multi-scale receptive fields and original information, resulting in better model learning performance. Furthermore, the residual connection mechanism can effectively alleviate the gradient vanishing problem, making the network training process more stable, converging faster, and improving the efficiency of model training.

[0014] In some optional embodiments, the step of generating the fused feature image based on the fusion of the visible light feature image and the infrared feature image includes: stitching the visible light feature image and the infrared feature image together to obtain a stitched feature image; generating a weight for each channel of the stitched feature image based on a preset attention model, and performing weighted processing on the corresponding channels based on the weight of each channel; splitting the weighted stitched feature image into two feature maps; and mapping the pixel with the larger pixel value in the two feature maps to a frame image element by element to obtain the fused feature image.

[0015] In the above embodiments, the fused feature image is obtained by taking the maximum value of each pixel, which retains the strongest feature information and makes the contrast between the background and defects in the fused feature image more obvious, thereby making the subsequent defect detection more accurate.

[0016] Secondly, embodiments of this application provide a defect detection method, which includes: acquiring a trained defect detection model; reparameterizing the first convolutional branch, the second convolutional branch, and the identity mapping branch of the trained defect detection model into a single-path inference branch to obtain an inference defect detection model; inputting a target visible light image and a target infrared image into the inference defect detection model to obtain a defect detection result of the object to be detected, wherein the target visible light image is the visible light image of the object to be detected, and the target infrared image is the infrared image of the object to be detected.

[0017] In the above embodiments, a trained defect detection model can be obtained. Then, the three branches of the trained defect detection model can be reparameterized into a single inference branch to obtain a defect detection model for inference. After that, the visible light image and infrared image of the object to be detected are input into the defect detection model for inference to obtain the defect detection result of the object to be detected.

[0018] As can be seen from the foregoing embodiments, the defect detection results of the object to be detected may include the defect area, defect temperature difference, and defect depth corresponding to the coordinates of each candidate defect location. Thus, the three-dimensional spatial distribution of the defects of the object to be detected can be accurately described by the candidate defect location coordinates, defect temperature difference, and defect depth, which facilitates the accurate processing of defects in the object to be detected (e.g., battery) and improves the efficiency of subsequent defect processing.

[0019] In some optional embodiments, the step of inputting the target visible light image and the target infrared image into the inference defect detection model to obtain the defect detection result of the object to be detected includes: generating a target fusion feature image based on the target visible light image and the target infrared image; inputting the target fusion feature image into the defect location detection head of the inference defect detection model to generate target candidate defect location coordinates; inputting the target fusion feature image into the thermal density estimation subnetwork of the inference defect detection model to generate a target temperature difference image and a target temperature gradient image; and generating the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image.

[0020] In the above embodiments, a target fusion feature image can be generated based on the visible light and infrared images of the object to be detected. Then, the defect location detection head of the inference defect detection model can be used to determine the target candidate defect location coordinates in the target fusion feature image. Furthermore, the preset thermal density estimation sub-network of the inference defect detection model is used to estimate the thermal density of the target fusion feature image, generating a target temperature difference image and a target temperature gradient image. Finally, based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image, the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates can be generated. Thus, the three-dimensional spatial distribution of defects in the object to be detected (e.g., a battery) can be accurately described using the target candidate defect location coordinates, target defect temperature difference, and target defect depth, facilitating subsequent accurate defect processing and improving the efficiency of subsequent defect processing.

[0021] In some optional embodiments, the method further includes: determining the defect level corresponding to the location coordinates of the target candidate defect based on the target defect area, the target defect temperature difference, and the target defect depth.

[0022] In the above embodiments, the defect level corresponding to each candidate defect region can be determined based on the defect area, defect temperature difference, and defect depth of each candidate defect region of the object to be inspected. This facilitates subsequent processing of defects in the object to be inspected according to the level, thereby improving the efficiency of defect processing for the object to be inspected.

[0023] In some optional embodiments, the method further includes: performing defect processing on the object to be detected based on the defect processing method corresponding to the defect level.

[0024] In the above embodiments, defects of the object to be tested can be processed according to different defect levels, thereby enabling targeted processing of defects of the object to be tested and improving the efficiency of defect processing.

[0025] In some optional embodiments, generating the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image includes: generating a target mask image corresponding to the target candidate defect location coordinates; and generating the target defect area, target defect temperature difference, and target defect depth based on the target mask image, the target temperature difference image, and the target temperature gradient image.

[0026] In the above embodiments, the target defect area, target defect temperature difference, and target defect depth of each target candidate defect region can be calculated using the mask image, target temperature difference image, and target temperature gradient image of each target candidate defect region. By combining the defect location information provided by the mask image with the physical characteristic information provided by the target temperature difference image and target temperature gradient image, irrelevant hotspots are not included in the calculation, reducing the risk of misidentification. Furthermore, this enables accurate and traceable quantitative assessment of defects, improving the accuracy of subsequent defect detection.

[0027] In some optional embodiments, generating a target fusion feature image based on the target visible light image and the target infrared image includes: extracting a target visible light feature image from the target visible light image based on the single-path inference branch; extracting a target infrared feature image from the target infrared image based on the infrared convolution model of the inference defect detection model; and fusing the target visible light feature image and the target infrared feature image to generate the target fusion feature image.

[0028] In the above embodiments, a single-path inference branch is used to perform feature recognition on the target visible light image to obtain the target visible light feature image. Since the single-path inference branch can improve the inference speed, thereby improving the inference efficiency, the defect detection model containing the single-path inference branch can be used to detect defects in the object to be detected more quickly, thus improving the defect detection efficiency of the object to be detected.

[0029] In some optional embodiments, the step of fusing the target visible light feature image and the target infrared feature image to generate the target fused feature image includes: stitching the target visible light feature image and the target infrared feature image together to obtain a target stitched feature image; generating target weights for each channel of the target stitched feature image based on the attention model of the inference defect detection model, and performing weighted processing on the corresponding channels based on the target weights of each channel; splitting the weighted target stitched feature image into two target feature maps; and mapping the pixel with the larger pixel value in the two target feature maps to a frame image element by element to obtain the target fused feature image.

[0030] In the above embodiments, the target fusion feature image is obtained by taking the maximum value of each pixel, which retains the strongest feature information and makes the contrast between the background and defects in the target fusion feature image more obvious, thereby making the defect detection result of the object to be detected more accurate.

[0031] Thirdly, embodiments of this application provide a training apparatus for a defect detection model. The apparatus includes: a first acquisition module for acquiring multiple sample data, wherein each sample data includes a visible light image and an infrared image of a known defect object; a training module for iteratively training a preset defect detection model based on the multiple sample data to obtain a trained defect detection model; wherein any iteration process includes: generating a fused feature image based on the sample data input in the current iteration process; inputting the fused feature image into a preset defect location detection head to generate candidate defect location coordinates; inputting the fused feature image into a preset thermal density estimation subnetwork to generate a temperature difference image and a temperature gradient image; and generating the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image.

[0032] Fourthly, embodiments of this application provide a defect detection device, which includes: a second acquisition module for acquiring a trained defect detection model; a structure processing module for reparameterizing the first convolutional branch, the second convolutional branch, and the identity mapping branch of the trained defect detection model into a single-path inference branch to obtain an inference defect detection model; and a detection module for inputting a target visible light image and a target infrared image into the inference defect detection model to obtain a defect detection result of the object to be detected, wherein the target visible light image is the visible light image of the object to be detected, and the target infrared image is the infrared image of the object to be detected.

[0033] Fifthly, embodiments of this application provide a sealing performance testing system, which includes a robot for performing the methods of the first or second aspect.

[0034] In a sixth aspect, embodiments of this application provide an electronic device including a processor and a memory, the processor being configured to execute a computer program stored in the memory to cause the electronic device to perform the method of the first aspect or the second aspect.

[0035] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of the first or second aspect.

[0036] In the technical solution provided in this application embodiment, a preset defect detection model can be iteratively trained based on known sample data to obtain a trained defect detection model. In each iteration, a fused feature image can be generated based on the visible light and infrared images of the sample data. Then, a preset defect location detection head can be used to determine the candidate defect location coordinates of the fused feature image, and a preset thermal density estimation sub-network can be used to estimate the thermal density of the fused feature image, generating a temperature difference image and a temperature gradient image. Finally, based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image, the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates can be generated. Subsequently, using the trained defect detection model to detect defects in the object to be detected, such as a battery, the defect location coordinates, defect temperature difference, and defect depth can be obtained. This allows for a precise description of the three-dimensional spatial distribution of battery defects, facilitating accurate subsequent processing of battery defects and improving the efficiency of subsequent battery defect processing. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the architecture of a defect detection system according to one or more embodiments;

[0039] Figure 2 This is a flowchart illustrating a training method for a defect detection model according to one or more embodiments;

[0040] Figure 3 This is a flowchart illustrating a method for training a preset defect detection model in any one iteration according to one or more embodiments.

[0041] Figure 4 This is a flowchart illustrating a defect detection method according to one or more embodiments;

[0042] Figure 5 This is a flowchart illustrating a method for implementing step S203 according to one or more embodiments;

[0043] Figure 6 This is a flowchart illustrating a defect detection method according to one or more embodiments;

[0044] Figure 7 This is a schematic diagram of the architecture of a preset defect detection model according to one or more embodiments;

[0045] Figure 8 This is a structural block diagram of a training device for a defect detection model according to one or more embodiments;

[0046] Figure 9 This is a structural block diagram of a defect detection device according to one or more embodiments. Detailed Implementation

[0047] The embodiments of the technical solution of this application are described below with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0048] It should be noted that the term "embodiment" as used in this application means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. Those skilled in the art will explicitly and implicitly understand that the embodiments described in this application can be combined with other embodiments.

[0049] Unless otherwise defined, the technical and scientific terms used in the embodiments of this application have the same meaning as understood by one of ordinary skill in the art to which this application belongs. The terminology used in the embodiments of this application is for the purpose of describing the embodiments only and is not intended to limit the application.

[0050] In the description of the embodiments in this application, the term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment illustrated as "exemplary" is not necessarily to be construed as superior to or better than other embodiments. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.

[0051] The terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion. The terms “first,” “second,” “third,” etc., are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, particular order, or primary or secondary relationship of the indicated technical features. The term “multiple” means two or more (including two), unless otherwise expressly and specifically defined.

[0052] The term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this text generally indicates that the preceding and following related objects are in an "or" relationship.

[0053] Unless otherwise explicitly specified and limited, the terms "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can be a mechanical connection or an electrical connection; they can be a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0054] Furthermore, the terms "upper," "lower," "inner," "outer," "front," "back," "left," "right," "top," and "bottom," etc., indicate the orientation or positional relationship based on the working state of the embodiments of this application. They are only used to facilitate the description of the embodiments of this application and to simplify the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0055] In the description of the embodiments of this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature and the second feature are in direct contact, or that the first feature and the second feature are in indirect contact through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0056] In the description of the embodiments of this application, "parallel" includes not only the case of absolute parallelism, but also the case of approximate parallelism as commonly understood in engineering; at the same time, "perpendicular" also includes not only the case of absolute perpendicularity, but also the case of approximate perpendicularity as commonly understood in engineering.

[0057] In the description of the embodiments of this application, the same reference numerals denote the same components, and for the sake of brevity, detailed descriptions of the same components are omitted in different embodiments. It should be understood that the thickness, length, and other dimensions of various components in the embodiments of this application shown in the drawings, as well as the overall thickness, length, and other dimensions of the integrated device, are merely illustrative and should not constitute any limitation on this application.

[0058] To facilitate understanding of the technical solutions of this application, the application scenarios of the technical solutions provided in the embodiments of this application will be described by way of example below.

[0059] With the development of battery technology, the application fields of batteries are becoming increasingly wide, and the requirements for battery performance are also becoming increasingly stringent. For example, before batteries leave the factory, it is necessary to conduct defect detection on their physical properties to ensure the consistency of their physical performance.

[0060] In related technologies, the physical performance of batteries can be tested by manual visual inspection. However, the human eye can only observe relatively obvious physical defects, such as bumps and spots on the surface of the battery. For internal defects of the battery or relatively small surface defects, the human eye usually cannot observe them.

[0061] Sensors, such as ultrasonic detectors and X-rays, can only detect whether there are defects inside the battery, but cannot accurately describe the three-dimensional spatial distribution of the defects.

[0062] In addition, image vision technology can be used to detect defects in batteries. However, image vision technology can only detect defects such as texture, color, and shape on the surface of the battery. It cannot identify defects inside the battery, nor can it accurately describe the three-dimensional spatial distribution of defects.

[0063] Based on this, in order to accurately describe the three-dimensional spatial distribution of battery defects and improve the accuracy of battery defect detection, this application provides a training method, a defect detection method, and an apparatus for a defect detection model. In this method, a preset defect detection model can be iteratively trained based on known sample data to obtain a trained defect detection model. In each iteration, a fused feature image can be generated based on the visible light and infrared images of the sample data. Then, a preset defect location detection head can be used to determine the coordinates of candidate defects in the fused feature image, and a preset thermal density estimation sub-network can be used to estimate the thermal density of the fused feature image, generating a temperature difference image and a temperature gradient image. Finally, based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image, the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates can be generated. Subsequently, using the trained defect detection model to detect defects in the target object, such as a battery, the defect location coordinates, defect temperature difference, and defect depth can be obtained. Thus, the three-dimensional spatial distribution of battery defects can be accurately described using the defect location coordinates, defect temperature difference, and defect depth, facilitating accurate subsequent processing of battery defects and improving the efficiency of subsequent battery defect processing.

[0064] It should be understood that the training method, defect detection method and apparatus of the above-mentioned defect detection model provided in the embodiments of this application can be applied not only to the defect detection scenario of batteries, but also to other application scenarios that require defect detection, such as defect detection scenarios in high-end manufacturing, precision electronics, new energy, etc., which will not be listed here one by one.

[0065] The system architecture of the embodiments of this application will be described by way of example below.

[0066] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of a defect detection system according to one or more embodiments. The training method for the defect detection model and the defect detection method provided in the embodiments of this application can both be applied to this defect detection system.

[0067] like Figure 1As shown, the defect detection system may include a visible light camera, an infrared camera, and a terminal device with a graphics processing unit (GPU). Both the visible light camera and the infrared camera are communicatively connected to the terminal device. It should be understood that the defect detection system may also include more or fewer devices than shown in the figure. For example, the defect detection system may also include one or more field-programmable gate array (FPGA) computing nodes and a programmable LED dome light source supporting strobe and 256 levels of brightness adjustment, etc., which are not limited in this application.

[0068] Optionally, the number of visible light cameras can be one or more, and this application does not limit this. For example, when defect detection is only required on a portion of the object to be inspected (e.g., a battery), and the visible light camera can successfully capture image information of the portion to be inspected, the number of visible light cameras can be one. As another example, when defect detection is required on the entire object to be inspected, and one visible light camera cannot capture all image information of the object, the number of visible light cameras can be multiple. It should be understood that the number of visible light cameras can be set to multiple in any scenario.

[0069] For example, in some optional embodiments, six visible light cameras can be set in the defect detection system, and these six visible light cameras can be arranged in a 120-degree ring, such that the field of view overlap between any two of the six visible light cameras is greater than or equal to 25%, so as to obtain omnidirectional images through these six visible light cameras, for example, to obtain omnidirectional images of the object to be detected through these six visible light cameras.

[0070] For example, to obtain clearer, higher-resolution images, a visible light camera with relatively high resolution can be selected. However, to balance image clarity and relatively low computational load, the defect detection system provided in this application embodiment can be equipped with a 4K visible light camera.

[0071] Similarly, optionally, the number of infrared cameras can be one or more. Optionally, the parameter settings of the infrared cameras can also be set according to the needs of the actual application scenario; this application does not impose any restrictions on this.

[0072] For example, in some optional embodiments, four infrared cameras can be set in the defect detection system. The mounting angles of the four infrared cameras intersect with those of the visible light camera, so that the infrared cameras and the visible light camera in the defect detection system can meet the field-of-view synchronization requirements.

[0073] For example, in the defect detection system provided in this application embodiment, a short-wave infrared (SWIR) camera with a wavelength of 900-1700 nanometers (nm) can be set.

[0074] Optionally, the terminal device can acquire visible light images captured by a visible light camera and infrared images captured by an infrared camera.

[0075] Optionally, the defect detection system may further include a rejection mechanism for removing defective objects, such as defective batteries, from the production line. For example, the defect detection system may include a high-speed rejection mechanism with a response time of less than or equal to 30 milliseconds (ms).

[0076] The functions or roles of each device in this defect detection system can be found in the following embodiments, and will not be described in detail here.

[0077] The training method of the defect detection model provided in the embodiments of this application will be described by way of example below.

[0078] See Figure 2 , Figure 2 This is a flowchart illustrating a training method for a defect detection model according to one or more embodiments. The method can be applied to terminal devices with GPUs (e.g.,...). Figure 1 (The terminal device shown). Figure 2 As shown, the method may include the following steps:

[0079] Step S101: Obtain multiple sample data.

[0080] Each sample data includes a visible light image and an infrared image of a known defective object. A known defective object refers to an object whose defects are known to correspond to the object to be detected. The object to be detected is the object whose defects are to be detected. For example, if the object to be detected is a battery, the known defective object is a battery with known defects. It should be understood that the defect types of the known defective objects corresponding to multiple sample data can include one or more types; that is, the defect types of the known defective objects corresponding to multiple sample data can be the same or different, and this application does not impose any restrictions on this.

[0081] For example, defect types may include texture defects, color defects, shape defects, etc. It should be understood that defect types may also include other types, which will not be listed here.

[0082] Each sample data includes visible light and infrared images of the same location (or region) of a known defective object. That is, for any given sample data, assuming the visible light image is obtained by photographing the target area of ​​the corresponding known defective object using a visible light camera, then the infrared image is obtained by photographing the target area of ​​the known defective object using an infrared camera.

[0083] Step S102: Iteratively train the preset defect detection model based on the multiple sample data to obtain the trained defect detection model.

[0084] The preset defect detection model may include a dual-branch network, one of which is a visible light branch network used to identify features of the visible light image, such as texture features, color features, and other visual features, to obtain the corresponding visible light feature image. For example, the visible light branch network can be used to identify features of the visible light image of a known defective object to obtain the visible light feature image of the known defective object.

[0085] Another branch network is the infrared branch network, which is used to identify features of infrared images, such as thermal distribution features, to obtain corresponding infrared feature images. For example, the infrared branch network can be used to identify features of infrared images of known defective objects to obtain infrared feature images of the known defective objects.

[0086] For example, the visible light branch network can be a RepVGG-A0 structured network, employing a three-branch structure during model training and a single-path structure during inference. The input image to the visible light branch network can be a visible light image, for example, an RGB image of 4096 pixels × 3072 pixels. The output image of the visible light branch network can be a visible light feature image with 384 feature channels × 128 pixels × 96 pixels.

[0087] For example, the visible light branch network may include five convolutional layers. The first convolutional layer can have a stride of 2, 24 feature channels, and an output size of 2048 pixels × 1536 pixels. The second convolutional layer can have a stride of 2, 48 feature channels, and an output size of 1024 pixels × 768 pixels. The third convolutional layer can have a stride of 2, 96 feature channels, and an output size of 512 pixels × 384 pixels. The fourth convolutional layer can have a stride of 2, 192 feature channels, and an output size of 256 pixels × 192 pixels. The fifth convolutional layer can have a stride of 2, 384 feature channels, and an output size of 128 pixels × 96 pixels.

[0088] For example, the infrared branch network can also be a RepVGG structured network. The input image of the infrared branch network can be an infrared image, for example, a single-channel infrared image (i.e., a grayscale image) of 640 pixels × 512 pixels.

[0089] For example, the infrared branch network may include three convolutional layers. The stride of the first convolutional layer can be set to 2, the number of feature channels can be set to 16, and the output size can be set to 320 pixels × 256 pixels. The stride of the second convolutional layer can be set to 2, the number of feature channels can be set to 32, and the output size can be set to 160 pixels × 128 pixels. The stride of the third convolutional layer can be set to 2, the number of feature channels can be set to 64, and the output size can be set to 80 pixels × 64 pixels. Through these three convolutional layers, a feature image with dimensions of 64 feature channels × 80 pixels × 64 pixels can be obtained. Based on this, the infrared branch network may optionally include a deconvolutional layer, set after the third convolutional layer, for deconvolving the feature image output by the third convolutional layer, aligning it with the dimensions of the visible light feature image, to obtain an infrared feature image with dimensions of 384 feature channels × 128 pixels × 96 pixels.

[0090] See Figure 3 , Figure 3 This is a flowchart illustrating a method for performing arbitrary iterations of training a preset defect detection model according to one or more embodiments. Figure 3 As shown, any iteration process may include the following steps:

[0091] Step S1021: Generate a fused feature image based on the sample data input in this iteration process.

[0092] Among them, the fused feature image is the feature image obtained by fusing the visible light feature image and the infrared feature image.

[0093] Step S1022: Input the fused feature image into the preset defect location detection head to generate candidate defect location coordinates.

[0094] Optionally, the preset defect detection model may further include one or more preset defect location detection heads. Each preset defect location detection head is used to identify a defect of a certain type, determine the location of a candidate defect region, and output the candidate defect location coordinates corresponding to the candidate defect region. The candidate defect region may also be referred to as a suspected defect region or a suspicious defect region, etc. The candidate defect location coordinates may also be referred to as suspected defect location coordinates or suspicious defect location coordinates, etc., and this application does not impose any limitations on this.

[0095] After obtaining the fused feature image, it can be input into a preset defect location detection head to obtain the candidate defect location coordinates for each candidate defect region. The candidate defect location coordinates are used to indicate the position of the candidate defect region in the original image, which may include the visible light image and infrared image corresponding to the input preset defect detection model.

[0096] Step S1023: Input the fused feature image into the preset thermal density estimation sub-network to generate a temperature difference image and a temperature gradient image.

[0097] Optionally, the preset defect detection model may further include a heat density estimation sub-network (or, more accurately, a heat density detection head) to acquire the three-dimensional temperature information of the candidate defect region and output the corresponding three-dimensional temperature feature map. The three-dimensional temperature information of the candidate defect region may include temperature difference information, X-axis temperature gradient information, and Y-axis temperature gradient information. Specifically, the temperature difference information indicates the temperature difference of the corresponding candidate defect region relative to the normal temperature. The X-axis temperature gradient information indicates the temperature gradient change of the corresponding candidate defect region along the X-axis. The Y-axis temperature gradient information indicates the temperature gradient change of the corresponding candidate defect region along the Y-axis.

[0098] For example, the three-dimensional temperature feature map may include a temperature difference image and a temperature gradient image. The temperature gradient image may include an X-axis temperature gradient image and a Y-axis temperature gradient image. The temperature difference image includes temperature difference information of the corresponding candidate defect region, the X-axis temperature gradient image includes X-axis temperature gradient information of the corresponding candidate defect region, and the Y-axis temperature gradient image includes Y-axis temperature gradient information of the corresponding candidate defect region.

[0099] For example, after inputting the fused feature image into the thermal density estimation subnetwork, the thermal density estimation subnetwork can perform a deconvolution operation on the fused feature image to upsample it to the original image resolution, obtaining a temperature difference image, an X-axis temperature gradient image, and a Y-axis temperature gradient image. Assuming the original image has dimensions of 4096 pixels × 3072 pixels × 3 channels, the temperatures in the temperature difference image, the X-axis temperature gradient image, and the Y-axis temperature gradient image are all 4096 pixels × 3072 pixels × 3 channels.

[0100] Step S1024: Based on the candidate defect location coordinates, temperature difference image, and temperature gradient image, generate the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates.

[0101] After obtaining the candidate defect location coordinates, temperature difference image, and temperature gradient image for each candidate defect region, the defect area, defect temperature difference, and defect depth of each candidate defect region can be calculated based on these parameters. Specifically, for any candidate defect region, the defect area refers to the area of ​​the region within that region where the temperature is higher than the baseline threshold; the defect temperature difference refers to the difference between the highest temperature in that candidate defect region and the baseline threshold; and the defect depth refers to the difference between the maximum depth of that candidate defect region and the baseline threshold.

[0102] The defect model training method provided in this application embodiment can iteratively train a preset defect detection model based on known sample data to obtain a trained defect detection model. In each iteration, a fused feature image can be generated based on the visible light and infrared images of the sample data. Then, a preset defect location detection head can be used to determine the candidate defect location coordinates of the fused feature image, and a preset thermal density estimation sub-network can be used to estimate the thermal density of the fused feature image, generating a temperature difference image and a temperature gradient image. Finally, based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image, the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates can be generated. Subsequently, using the trained defect detection model to detect defects in an object to be detected, such as a battery, the defect location coordinates, defect temperature difference, and defect depth can be obtained. This allows for a precise description of the three-dimensional spatial distribution of battery defects, facilitating accurate subsequent processing of battery defects and improving the efficiency of subsequent battery defect processing.

[0103] In some optional embodiments, any iteration process may further include: determining the defect level corresponding to the candidate defect location coordinates based on the defect area, defect temperature difference and defect depth corresponding to the candidate defect location coordinates.

[0104] For example, defect levels can be preset, and a set of corresponding parameter value ranges can be configured for each defect level. The parameter value ranges may include the value range of defect area, the value range of defect temperature difference, and the value range of defect depth.

[0105] For any candidate defect region, the defect level can be determined based on its corresponding defect area, defect temperature difference, defect depth, and a pre-set range of parameter values.

[0106] For example, a defect level classification model can also be configured for the preset defect detection model, and the defect level corresponding to each candidate defect region can be determined through the defect level classification model.

[0107] In the above embodiments, the defect level corresponding to each candidate defect region can be determined based on the defect area, defect temperature difference, and defect depth of each candidate defect region, which facilitates subsequent processing of defects according to the level and thus improves defect processing efficiency.

[0108] In some optional embodiments, the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates can be generated based on the candidate defect location coordinates, temperature difference image, and temperature gradient image. This can be achieved by generating a mask image corresponding to the candidate defect location coordinates; and generating the defect area, defect temperature difference, and defect depth corresponding to the defect location coordinates based on the mask image, temperature difference image, and temperature gradient image.

[0109] For any candidate defect region, after obtaining the candidate defect location coordinates, temperature difference image, and temperature gradient image, a mask image corresponding to the candidate defect region can be constructed based on the candidate defect location coordinates. The mask image of the candidate defect region is a binary image with the same size as the corresponding visible light image. In the region corresponding to the candidate defect location coordinates in the mask image, the pixel value of each pixel is 1 or 255, while the pixel value of each pixel in the remaining regions of the mask image is 0.

[0110] Then, based on the mask image, temperature difference image, and temperature gradient image of the candidate defect region, the defect area, defect temperature difference, and defect depth of the candidate defect region can be calculated.

[0111] In the above embodiments, the defect area, defect temperature difference, and defect depth of each candidate defect region can be calculated using the mask image, temperature difference image, and temperature gradient image of each candidate defect region. By combining the defect location information provided by the mask image with the physical characteristic information provided by the temperature difference image and temperature gradient image, irrelevant hotspots are not included in the calculation, reducing the risk of misidentification. Furthermore, this enables accurate and traceable quantitative evaluation of defects, improving the accuracy of subsequent defect detection.

[0112] In some optional embodiments, generating a fused feature image based on the sample data input in the current iteration process can be achieved as follows: extracting a visible light feature image from the visible light image input in the current iteration based on a preset visible light convolution model, wherein the preset visible light convolution model includes a parallel first convolution branch, a second convolution branch, and an identity mapping branch; extracting an infrared feature image from the infrared image input in the current iteration based on a preset infrared convolution model; and fusing the visible light feature image and the infrared feature image to generate the fused feature image.

[0113] For example, the preset visible light convolutional model can be the visible light branch network of the aforementioned preset defect detection model. The first convolutional branch can be a convolutional branch with a kernel size of 3×3, and the second convolutional branch can be a convolutional branch with a kernel size of 1×1. The specific implementation method for extracting visible light feature images from the visible light image input in this iteration based on the preset visible light convolutional model can be found in the content of the aforementioned embodiments, and will not be repeated here.

[0114] For example, the preset infrared convolution model can be the infrared branch network of the aforementioned preset defect detection model. The infrared feature image is extracted from the infrared image input in this iteration based on the preset infrared convolution model. The content of the aforementioned embodiments can also be referred to, and will not be repeated here.

[0115] In the above embodiments, during the training of the defect detection model, visible light feature images can be extracted through three parallel branches. The multi-branch structure integrates multi-scale receptive fields and original information, resulting in better model learning performance. Furthermore, the residual connection mechanism can effectively alleviate the gradient vanishing problem, making the network training process more stable, converging faster, and improving the efficiency of model training.

[0116] In some optional embodiments, the generation of a fused feature image based on the fusion of visible light feature images and infrared feature images can be achieved as follows: the visible light feature images and infrared feature images are stitched together to obtain a stitched feature image; a weight for each channel of the stitched feature image is generated based on a preset attention model, and the corresponding channels are weighted based on the weight of each channel; the weighted stitched feature image is split into two feature maps; and the pixel with the larger pixel value in the two feature maps is mapped to a frame image element by element to obtain the fused feature image.

[0117] For example, when stitching visible light feature images and infrared feature images, channel stitching can be performed directly to obtain a stitched feature image. For instance, if the visible light feature image is a feature image with 384 feature channels × 128 pixels × 96 pixels, and the infrared feature image is also a feature image with 384 feature channels × 128 pixels × 96 pixels, then the stitched feature image obtained can be a feature image with 768 feature channels × 128 pixels × 96 pixels.

[0118] For example, the preset attention model can be a Channel Attention adaptive gating attention model, and the weights of each channel can be generated by the following relation w∈R^(384×1×1), where w represents the weight and R is used to indicate that each element in w is a real number.

[0119] By applying weights to each channel to obtain a weighted, stitched feature image, the channels of this stitched feature image can be split to obtain two feature maps. For example, if the weighted, stitched feature image is a feature image with 768 feature channels × 128 pixels × 96 pixels, then the two split feature maps are both feature images with 384 feature channels × 128 pixels × 96 pixels.

[0120] By mapping the pixel with the larger pixel value in the two feature maps to a single frame image element by element, a fused feature image can be obtained. That is, each pixel in the two feature maps is compared element by element, the larger pixel value is selected, and then the corresponding pixel is mapped to a blank frame image. The value of the pixel is set to the larger selected pixel value. After all the pixels are mapped, the fused feature image can be obtained.

[0121] In the above embodiments, the fused feature image is obtained by taking the maximum value of each pixel, which retains the strongest feature information and makes the contrast between the background and defects in the fused feature image more obvious, thereby making the subsequent defect detection more accurate.

[0122] The defect detection method provided in the embodiments of this application will be described by way of example below.

[0123] See Figure 4 , Figure 4 This is a flowchart illustrating a defect detection method according to one or more embodiments. The method can be applied to terminal devices with GPUs (e.g.,...). Figure 1 (The terminal device shown). Figure 4 As shown, the method may include the following steps:

[0124] Step S201: Obtain the trained defect detection model.

[0125] For example, in order to reduce computational load and improve defect detection efficiency, it is possible to pre-calculate based on... Figure 2 and Figure 3 The training method shown yields a trained defect detection model. The trained defect detection model is then stored, and can be loaded from the system during step S201.

[0126] For example, in order to further improve the accuracy of defect detection, when performing step S201, it can also be done according to... Figure 2 and Figure 3 The training method shown uses the latest sample data to train the preset defect detection model, resulting in a trained defect detection model.

[0127] Step S202: The first convolutional branch, the second convolutional branch, and the identity mapping branch of the trained defect detection model are reparameterized into a single inference branch to obtain the defect detection model for inference.

[0128] The specific architecture of the first convolutional branch, the second convolutional branch, and the identity mapping branch of the trained defect detection model can be referred to the content of the aforementioned embodiments, and will not be repeated here.

[0129] By reparameterizing the first convolutional branch, the second convolutional branch, and the identity mapping branch of the trained defect detection model into a single inference branch, the inference speed can be improved, thereby increasing the inference efficiency. Subsequently, by using the inference model to perform defect detection on the object to be detected, the defect detection results of the object to be detected can be obtained more quickly.

[0130] Step S203: Input the visible light image and infrared image of the target into the inference defect detection model to obtain the defect detection result of the object to be detected.

[0131] The target visible light image is the visible light image of the object to be detected. The target infrared image is the infrared image of the object to be detected.

[0132] In the defect detection method provided in this application embodiment, a trained defect detection model can be obtained. Then, the three branch structures of the trained defect detection model can be reparameterized into a single inference branch to obtain an inference defect detection model. After that, the visible light image and infrared image of the object to be detected are input into the inference defect detection model to obtain the defect detection result of the object to be detected.

[0133] As can be seen from the foregoing embodiments, the defect detection results of the object to be detected may include the defect area, defect temperature difference, and defect depth corresponding to the coordinates of each candidate defect location. Thus, the three-dimensional spatial distribution of the defects of the object to be detected can be accurately described by the candidate defect location coordinates, defect temperature difference, and defect depth, which facilitates the accurate processing of defects in the object to be detected (e.g., battery) and improves the efficiency of subsequent defect processing.

[0134] In some alternative embodiments, such as Figure 5 As shown, inputting the visible light image and infrared image of the target into the inference defect detection model to obtain the defect detection result of the object to be detected can be achieved in the following way:

[0135] Step S2031: Generate a target fusion feature image based on the target visible light image and the target infrared image.

[0136] Among them, the target fusion feature image is the fusion feature image corresponding to the object to be detected.

[0137] Step S2032: Input the target fused feature image into the defect location detection head of the defect detection model for inference to generate the target candidate defect location coordinates.

[0138] Among them, the target candidate defect location coordinates are the candidate defect location coordinates of the candidate defect region of the object to be detected.

[0139] Step S2033: Input the target fusion feature image into the thermal density estimation sub-network of the defect detection model for inference to generate the target temperature difference image and the target temperature gradient image.

[0140] The target temperature difference image is the temperature difference image of the object to be detected. The target temperature gradient image is the temperature gradient image of the object to be detected. The target temperature gradient image includes the target X-axis temperature gradient image and the target Y-axis temperature gradient image.

[0141] Step S2034: Based on the target candidate defect location coordinates, target temperature difference image, and target temperature gradient image, generate the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates.

[0142] Among them, the target defect area, the target defect temperature difference, and the target defect depth are the defect area, defect temperature difference, and defect depth of the candidate defect region of the object to be detected, respectively.

[0143] The specific implementation methods of steps S2031 to S2034 can be referred to the content of the foregoing embodiments, and will not be repeated here.

[0144] In the above embodiments, a target fusion feature image can be generated based on the visible light and infrared images of the object to be detected. Then, the defect location detection head of the inference defect detection model can be used to determine the target candidate defect location coordinates in the target fusion feature image. Furthermore, the preset thermal density estimation sub-network of the inference defect detection model is used to estimate the thermal density of the target fusion feature image, generating a target temperature difference image and a target temperature gradient image. Finally, based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image, the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates can be generated. Thus, the three-dimensional spatial distribution of defects in the object to be detected (e.g., a battery) can be accurately described using the target candidate defect location coordinates, target defect temperature difference, and target defect depth, facilitating subsequent accurate defect processing and improving the efficiency of subsequent defect processing.

[0145] In some optional embodiments, the defect detection method may further include: determining the defect level corresponding to the location coordinates of the target candidate defect based on the target defect area, the target defect temperature difference, and the target defect depth. Specific implementation details can be found in the foregoing embodiments and will not be repeated here.

[0146] In the above embodiments, the defect level corresponding to each candidate defect region can be determined based on the defect area, defect temperature difference, and defect depth of each candidate defect region of the object to be inspected. This facilitates subsequent processing of defects in the object to be inspected according to the level, thereby improving the efficiency of defect processing for the object to be inspected.

[0147] In some optional embodiments, the defect detection method may further include: performing defect processing on the object to be detected based on the defect processing method corresponding to the defect level.

[0148] In the above embodiments, defects of the object to be tested can be processed according to different defect levels, thereby enabling targeted processing of defects of the object to be tested and improving the efficiency of defect processing.

[0149] In some optional embodiments, the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates can be generated based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image. This can be achieved as follows: generate a target mask image corresponding to the target candidate defect location coordinates; and generate the target defect area, target defect temperature difference, and target defect depth based on the target mask image, the target temperature difference image, and the target temperature gradient image.

[0150] The target mask image refers to the mask image of the object to be detected. The specific content of the mask image can be referred to the content of the foregoing embodiments, and will not be described in detail here.

[0151] The specific implementation of generating the target defect area, target defect temperature difference, and target defect depth based on the target mask image, target temperature difference image, and target temperature gradient image can be found in the foregoing embodiments and will not be repeated here.

[0152] In the above embodiments, the target defect area, target defect temperature difference, and target defect depth of each target candidate defect region can be calculated using the mask image, target temperature difference image, and target temperature gradient image of each target candidate defect region. By combining the defect location information provided by the mask image with the physical characteristic information provided by the target temperature difference image and target temperature gradient image, irrelevant hotspots are not included in the calculation, reducing the risk of misidentification. Furthermore, this enables accurate and traceable quantitative assessment of defects, improving the accuracy of subsequent defect detection.

[0153] In some optional embodiments, generating a target fusion feature image based on a target visible light image and a target infrared image can be achieved as follows: extracting a target visible light feature image from a target visible light image based on a single-path inference branch; extracting a target infrared feature image from a target infrared image based on an infrared convolution model of an inference defect detection model; and generating a target fusion feature image by fusing the target visible light feature image and the target infrared feature image.

[0154] The architecture of the infrared convolutional model for the defect detection model used for inference can refer to the architecture of the infrared branch network in the aforementioned embodiments, and will not be described in detail here.

[0155] In the above embodiments, a single-path inference branch is used to perform feature recognition on the target visible light image to obtain the target visible light feature image. Since the single-path inference branch can improve the inference speed, thereby improving the inference efficiency, the defect detection model containing the single-path inference branch can be used to detect defects in the object to be detected more quickly, thus improving the defect detection efficiency of the object to be detected.

[0156] In some optional embodiments, the generation of a target fused feature image based on the fusion of the target visible light feature image and the target infrared feature image can be implemented as follows: The target visible light feature image and the target infrared feature image are stitched together to obtain a target stitched feature image; target weights for each channel of the target stitched feature image are generated based on the attention model of the inference defect detection model, and the corresponding channels are weighted based on the target weights of each channel; the weighted target stitched feature image is split into two target feature maps; and the pixel with the larger pixel value in each of the two target feature maps is mapped to a single frame image element by element to obtain the target fused feature image. For specific implementation details, please refer to the foregoing embodiments; they will not be repeated here.

[0157] In the above embodiments, the target fusion feature image is obtained by taking the maximum value of each pixel, which retains the strongest feature information and makes the contrast between the background and defects in the target fusion feature image more obvious, thereby making the defect detection result of the object to be detected more accurate.

[0158] The embodiments of this application will be described below using a battery as an example.

[0159] See Figure 6 , Figure 6 This is a flowchart illustrating a defect detection method according to one or more embodiments. Figure 6 As shown, it can be done according to Figure 6 The method shown is for defect detection of batteries, and the method may include the following steps:

[0160] Step S301: Synchronize the visible light camera and the infrared camera.

[0161] The specific setup methods for the visible light camera and the infrared camera can be found in the aforementioned embodiments, and will not be repeated here.

[0162] It should be understood that for any pair of visible light cameras and infrared cameras used to photograph the same area (e.g., the same area of ​​a battery under inspection or the same area of ​​a defective battery), synchronization is required to achieve synchronization between the visible light image and the infrared image.

[0163] For example, for any pair of visible light cameras and infrared cameras used to photograph the same area, the shooting time of the visible light cameras and infrared cameras can be synchronized through a precision time protocol (PTP), so that the visible light cameras and infrared cameras can photograph the battery under inspection or the defective battery in a synchronized manner, thereby making the frame-level synchronization error of the captured visible light images and infrared images less than 100 microseconds (μs).

[0164] For example, for any pair of visible light cameras and infrared cameras used to capture the same area, the system extrinsic parameters between the visible light cameras and infrared cameras, i.e., the relative extrinsic parameters between visible light and infrared, can be calibrated so that the reprojection error between the visible light image captured by the visible light camera and the infrared image captured by the infrared camera is less than 0.1 pixels.

[0165] Step S302: Use the synchronized visible light camera and infrared camera to photograph multiple defective batteries to obtain visible light images and infrared images of each defective battery.

[0166] Step S303: Generate multiple sample data based on visible light and infrared images of multiple defective batteries.

[0167] One sample data set includes a visible light image and an infrared image, wherein the visible light image and the infrared image correspond to the same area of ​​the same defective battery.

[0168] Step S304: Use multiple sample data to train the preset defect detection model to obtain the trained defect detection model.

[0169] For example, the architecture of the preset defect detection model can be referred to Figure 7 ,like Figure 7 As shown, the preset defect detection model can include the following architecture:

[0170] S1, dual-input branch.

[0171] One branch of the dual-input branch is used to input a 4096-pixel × 3072-pixel RGB image, i.e., a visible light image. For example, during the training phase of a pre-defined defect detection model, this branch can be used to input a visible light image of a defective battery. The other branch of the dual-input branch is used to input a 640-pixel × 512-pixel single-channel image (grayscale image), i.e., an infrared image. For example, during the training phase of a pre-defined defect detection model, this branch can be used to input an infrared image of a defective battery.

[0172] S2, lightweight feature extraction architecture.

[0173] The lightweight feature extraction architecture may include a RepVGG-V5 layer network feature extraction architecture (hereinafter referred to as the RepVGG-V architecture) for lightweight feature extraction of visible light images, and a RepVGG-T3 layer network feature extraction architecture (hereinafter referred to as the RepVGG-T architecture) for lightweight feature extraction of infrared images.

[0174] For example, the RepVGG-V architecture may include five convolutional layers. The first convolutional layer can have a stride of 2, 24 feature channels, and an output size of 2048 pixels × 1536 pixels. The second convolutional layer can have a stride of 2, 48 feature channels, and an output size of 1024 pixels × 768 pixels. The third convolutional layer can have a stride of 2, 96 feature channels, and an output size of 512 pixels × 384 pixels. The fourth convolutional layer can have a stride of 2, 192 feature channels, and an output size of 256 pixels × 192 pixels. The fifth convolutional layer can have a stride of 2, 384 feature channels, and an output size of 128 pixels × 96 pixels.

[0175] For example, during the training phase, the RepVGG-V architecture may include three branches: a convolutional branch with a kernel size of 3×3, a convolutional branch with a kernel size of 1×1, and an identity mapping branch.

[0176] After inputting a 4096-pixel × 3072-pixel RGB image into a preset defect detection model, the RepVGG-V architecture can be used to extract features from the RGB image, resulting in a visible light feature image with 384 feature channels × 128 pixels × 96 pixels. In this embodiment, the visible light feature image can be denoted as FV. That is, using the RepVGG-V architecture, an FV feature output can be obtained, and the dimensions of the output feature image are 384 feature channels × 128 pixels × 96 pixels. For example, after inputting a 4096-pixel × 3072-pixel RGB image of a defective battery into the preset defect detection model, the visible light feature image FV of the defective battery can be obtained.

[0177] For example, the RepVGG-T architecture may include three convolutional layers and a deconvolutional layer following the third convolutional layer. The stride of the first convolutional layer can be set to 2, the number of feature channels can be set to 16, and the output size can be set to 320 pixels × 256 pixels. The stride of the second convolutional layer can be set to 2, the number of feature channels can be set to 32, and the output size can be set to 160 pixels × 128 pixels. The stride of the third convolutional layer can be set to 2, the number of feature channels can be set to 64, and the output size can be set to 80 pixels × 64 pixels. The deconvolutional layer is used to deconvolve the feature image output by the third convolutional layer, so that the infrared feature image output by the RepVGG-T architecture is dimensionally aligned with the visible light feature image.

[0178] After inputting a 640-pixel × 512-pixel single-channel image into a preset defect detection model, the RepVGG-T architecture can be used to extract features from this single-channel image, resulting in an infrared feature image with 384 feature channels × 128 pixels × 96 pixels. In this embodiment, the infrared feature image can be denoted as FT. That is, using the RepVGG-T architecture, FT feature output can be obtained, and the dimensions of the output feature image are 384 feature channels × 128 pixels × 96 pixels. For example, after inputting the infrared image of a defective battery into a preset defect detection model, the infrared feature image FT of the defective battery can be obtained.

[0179] Alternatively, the deconvolution layer can be placed outside the RepVGG-T architecture. First, the RepVGG-T architecture is used to obtain the initial FT feature output, which outputs a feature image with 64 feature channels × 80 pixels × 64 pixels. Then, the deconvolution layer is used to perform a deconvolution operation on the initial FT feature output. The deconvolution operation is used to align the dimensions with the visible light feature image, resulting in the final FT feature output, which is an infrared feature image with 384 feature channels × 128 pixels × 96 pixels.

[0180] S3 and CS-CA feature fusion architecture.

[0181] The CS-CA feature fusion architecture can be used to fuse the FV and FT features output by the lightweight feature extraction architecture. That is, it can fuse visible light feature images and infrared feature images to obtain a fused feature image. For example, during the training phase, the CS-CA feature fusion architecture can be used to fuse the visible light feature images and infrared feature images of defective batteries to obtain a fused feature image of the defective battery.

[0182] For example, during the training phase, the visible light feature image and infrared feature image of the defective battery can be input into the CS-CA feature fusion architecture. First, feature vector concatenation is performed, concatenating the visible light feature image (384 feature channels × 128 pixels × 96 pixels) with the infrared feature image (384 feature channels × 128 pixels × 96 pixels) to obtain a concatenated feature image F = [FV, FT] with 768 feature channels × 128 pixels × 96 pixels. Then, using the ChannelAttention adaptive gating attention model, the following... The relation w∈R^(384×1×1) generates the weights for each channel, and these weights are applied to the stitched feature image F, resulting in a weighted stitched feature image F1=w×F, where w represents the weights and R indicates that each element in w is a real number. Then, the weighted stitched feature image F1 is split and fused to obtain a fused feature image. Specifically, F1 with 768 feature channels is first split into two feature images with 384 feature channels × 128 pixels × 96 pixels. Then, element-wise max→ F_fusion ∈ R^(384×128×96) is performed, comparing each pixel in the two split feature images element-wise, selecting the larger pixel value, and then mapping the pixel to a blank frame, setting the value of that pixel to the larger selected pixel value. After mapping all pixels, the fused feature image F_fusion is obtained, where F_fusion is a real tensor with dimensions 384×128×96.

[0183] S4, Decoupled Head Detection Head Architecture.

[0184] For example, a decoupled head inspection architecture may include multiple inspection heads, which may include a defect location inspection head and a thermal density inspection head.

[0185] For example, the defect location detection head can be used to determine the location of the candidate defect region from the fused feature image and output the candidate defect location coordinates [x1, x2, y1, y2]. The candidate defect location coordinates are the coordinates of the original image that maps the candidate defect region to 4096 pixels × 3072 pixels × 3 channels.

[0186] For example, a thermal density detection head can be used to perform upsampling using a lightweight deconvolution branch, resampling the depth semantics to the original spatial resolution to obtain a 3D temperature feature map. That is, the thermal density detection head can be used to perform a deconvolution operation on the fused feature image, upsampling it to the original image resolution to obtain ( T / x, T / The image contains three plots: a temperature difference image (x-axis), a temperature gradient image (x-axis), and a temperature gradient image (y-axis). All three plots are 4096 pixels × 3072 pixels × 3 channels. T / x represents the temperature gradient along the X-axis. T / y represents the temperature gradient along the Y-axis, and ΔT represents the temperature difference between the candidate defect region and the normal temperature.

[0187] S5, Defect Hierarchy Architecture.

[0188] The defect grading architecture is used to determine defect indicators based on candidate defect location coordinates, temperature difference images, X-axis temperature gradient images, and Y-axis temperature gradient images. For example, defect indicators may include: 1. Candidate defect location coordinates; 2. Defect area, defect temperature difference (or defect temperature rise), and defect depth corresponding to the candidate defect location coordinates.

[0189] After obtaining the defect indicators, defects in the corresponding candidate defect areas can be classified based on these indicators. For example, for any candidate defect area of ​​a battery (e.g., a defective battery), if the defect area is greater than area threshold 1, the defect temperature difference is greater than the temperature rise threshold, and the defect depth is greater than the depth threshold, the defect level of the corresponding candidate defect area can be considered as scrap level. Subsequently, the corresponding battery can be removed from the battery production line for scrapping using a high-speed rejection mechanism. Alternatively, if the defect area is greater than area threshold 2 but less than area threshold 1, and the defect temperature difference is less than the temperature rise threshold, the defect level of the corresponding candidate defect area can be considered as manually judged level. Subsequently, the corresponding candidate defect area of ​​the corresponding battery can be manually judged. Alternatively, if the defect area is less than area threshold 2, the defect temperature difference is less than the temperature rise threshold, and the defect depth is less than the depth threshold, the defect level of the corresponding candidate defect area can be considered as observation level. Subsequently, the corresponding area of ​​the corresponding battery can be marked for observation and key monitoring. The area threshold 1, area threshold 2, temperature rise threshold, and depth threshold can all be set according to the needs of the actual application scenario.

[0190] It should be understood that corresponding defect classification strategies can also be formulated based on the needs of actual application scenarios, and this application does not impose any restrictions on this.

[0191] Step S305: Reparameterize the RepVGG-V architecture of the trained defect detection model into a single inference branch to obtain the defect detection model for inference.

[0192] Step S306: Use the synchronized visible light camera and infrared camera to take pictures of the battery under test to obtain visible light images and infrared images of the battery under test.

[0193] Step S307: Input the visible light image and infrared image of the battery to be tested into the defect detection model for inference to obtain the defect detection result of the battery to be tested.

[0194] For example, the defect detection results of the battery under test may include the candidate defect location coordinates of the candidate defect region of the battery under test, as well as the defect area, defect temperature difference and defect depth corresponding to the candidate defect location coordinates.

[0195] For example, the defect detection result of the battery under test may also include the candidate defect location coordinates of the candidate defect region of the battery under test, and the defect level corresponding to the candidate defect location coordinates.

[0196] For example, the defect detection results of the battery under test may also include the candidate defect location coordinates of the candidate defect region of the battery under test, as well as the defect area, defect temperature difference, defect depth and defect level corresponding to the candidate defect location coordinates.

[0197] The defect detection method provided in the above embodiments can accurately describe the three-dimensional spatial distribution of defects in the battery under test by using the candidate defect location coordinates, defect temperature difference, and defect depth. This facilitates precise processing of defects in the battery under test and improves the efficiency of battery defect processing.

[0198] Based on the same inventive concept, embodiments of this application also provide a training device for a defect detection model.

[0199] See Figure 8 , Figure 8 This is a structural block diagram of a training apparatus for a defect detection model according to one or more embodiments. The training apparatus 800 for the defect detection model may include: a first acquisition module 801 and a training module 802.

[0200] The first acquisition module 801 is used to acquire multiple sample data, wherein each sample data includes a visible light image and an infrared image of a known defective object;

[0201] Training module 802 is used to iteratively train the preset defect detection model based on the multiple sample data to obtain the trained defect detection model;

[0202] The training module is used to execute any iteration process, including: generating a fused feature image based on the sample data input in the current iteration process; inputting the fused feature image into a preset defect location detection head to generate candidate defect location coordinates; inputting the fused feature image into a preset thermal density estimation sub-network to generate a temperature difference image and a temperature gradient image; and generating the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image.

[0203] In some optional embodiments, the training module is further configured to: determine the defect level corresponding to the candidate defect location coordinates based on the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates.

[0204] In some optional embodiments, the training module is used to generate the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image, including: the training module is used to: generate a mask image corresponding to the candidate defect location coordinates; and generate the defect area, defect temperature difference, and defect depth corresponding to the defect location coordinates based on the mask image, the temperature difference image, and the temperature gradient image.

[0205] In some optional embodiments, the training module is used to generate a fused feature image based on the sample data input in the current iteration process, including: extracting a visible light feature image from the visible light image input in the current iteration based on a preset visible light convolution model, wherein the preset visible light convolution model includes a parallel first convolution branch, a second convolution branch, and an identity mapping branch; extracting an infrared feature image from the infrared image input in the current iteration based on a preset infrared convolution model; and fusing the visible light feature image and the infrared feature image to generate the fused feature image.

[0206] In some optional embodiments, the training module is used to generate the fused feature image based on the fusion of the visible light feature image and the infrared feature image, including: the training module is used to: stitch the visible light feature image and the infrared feature image to obtain a stitched feature image; generate the weight of each channel of the stitched feature image based on a preset attention model, and perform weighted processing on the corresponding channels based on the weight of each channel; split the weighted stitched feature image into two feature maps; and map the pixel with the larger pixel value in the two feature maps to a frame image element by element to obtain the fused feature image.

[0207] Based on the same inventive concept, embodiments of this application also provide a defect detection device.

[0208] See Figure 9 , Figure 9 This is a structural block diagram of a defect detection device according to one or more embodiments. The defect detection device 900 may include: a second acquisition module 901, a structure processing module 902, and a detection module 903.

[0209] The second acquisition module 901 is used to acquire the trained defect detection model;

[0210] The structure processing module 902 is used to reparameterize the first convolutional branch, the second convolutional branch, and the identity mapping branch of the trained defect detection model into a single inference branch to obtain a defect detection model for inference.

[0211] The detection module 903 is used to input the target visible light image and the target infrared image into the inference defect detection model to obtain the defect detection result of the object to be detected, wherein the target visible light image is the visible light image of the object to be detected, and the target infrared image is the infrared image of the object to be detected.

[0212] In some optional embodiments, the detection module is used to input the target visible light image and the target infrared image into the inference defect detection model to obtain the defect detection result of the object to be detected, including: the detection module is used to: generate a target fusion feature image based on the target visible light image and the target infrared image; input the target fusion feature image into the defect location detection head of the inference defect detection model to generate target candidate defect location coordinates; input the target fusion feature image into the thermal density estimation subnetwork of the inference defect detection model to generate a target temperature difference image and a target temperature gradient image; and generate the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image.

[0213] In some optional embodiments, the defect detection device further includes a determination module, used to determine the defect level corresponding to the location coordinates of the target candidate defect based on the target defect area, the target defect temperature difference, and the target defect depth.

[0214] In some optional embodiments, the defect detection device further includes a defect processing module, used to process the object to be detected based on the defect processing method corresponding to the defect level.

[0215] In some optional embodiments, the detection module is used to generate the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image, including: the detection module is used to: generate a target mask image corresponding to the target candidate defect location coordinates; and generate the target defect area, target defect temperature difference, and target defect depth based on the target mask image, the target temperature difference image, and the target temperature gradient image.

[0216] In some optional embodiments, the detection module is used to generate a target fusion feature image based on the target visible light image and the target infrared image, including: the detection module is used to: extract a target visible light feature image from the target visible light image based on the single-path inference branch; extract a target infrared feature image from the target infrared image based on the infrared convolution model of the inference defect detection model; and generate the target fusion feature image by fusing the target visible light feature image and the target infrared feature image.

[0217] In some optional embodiments, the detection module is used to generate the target fused feature image based on the fusion of the target visible light feature image and the target infrared feature image, including: the detection module stitching the target visible light feature image and the target infrared feature image to obtain a target stitched feature image; generating target weights for each channel of the target stitched feature image based on the attention model of the inference defect detection model, and performing weighted processing on the corresponding channels based on the target weights of each channel; splitting the weighted target stitched feature image into two target feature maps; and mapping the pixel with the larger pixel value in the two target feature maps to a frame image element by element to obtain the target fused feature image.

[0218] Based on the same inventive concept, embodiments of this application also provide a defect detection system, which includes a terminal device with a GPU, the terminal device being used to execute... Figures 2-6 The method in any of the method embodiments shown in the figures is illustrated. It should be understood that the defect detection system may also include other devices, such as FPGA computing nodes, etc. Exemplarily, the defect detection system can be... Figure 1 The defect detection system shown.

[0219] Based on the same inventive concept, embodiments of this application also provide an electronic device, which includes a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so as to cause the electronic device to perform... Figures 2-6 The method in any of the method embodiments shown in any of the figures.

[0220] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when run on an electronic device, causes the electronic device to perform... Figures 2-6 The method in any of the method embodiments shown in any of the figures.

[0221] Based on the same inventive concept, this application also provides a computer program product, which includes a computer program that, when run on an electronic device, causes the electronic device to perform... Figures 2-6 The method in any of the method embodiments shown in any of the figures.

[0222] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0223] If these functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0224] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the training device, defect detection device, defect detection system, electronic device, computer-readable storage medium, and computer program product of the defect detection model described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0225] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, system, electronic device, and method can be implemented in other ways. For example, the embodiments of the apparatus and electronic device described above are merely illustrative; multiple components may be combined or integrated into another system, or some features may be omitted or not performed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection between devices or units through some interface, and may be electrical, mechanical, or other forms.

[0226] It should be understood that in the various embodiments of this application, the execution order of each step should be determined by its function and internal logic, and the size of each step number does not mean the order of execution, and does not constitute a limitation on the implementation process of the embodiments.

[0227] The various parts of this specification are described in a progressive manner. Similar or identical parts between the various embodiments can be referred to interchangeably. Each embodiment focuses on the differences from other embodiments. In particular, the embodiments for training devices, defect detection devices, defect detection systems, electronic devices, computer-readable storage media, and computer program products for defect detection models are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant details can be found in the descriptions within the method embodiments.

[0228] In this document, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Those skilled in the art will understand the specific meaning of the above terms in this application according to the specific circumstances. It should be noted that, without conflict, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to any single aspect, nor to any single embodiment, nor to any combination and / or substitution of these aspects and / or embodiments. Moreover, each aspect and / or embodiment of this application can be used alone or in combination with one or more other aspects and / or embodiments thereof.

[0229] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of this application.

Claims

1. A training method for a defect detection model, characterized in that, The method includes: Acquire multiple sample data, wherein each sample data includes a visible light image and an infrared image of a known defective object, the known defective object including a battery; Based on the multiple sample data, the preset defect detection model is iteratively trained to obtain a trained defect detection model. Each iteration process includes: A fused feature image is generated based on the sample data input in this iteration process; The fused feature image is input into a preset defect location detection head to generate candidate defect location coordinates corresponding to the candidate defect region. The fused feature image is input into a preset thermal density estimation sub-network to generate a temperature difference image and a temperature gradient image. Based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image, the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates are generated. The temperature difference image includes temperature difference information, which is used to indicate the temperature difference of the candidate defect region relative to the normal temperature. The temperature gradient image includes an X-axis temperature gradient image and a Y-axis temperature gradient image; The X-axis temperature gradient image includes X-axis temperature gradient information corresponding to the candidate defect region, and the X-axis temperature gradient information is used to indicate the temperature gradient change of the candidate defect region in the X-axis direction. The Y-axis temperature gradient image includes Y-axis temperature gradient information corresponding to the candidate defect region, and the Y-axis temperature gradient information is used to indicate the temperature gradient change of the candidate defect region in the Y-axis direction.

2. The method according to claim 1, characterized in that, The process of any given iteration also includes: Based on the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates, the defect level corresponding to the candidate defect location coordinates is determined.

3. The method according to claim 1 or 2, characterized in that, The step of generating the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image includes: Generate a mask image corresponding to the coordinates of the candidate defect location; Based on the mask image, the temperature difference image, and the temperature gradient image, the defect area, defect temperature difference, and defect depth corresponding to the defect location coordinates are generated.

4. The method according to claim 1 or 2, characterized in that, The generation of the fused feature image based on the sample data input in this iteration process includes: The visible light feature image is extracted from the visible light image input in this iteration based on a preset visible light convolution model, which includes a first convolution branch, a second convolution branch, and an identity mapping branch in parallel. Infrared feature images are extracted from the infrared images input in this iteration based on a preset infrared convolution model; The fused feature image is generated by fusing the visible light feature image and the infrared feature image.

5. The method according to claim 4, characterized in that, The process of generating the fused feature image based on the fusion of the visible light feature image and the infrared feature image includes: The visible light feature image and the infrared feature image are stitched together to obtain a stitched feature image; The weights of each channel of the stitched feature image are generated based on a preset attention model, and the corresponding channels are weighted based on the weights of each channel. The weighted, stitched feature image is split into two feature maps; The pixel with the larger pixel value in each of the two feature images is mapped to a single frame image element by element to obtain the fused feature image.

6. A defect detection method, characterized in that, The method includes: A trained defect detection model is obtained using the method described in any one of claims 1-5; The first convolutional branch, the second convolutional branch, and the identity mapping branch of the trained defect detection model are reparameterized into a single inference branch to obtain a defect detection model for inference. The visible light image and infrared image of the target are input into the defect detection model for inference to obtain the defect detection result of the object to be detected. The visible light image of the target is the visible light image of the object to be detected, and the infrared image of the target is the infrared image of the object to be detected. The object to be detected includes a battery.

7. The method according to claim 6, characterized in that, The step of inputting the visible light image and infrared image of the target into the inference defect detection model to obtain the defect detection result of the object to be detected includes: A target fusion feature image is generated based on the target visible light image and the target infrared image; The target fusion feature image is input into the defect location detection head of the inference defect detection model to generate the target candidate defect location coordinates; The target fusion feature image is input into the thermal density estimation subnetwork of the inference defect detection model to generate a target temperature difference image and a target temperature gradient image. Based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image, the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates are generated.

8. The method according to claim 7, characterized in that, The method further includes: Based on the target defect area, target defect temperature difference, and target defect depth, the defect level corresponding to the location coordinates of the target candidate defect is determined.

9. The method according to claim 8, characterized in that, The method further includes: performing defect processing on the object to be detected based on the defect processing method corresponding to the defect level.

10. The method according to claim 7, characterized in that, The step of generating the target defect area, target defect temperature difference, and target defect depth corresponding to the target candidate defect location coordinates based on the target candidate defect location coordinates, the target temperature difference image, and the target temperature gradient image includes: Generate a target mask image corresponding to the coordinates of the target candidate defect location; Based on the target mask image, the target temperature difference image, and the target temperature gradient image, the target defect area, the target defect temperature difference, and the target defect depth are generated.

11. The method according to claim 7, characterized in that, The step of generating a target fusion feature image based on the target visible light image and the target infrared image includes: Based on the single-path inference branch, extract the target visible light feature image from the target visible light image; Based on the inference defect detection model, the infrared convolution model extracts the target infrared feature image from the target infrared image; The target fused feature image is generated by fusing the target visible light feature image and the target infrared feature image.

12. The method according to claim 11, characterized in that, The process of generating the target fused feature image based on the fusion of the target visible light feature image and the target infrared feature image includes: The visible light feature image and the infrared feature image of the target are stitched together to obtain a stitched feature image of the target. Based on the attention model of the inference defect detection model, the target weights of each channel of the target stitched feature image are generated, and the corresponding channels are weighted based on the target weights of each channel. The weighted target stitched feature image is split into two target feature maps; The pixel with the larger pixel value in the two target feature maps is mapped to a frame image element by element to obtain the target fused feature image.

13. A training device for a defect detection model, characterized in that, The device includes: The first acquisition module is used to acquire multiple sample data, wherein each sample data includes a visible light image and an infrared image of a known defective object, and the known defective object includes a battery; The training module is used to iteratively train the preset defect detection model based on the multiple sample data to obtain the trained defect detection model. Each iteration process includes: A fused feature image is generated based on the sample data input in this iteration process; The fused feature image is input into a preset defect location detection head to generate candidate defect location coordinates; The fused feature image is input into a preset thermal density estimation sub-network to generate a temperature difference image and a temperature gradient image. Based on the candidate defect location coordinates, the temperature difference image, and the temperature gradient image, the defect area, defect temperature difference, and defect depth corresponding to the candidate defect location coordinates are generated. The temperature difference image includes temperature difference information, which is used to indicate the temperature difference of the candidate defect region relative to the normal temperature. The temperature gradient image includes an X-axis temperature gradient image and a Y-axis temperature gradient image; The X-axis temperature gradient image includes X-axis temperature gradient information corresponding to the candidate defect region, and the X-axis temperature gradient information is used to indicate the temperature gradient change of the candidate defect region in the X-axis direction. The Y-axis temperature gradient image includes Y-axis temperature gradient information corresponding to the candidate defect region, and the Y-axis temperature gradient information is used to indicate the temperature gradient change of the candidate defect region in the Y-axis direction.

14. A defect detection device, characterized in that, The device includes: The second acquisition module is used to acquire the trained defect detection model using the method described in any one of claims 1-5; The structure processing module is used to reparameterize the first convolutional branch, the second convolutional branch, and the identity mapping branch of the trained defect detection model into a single inference branch to obtain a defect detection model for inference. The detection module is used to input the target visible light image and the target infrared image into the inference defect detection model to obtain the defect detection result of the object to be detected, wherein the target visible light image is the visible light image of the object to be detected, the target infrared image is the infrared image of the object to be detected, and the object to be detected includes a battery.

15. A defect detection system, characterized in that, The system includes: a terminal device having a graphics processor, the terminal device being used to perform the method of any one of claims 1-5 or 6-12.

16. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program stored in the memory to cause the electronic device to perform the method of any one of claims 1-5 or 6-12.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-5 or 6-12.

Images