Image-based appearance defect detection method, device, medium, and program

Abstract

Embodiments of the present application disclose a kind of appearance defect detection method, equipment, medium and program based on image, wherein, method includes: different exposure acquisition image of target detection object under the same angle of view is obtained;Different exposure acquisition image of the target detection object under the same angle of view is synthesized into the pseudo-color image of the target detection object;According to the pseudo-color image of the target detection object, inference image data is generated;According to the inference image data, defect recognition and classification are carried out, and the appearance defect detection result of the target detection object is obtained.The technical scheme of the embodiment of the present application can improve the generality and detection precision of the appearance defect detection method based on image.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image-based method, apparatus, electronic device, storage medium and program for detecting appearance defects. Background Technology

[0002] Image-based surface defect detection is one of the main applications of computer vision. Computer vision technology acquires product images and uses algorithms to analyze them, enabling automated detection of surface defects. Its applications cover multiple fields, including automotive parts, electronic components, and new energy products, and the detected defects include, but are not limited to, scratches, color differences, and foreign objects.

[0003] Currently, the main implementation methods for image-based appearance defect detection methods include core steps such as image preprocessing, feature extraction, and template matching, combining traditional machine vision and deep learning technologies to achieve defect detection.

[0004] In the process of realizing this invention, the following shortcomings of the existing technology were discovered: Existing image-based appearance defect detection methods often only collect a single image of the object to be detected as the data source, perform core steps such as image preprocessing, feature extraction, and template matching, and then output the appearance defect detection result. Or traditional image processing methods rely on parameter settings, which are sensitive to the grayscale, contrast, and morphological features of the image background and the defect target. However, the shape and color of defects are varied, so traditional image processing methods are difficult to set a set of universal detection parameters, that is, it is difficult to determine a set of parameter settings that can adapt to defects of various shapes, and it is difficult to capture complex or abstract defect patterns. During product switching, parameters need to be frequently adjusted to adapt to changes in ambient light. Under non-uniform lighting conditions, threshold segmentation may fail, resulting in false detections. They also have disadvantages such as weak processing capability for complex scenes and difficulty in scene transfer. Summary of the Invention

[0005] This invention provides an image-based method, apparatus, electronic device, storage medium, and program for detecting appearance defects, which can improve the versatility and detection accuracy of image-based appearance defect detection methods.

[0006] According to one aspect of the present invention, an image-based method for detecting appearance defects is provided, comprising:

[0007] Acquire images of the target object at different exposures from the same viewpoint;

[0008] The images of the target object acquired at different exposures from the same viewpoint are combined to form a pseudo-color image of the target object.

[0009] Generate inference image data based on the pseudo-color image of the target detection object;

[0010] Based on the inference image data, defect identification and classification are performed to obtain the appearance defect detection results of the target detection object.

[0011] According to another aspect of the present invention, an image-based appearance defect detection device is provided, comprising:

[0012] The exposure acquisition image module is used to acquire images of the target object under different exposures from the same viewpoint;

[0013] The pseudo-color image synthesis module is used to synthesize images of the target detection object acquired at different exposures from the same viewpoint into a pseudo-color image of the target detection object.

[0014] The inference image data generation module is used to generate inference image data based on the pseudo-color image of the target detection object;

[0015] The appearance defect detection result acquisition module is used to identify and classify defects based on the inference image data to obtain the appearance defect detection result of the target detection object.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the image-based appearance defect detection method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the image-based appearance defect detection method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the image-based appearance defect detection method according to any embodiment of the present invention.

[0021] This invention acquires images of a target object under different exposures from the same viewpoint, and then synthesizes these images into a pseudo-color image of the target object. Further, the pseudo-color image is processed to generate inference image data, which is then used for defect identification and classification to obtain the appearance defect detection result of the target object. This technical solution improves the uniformity and feature clarity of defect imaging in the data source used for image inference detection by synthesizing images of the target object under different exposures from the same viewpoint into a pseudo-color image. It addresses the problems of low versatility and detection accuracy in existing image-based appearance defect detection methods, thereby improving the versatility and detection accuracy of image-based appearance defect detection methods.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of an image-based appearance defect detection method provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart of an image-based appearance defect detection method provided in Embodiment 2 of the present invention;

[0026] Figure 3 This is a schematic diagram of an image-based appearance defect detection device provided in Embodiment 3 of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," and "third," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Example 1

[0031] Figure 1 This is a flowchart of an image-based appearance defect detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where appearance defects are detected by acquiring images of a target object at different exposures from the same viewpoint. The method can be executed by an image-based appearance defect detection device, which can be implemented in software and / or hardware, and is generally integrated into an electronic device. This electronic device can be a terminal device or a server device, as long as it can execute the image-based appearance defect detection method. The present invention does not limit the specific type of electronic device. Accordingly, as... Figure 1 As shown, the method includes the following operations:

[0032] S110. Acquire images of the target object at different exposures from the same viewpoint.

[0033] The target detection object can be any object requiring image-based appearance defect detection. For example, the target detection object may include, but is not limited to, automotive parts, electronic components, processed foods, textiles, and new energy products. This embodiment of the invention does not limit the specific type of the target detection object.

[0034] In this embodiment of the invention, when it is necessary to automatically detect appearance defects on a target object using an image-based appearance defect detection method, an initial image data set of the target object can be obtained first. This initial image data set may include multiple surface images of the target object captured by different optical cameras at different acquisition angles and under different lighting conditions. For example, when the target object is a power battery, the initial image data set may include surface images of a power battery pack (referring to the process of connecting multiple cells in series and parallel and packaging them into a directly usable energy storage unit). These power battery pack surface images may be multiple pack surface images captured by different optical cameras at different acquisition angles and under different lighting conditions.

[0035] The initial image dataset of the target object serves as the foundational data source for subsequent appearance defect detection. Multiple target objects can be composed of multiple surface images acquired by different optical cameras at different acquisition angles and under different lighting conditions. Optionally, the acquired images of the same target object can be images acquired at different exposures from the same viewpoint. That is, each target object can simultaneously acquire multiple images of it at different exposures from different optical cameras at the same viewpoint under different exposure conditions. Different acquisition settings can comprehensively capture information about the surface appearance of the target object. For example, the target object can be illuminated by light sources of different directions and angles around a single camera, and then photographed sequentially. These light sources can be distributed around the camera, and the angle and brightness of each light source can be set differently according to the shape of the target object, such as a 30-degree angle combined with low brightness or a 45-degree angle combined with high brightness.

[0036] S120. Combine the images of the target object acquired at different exposures from the same viewpoint into a pseudo-color image of the target object.

[0037] In this context, a pseudo-color image can be an image generated by converting images with different exposures.

[0038] Under different exposure conditions, the appearance of the target object may vary due to differences in lighting. Specifically, in some low-exposure images, white defects or defects with clear textures are imaged more clearly, while black defects or defects with faint textures are imaged blurry; in some high-exposure images, black defects are imaged clearly, but white defects are prone to overexposure, affecting the identification of defects.

[0039] To improve image processing efficiency and enhance the representation of feature information in images, images of the target object acquired at different exposures from the same viewpoint can be synthesized into a pseudo-color image of the target object. That is, by combining images of the target object acquired at different exposures from the same viewpoint into a single image, the synthesized pseudo-color image can simultaneously include image feature information of the target object at different exposures from the same viewpoint, offering advantages such as uniform defect imaging and clear feature representation. Therefore, the aforementioned pseudo-color image synthesis method can adjust the overall features of the image based on the local regional features of different images.

[0040] S130. Generate inference image data based on the pseudo-color image of the target detection object.

[0041] In this context, the inference image data can be the data used as input to the corresponding image processing model for automated image processing.

[0042] After generating pseudo-color images of the target detection objects, these images can be used as a baseline data source for segmentation to obtain multiple inference-based image data.

[0043] S140. Based on the inference image data, perform defect identification and classification to obtain the appearance defect detection result of the target detection object.

[0044] After processing and generating the inference image data of the target detection object, the inference image data can first be input into the defect recognition model to generate the defect recognition result of the target detection object. Further, the defect recognition result of the target detection object is input into the defect classification model for classification. Through the logical correlation of multiple models, the appearance defects of the target detection object can be accurately identified and classified. It is understood that the defect recognition model and the defect classification model can be pre-trained using sample data of the target detection object before use.

[0045] In an optional embodiment of the present invention, the target detection object may include a power battery, and the acquired image of the target detection object may be an image of the surface of the power battery pack. In the manufacturing process of power battery packs, the production quality of each component of the power battery pack affects the safety of the finished power battery product. Defects on the surface of the power battery pack can also become significant safety hazards. Therefore, the detection of appearance defects in power battery packs is increasingly becoming a crucial step in the power battery production process.

[0046] In the scenario of detecting appearance defects in power battery packs, advanced optical hardware can be used to acquire surface images of the power battery packs. Combined with the image-based appearance defect detection method described above, and employing computer vision detection technology, various defects such as scratches, damage, impurities, and foreign objects generated during the power battery production process can be automatically identified through training with a small number of samples. This allows for the timely elimination of safety hazards, ensuring that products meet safety standards and preventing safety accidents or performance degradation caused by quality defects. Furthermore, since the pseudo-color images used for appearance defect detection are synthesized from images of the target object acquired at different exposures from the same viewpoint, the pseudo-color images generated from different types of target objects in different application scenarios possess unified detection parameters for morphological defects. The synthesized pseudo-color images effectively capture complex or abstract defect patterns, thereby improving defect detection accuracy under non-uniform lighting conditions, and significantly enhancing the versatility and adaptability of the detection method. Therefore, the image-based appearance defect detection method described above can effectively solve the problems of human fatigue and inconsistent standards in manual visual inspection, reducing rework and production line interruptions caused by human error. It can also address the challenges of traditional image processing-based defect detection methods, such as reliance on prior knowledge, weak processing capabilities in complex scenes, and difficulties in scene transfer.

[0047] Applying the above image-based appearance defect detection method to the target inspection object can monitor the product quality of the target inspection object in real time, quickly provide feedback on quality problems, facilitate rapid adjustment of process parameters, reduce batch defective products, improve the overall product quality and reliability, and thus enhance market competitiveness.

[0048] This invention acquires images of a target object under different exposures from the same viewpoint, and then synthesizes these images into a pseudo-color image of the target object. Further, the pseudo-color image is processed to generate inference image data, which is then used for defect identification and classification to obtain the appearance defect detection result of the target object. This technical solution improves the uniformity and feature clarity of defect imaging in the data source used for image inference detection by synthesizing images of the target object under different exposures from the same viewpoint into a pseudo-color image. It addresses the problems of low versatility and detection accuracy in existing image-based appearance defect detection methods, thereby improving the versatility and detection accuracy of image-based appearance defect detection methods.

[0049] Example 2

[0050] Figure 2This is a flowchart of an image-based appearance defect detection method provided in Embodiment 2 of the present invention. This embodiment is a specific implementation based on the above embodiment. In this embodiment, various specific optional implementation methods are given for synthesizing images of the target detection object acquired at different exposures from the same viewpoint into a pseudo-color image of the target detection object, and for generating inference image data based on the pseudo-color image of the target detection object. Correspondingly, such as Figure 2 As shown, the method in this embodiment may include:

[0051] S210. Acquire images of the target object at different exposures from the same viewpoint.

[0052] S220. Denoise the images of the target object acquired at different exposures from the same viewpoint to obtain denoised image data.

[0053] Among them, the denoised image data can be the image data obtained by denoising images of the target detection object acquired at different exposures from the same viewpoint.

[0054] To improve the quality of image data input to the model, after acquiring images of the target object at different exposures from the same viewpoint, image preprocessing can be performed first to improve the quality of these images, facilitating feature extraction by the subsequent deep learning model. During image preprocessing, denoising can be performed on the images of the target object at different exposures from the same viewpoint. Optionally, a combination of Gaussian filtering and bilateral filtering can be used to eliminate random noise in the images at different exposures while preserving defect edge information, generating denoised image data.

[0055] Gaussian filtering is a linear smoothing filtering technique that uses a Gaussian function to weighted average image pixel values. It achieves filtering by scanning the image with a template and calculating the weighted average of neighboring pixels to replace the center pixel value; the weights are determined by the Gaussian function. Gaussian filtering effectively eliminates random noise in images. Bilateral filtering is a non-linear image filtering technique that combines spatial proximity and pixel value similarity, smoothing images while preserving edge details. In this embodiment, Gaussian filtering and bilateral filtering can be used in conjunction. First, Gaussian filtering is applied to initial image datasets with different exposures to remove random noise, and then bilateral filtering is applied to smooth the image while preserving defect edges, ultimately generating denoised image data. This collaborative filtering approach can more effectively improve image quality and reduce the impact of noise on subsequent feature extraction and defect identification.

[0056] S230. Generate three-channel image data based on the exposure difference information of the denoised image data, and synthesize the three-channel image data to obtain a pseudo-color image of the target detection object.

[0057] Among them, three-channel image data can be image data composed of 3 channels.

[0058] Optionally, an image synthesis algorithm can be used to generate three-channel image data from the filtered denoised image data at different exposures according to their exposure difference information. Each channel of the three-channel image data can correspond to a certain exposure. Thus, multiple single-channel black and white images of the denoised image data at the same angle under different exposure conditions can be used as one dimension of the three-channel color image to generate a pseudo-color image.

[0059] In an optional embodiment of the present invention, the step of generating three-channel image data based on the exposure difference information of the denoised image data and synthesizing the three-channel image data to obtain a pseudo-color image of the target detection object may include: using the first exposure dimension single-channel image in the denoised image data as the first channel image data; using the second exposure dimension single-channel image in the denoised image data as the second channel image data; generating third channel image data based on the first channel image data and the second channel image data; and synthesizing the first channel image data, the second channel image data, and the third channel image data to obtain a pseudo-color image of the target detection object.

[0060] The first exposure dimension single-channel image can be a single-channel image with high exposure. The second exposure dimension single-channel image can be a single-channel image with low exposure. The third exposure dimension single-channel image can be a single-channel image with moderate exposure.

[0061] Specifically, the high-exposure single-channel image from different exposure images taken by the same camera in the denoised image data can be used as the first exposure dimension single-channel image, and this first exposure dimension single-channel image can be used as the first channel image data. Simultaneously, the low-exposure single-channel image from different exposure images taken by the same camera in the denoised image data can be used as the second exposure dimension single-channel image, and this second exposure dimension single-channel image can be used as the second channel image data. For single-channel images with moderate exposure, the first and second channel image data can be fused based on their corresponding exposure values, and the fused image with the correct exposure can be used as the third channel image data. Finally, the first, second, and third channel image data can be synthesized into a single color image to obtain a pseudo-color image of the target object with uniform and distinct defect imaging.

[0062] In an optional embodiment of the present invention, generating third channel image data based on the first channel image data and the second channel image data may include: calculating a first product value between the first channel image data and a scaling factor; calculating a reference coefficient of the scaling factor and calculating a second product value between the second channel image data and the reference coefficient; and calculating the sum of the first product value and the second product value to obtain the third channel image data.

[0063] Specifically, when fusing the first and second channel image data based on their corresponding exposure levels, a scaling factor for the target detection object can be determined first. Optionally, the scaling factor can vary depending on the product type, production environment, and specific application scenario of the target detection object. That is, the scaling factor can be dynamically configured based on the product type, production environment, and application scenario of the target detection object. After determining the scaling factor, the first product between the first channel image data and the scaling factor can be calculated, along with a reference factor for the scaling factor and a second product between the second channel image data and the reference factor. Finally, the sum of the first and second products can be calculated to achieve weighted fusion of high and low exposure levels, yielding the required third channel image data.

[0064] For example, when the scaling factor is When the proportionality coefficient is used, the reference coefficient can be... .

[0065] For example, a pseudo-color image of the target object can be generated based on the following formula:

[0066]

[0067] in, A pseudo-color image of the target object. This is the first channel image data corresponding to the high-exposure image. This is the second channel image data corresponding to the low-exposure image. This is the proportionality coefficient. This is the third channel image data.

[0068] S240. The image of the cropped region to be detected is obtained by cropping from the pseudo-color image of the target detection object, and the image of the cropped region to be detected is segmented using a redundant cropping method to obtain the inference image data.

[0069] The image of the cropped region to be detected can be a region image in a pseudo-color image that includes actual effective feature information.

[0070] Understandably, target inspection objects, such as battery components in pack manufacturing, are mostly transported by AGVs (Automated Guided Vehicles) during the production process. Therefore, due to limitations imposed by the placement and operating status of the target inspection objects, industrial cameras cannot accurately and repeatedly capture the surface areas of the target inspection objects during the acquisition of raw images. Consequently, the acquired raw images contain a large number of redundant areas, and inspecting these areas would result in a significant waste of computational resources. Therefore, to improve the efficiency of appearance defect detection, the effective detection area of ​​the pseudo-color image of the target inspection object can be located, determining the cropped area image to be inspected.

[0071] When locating the effective detection region and determining the cropping region image of a pseudo-color image of a target object, several optional implementation methods can be adopted. One option is to set a region detection prior box for areas with small fluctuations in the pseudo-color image, i.e., drawing the detection region on the pseudo-color image. Another option is to use a deep learning algorithm model to automatically locate and crop the regions with large fluctuations in the pseudo-color image, obtaining the corresponding cropping region image.

[0072] In an optional embodiment of the present invention, the step of cropping the image of the cropped region to be detected from the pseudo-color image of the target detection object may include: reducing the pseudo-color image of the target detection object by a set ratio to obtain a scaled pseudo-color image; inputting the scaled pseudo-color image into a detection region localization model to obtain the initial coordinates of the detection image region; and enlarging the initial coordinates of the detection image region by the set ratio to obtain the image of the cropped region to be detected.

[0073] The scaled pseudo-color image can be obtained by reducing the size of the pseudo-color image of the target object. The detection region localization model can be a deep learning algorithm model used for automatic localization and cropping of the pseudo-color image. The detection image region can be a valid image region in the pseudo-color image that can be used to detect appearance defects of the target object.

[0074] Before cropping the target region image from the pseudo-color image of the object to be detected using a deep learning algorithm model, a detection region localization model can be pre-trained. During the training of the detection region localization model, pseudo-color image samples of different shapes of the target object can be collected in batches. These samples are then scaled down according to a set ratio to a specified size, and the surface regions within the pseudo-color image sample data are labeled. The labeled sample data is then used to train and fine-tune the corresponding deep learning model to adapt it for the localization and detection of the surface regions of the target object.

[0075] After the detection region localization model is trained, the pseudo-color image of the target object used for inference is scaled down to a fixed size according to a set ratio, resulting in a scaled pseudo-color image. This scaled pseudo-color image is then input into the trained detection region localization model to initially detect the initial coordinates of the detection image region based on the scaled pseudo-color image. Further, the initial coordinates of the detection image region are then enlarged according to the set scaling ratio to obtain the actual coordinates of the target cropping region image within the pseudo-color image. Based on these actual coordinates, the target cropping region image is then cropped from the pseudo-color image, resulting in the target cropping region image.

[0076] After determining the image of the cropped region to be detected, since it is still a high-resolution image and cannot be directly used for training and inference of the deep learning model, the image of the cropped region to be detected can be used as a baseline to further segment it into fixed-size inference image data using redundant cropping. Assuming the number of redundant overlapping pixels between the inference image data during the segmentation of the cropped region to be detected is *a*, the coordinates of the s-coordinate point relative to the cropped region in the (i,j)th inference image data are calculated using the following formula:

[0077]

[0078]

[0079] Where i and j represent the row and column coordinates of pixels in the inference image data. Let represent the x-coordinate of the s-coordinate point in the (i,j)th inference image data relative to the image's cropping region to be detected. Let represent the ordinate of the s-coordinate point in the (i,j)th inference image data relative to the cropped region to be detected; Let represent the x-coordinate of the s-coordinate point in the (i,j)th inference image data in the segmented inference image data; Let represent the ordinate of the s-coordinate point in the (i,j)th inference image data in the segmented inference image data; The x-coordinate of the s-coordinate point in the cropping region to be detected is the image coordinate of the s-coordinate point. The s-coordinate is the vertical coordinate of the image at the point to be detected in the cropping region.

[0080] S250. Based on the inference image data, perform defect identification and classification to obtain the appearance defect detection result of the target detection object.

[0081] In an optional embodiment of the present invention, the step of performing defect identification and classification based on the inference image data to obtain the appearance defect detection result of the target detection object may include: identifying and cropping abnormal regions on the inference image data through a defect identification model to obtain an image cropping region; scaling the image cropping region to a target size to obtain a uniform image cropping region; and classifying defects in the uniform image cropping region through a defect classification model to obtain the appearance defect detection result of the target detection object.

[0082] The target size can be set according to actual needs, and the embodiments of the present invention do not limit the specific value of the target size. The uniform image cropping area can be an image cropping area with a uniform fixed size that includes abnormal defect information.

[0083] When performing defect identification and classification based on inference image data, anomaly regions on the inference image data can be identified first using a defect identification model. These anomalous regions are then cropped out and scaled to a fixed size using padding and scaling. This scaled anomalous image is then fed into a defect classification model for classification. The logical correlation between multiple models accurately identifies and classifies appearance defects. Both the defect identification and classification models can employ deep learning models. Deep learning feature extraction is a self-learning process by the algorithm model. This image preprocessing step focuses on merging the initial set of image data and then performing spatial processing to amplify the defect region features in the initial image set and filter out interfering region features.

[0084] Optionally, existing object detection models can be used as the foundation for defect recognition models, with improvements to the basic feature extraction module and loss function to further enhance the algorithm's defect recognition capabilities. Lightweight object detection models have fewer parameters and computational requirements, lower hardware resource demands, and faster operation. Optionally, the network structure of the object detection model can consist of four parts: an input module, a feature extraction backbone module, a feature fusion module, and an output module. This structural design enables the object detection model to effectively process image data and achieve high-precision object detection. Specifically, the feature extraction backbone module extracts initial image features from the input image, the feature fusion module further fuses feature information at different scales, and the output module generates the final defect detection result.

[0085] In an embodiment of the present invention, optionally, the convolutional neural structure in the feature extraction backbone module within the target detection model can be replaced with the self-attention structure of the Transformer. The self-attention structure directly captures the long-distance dependencies between any positions in the input sequence without relying on the local receptive field, thereby effectively extracting the initial features of the image.

[0086] Optionally, for the defect classification model, a pure convolutional neural network model can be used as the basic architecture, and a weighted loss function and a center loss function can be used to optimize the network's classification ability for imbalanced sample data. The formula for calculating the weighted loss function is as follows:

[0087]

[0088] in, This represents the weighted loss function. Let i be the weight of the i-th sample. The basic loss function is N, where N is the total number of samples.

[0089] The formula for calculating the center loss function is as follows:

[0090]

[0091] Where N is the sample size. Let be the feature vector of the i-th sample. Let i be the true class label of the i-th sample. For category Corresponding feature centers It is the square of the L2 norm.

[0092] Feature extraction and detection based on deep learning methods require training on large amounts of data. Therefore, it is necessary to construct corresponding sample datasets according to different tasks for training the defect recognition and defect classification models. The sample images of the defect recognition model involved in this invention can be image samples of the cropped detection region generated by locating and cropping through a detection region localization model. These image samples can be of a size that can be... or The image samples used by the defect classification model can be the defect region image samples obtained by identifying and cropping the image samples of the cropped region to be detected by the defect recognition model. The size of these image samples can be... or .

[0093] When constructing a training dataset from image samples, the collected images need to be cleaned to ensure that defects in the selected images are clearly identifiable and form a significant contrast with the background. This contrast helps the model learn the contrast between defects and the background, thereby improving the model's detection capability. If the defects are blurry or obscured by the background, such image samples may not provide sufficient information to effectively train the model.

[0094] Sample annotation is a crucial step in training deep learning models. The quality of annotated image samples directly impacts the model's accuracy and generalization ability. Therefore, it's essential to adhere to relevant annotation guidelines and specifications to ensure accuracy and consistency. Optionally, for defect recognition models implemented using object detection models, the corresponding image sample data can be labeled with bounding boxes. After annotating the image samples with bounding boxes, a txt (text file) label file is exported based on the annotated bounding boxes. An annotation information import tool can be used to import the annotation categories and the image samples to be annotated, and then each imported image sample is annotated sequentially to generate annotation label data. For defect classification models, the corresponding image sample data annotation uses a folder format, with image samples of the same type of defect stored in the same folder. After the above processing, the annotated image samples are organized to form an image sample set.

[0095] Furthermore, the two prepared image sample datasets with annotation information are uniformly divided into training, validation, and test sets in a 7:1:2 ratio. These sets are then used to train the improved defect recognition and defect classification models. Specifically, during the training of the defect recognition model, the optimizer can use SGD (Stochastic Gradient Descent), with a batch size of 128, a total of 300 iterations, early stopping of 100, an initial learning rate of 0.008, momentum of 0.95, a non-maximum threshold of 0.6, and other parameters using the default settings of the original network for the object detection model. During the training of the defect classification model, the optimizer can use AdamP (Adaptive Moment Estimation with Path-wise Adaptation, an improved version of the Adam optimizer), with a batch size of 48, a total of 200 iterations, an initial learning rate of 0.002, and other parameters using the default network settings. After training, the weights of the best-performing model are saved in .pt format (PyTorch, a binary file used by the PyTorch framework to save and load model weights, parameters, and optimizer states). For easier deployment and application, the .pt format weight file can be converted to ONNX (Open Neural Network Exchange) format.

[0096] After model training, images of the target object, such as a power battery pack, can be acquired in real time from the same viewpoint at different exposure levels. These images are then processed to generate corresponding inference image data, which is then input into the defect recognition model to identify abnormal regions in the images. The abnormal regions are then cropped and padded to a fixed size before being fed into the defect classification model to determine whether they are defects and their type. The defect category is obtained through inference from both models.

[0097] Based on the defect identification category results and the measured dimensions of the defects, the final detection result information is output. If the abnormal area in the image is determined to be a defect, and the obtained category information corresponds to the defect type, the numerical value of the quantitative index corresponding to the defect category can be calculated in subsequent processes. The quantitative index corresponding to the defect category may include, but is not limited to, the area, length, width, and number of defects. When the defect category and the quantitative index meet the set specifications, information indicating an abnormal appearance of the target object is output; if the set specifications are not met, information indicating a normal appearance of the target object is output.

[0098] The aforementioned technical solution synthesizes a pseudo-color image of the target object by combining initial images of the target object captured by the camera under different exposure conditions from the same viewpoint using an image synthesis algorithm. This amplifies the defect region features in the initial image set and filters out interference region features. By setting prior bounding boxes and combining them with a detection region localization model, the target cropping region image is located in the image, reducing redundant information in the target cropping region image and lowering computational resource consumption. In the feature extraction backbone module of the defect recognition algorithm model based on the target detection model, the self-attention structure of the Transformer is used to replace the convolutional neural network structure, which improves the feature extraction capability of the surface image. The defect classification model improves the network's ability to classify imbalanced sample data by using a weighted loss function and a center loss function. The improved defect recognition model enhances the ability to identify image defects, and all suspected defects can be detected; while the combined defect classification model can determine whether abnormal data is a defect and output the defect category. In summary, the image-based appearance defect detection method described above utilizes deep learning computer vision detection technology. It eliminates the need for frequent parameter adjustments during product switching to adapt to changes in ambient light, enabling real-time and accurate identification of minute defects generated during the production process. This improves the accuracy of appearance defect detection, its processing capability in complex scenarios, and its scene migration capability.

[0099] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant regions.

[0100] It should be noted that any arrangement or combination of the technical features in the above embodiments also falls within the protection scope of this invention.

[0101] Example 3

[0102] Figure 3 This is a schematic diagram of an image-based appearance defect detection device provided in Embodiment 3 of the present invention, as shown below. Figure 3 As shown, the device includes: an exposure acquisition image acquisition module 310, a pseudo-color image synthesis module 320, an inference image data generation module 330, and an appearance defect detection result acquisition module 340, wherein:

[0103] The exposure acquisition image module 310 is used to acquire images of the target detection object at different exposures from the same viewpoint.

[0104] The pseudo-color image synthesis module 320 is used to synthesize images of the target detection object acquired at different exposures from the same viewpoint into a pseudo-color image of the target detection object.

[0105] The inference image data generation module 330 is used to generate inference image data based on the pseudo-color image of the target detection object;

[0106] The appearance defect detection result acquisition module 340 is used to identify and classify defects based on the inference image data to obtain the appearance defect detection result of the target detection object.

[0107] This invention acquires images of a target object under different exposures from the same viewpoint, and then synthesizes these images into a pseudo-color image of the target object. Further, the pseudo-color image is processed to generate inference image data, which is then used for defect identification and classification to obtain the appearance defect detection result of the target object. This technical solution improves the uniformity and feature clarity of defect imaging in the data source used for image inference detection by synthesizing images of the target object under different exposures from the same viewpoint into a pseudo-color image. It addresses the problems of low versatility and detection accuracy in existing image-based appearance defect detection methods, thereby improving the versatility and detection accuracy of image-based appearance defect detection methods.

[0108] Optionally, the pseudo-color image synthesis module 320 is further configured to: perform denoising processing on images of the target detection object acquired at different exposures under the same viewpoint to obtain denoised image data; generate three-channel image data based on the exposure difference information of the denoised image data; and synthesize the three-channel image data to obtain a pseudo-color image of the target detection object.

[0109] Optionally, the pseudo-color image synthesis module 320 is further configured to: use the first exposure dimension single-channel image in the denoised image data as the first channel image data; use the second exposure dimension single-channel image in the denoised image data as the second channel image data; generate third channel image data based on the first channel image data and the second channel image data; and synthesize the first channel image data, the second channel image data, and the third channel image data to obtain a pseudo-color image of the target detection object.

[0110] Optionally, the pseudo-color image synthesis module 320 is further configured to: calculate a first product value between the first channel image data and the scaling factor; calculate a reference coefficient of the scaling factor and calculate a second product value between the second channel image data and the reference coefficient; calculate the sum of the first product value and the second product value to obtain the third channel image data.

[0111] Optionally, the inference image data generation module 330 is further configured to: crop out a cropped region image to be detected from the pseudo-color image of the target detection object; and segment the cropped region image to be detected using a redundant cropping method to obtain the inference image data.

[0112] Optionally, the inference image data generation module 330 is further configured to: reduce the pseudo-color image of the target detection object by a set ratio to obtain a scaled pseudo-color image; input the scaled pseudo-color image into the detection area localization model to obtain the initial coordinates of the detection image area; and enlarge the initial coordinates of the detection image area by the set ratio to obtain the image of the cropped area to be detected.

[0113] Optionally, the target detection object includes a power battery; the acquired image of the target detection object is a surface image of the power battery pack.

[0114] Optionally, the appearance defect detection result acquisition module 340 is further configured to: identify and crop abnormal regions on the inference image data through a defect recognition model to obtain an image cropping region; scale the image cropping region to a target size to obtain a unified image cropping region; and classify defects in the unified image cropping region through a defect classification model to obtain the appearance defect detection result of the target detection object.

[0115] The image-based appearance defect detection device described above can execute the image-based appearance defect detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the image-based appearance defect detection method provided in any embodiment of the present invention.

[0116] Since the image-based appearance defect detection device described above is an apparatus capable of executing the image-based appearance defect detection method in the embodiments of the present invention, those skilled in the art can understand the specific implementation and various variations of the image-based appearance defect detection device in this embodiment based on the image-based appearance defect detection method described in the embodiments of the present invention. Therefore, how the image-based appearance defect detection device implements the image-based appearance defect detection method in the embodiments of the present invention will not be described in detail here. Any apparatus used by those skilled in the art to implement the image-based appearance defect detection method in the embodiments of the present invention falls within the scope of protection of this application.

[0117] Example 4

[0118] Figure 4 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0119] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0120] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0121] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image-based appearance defect detection methods.

[0122] Optionally, the image-based appearance defect detection method may include: acquiring images of the target object under different exposures from the same viewpoint; combining the images of the target object under different exposures from the same viewpoint into a pseudo-color image of the target object; generating inference image data based on the pseudo-color image of the target object; and performing defect identification and classification based on the inference image data to obtain the appearance defect detection result of the target object.

[0123] In some embodiments, the image-based appearance defect detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image-based appearance defect detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the image-based appearance defect detection method by any other suitable means (e.g., by means of firmware).

[0124] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0126] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0128] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0129] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the embodiments of the present invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in the embodiments of the present invention can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for detecting appearance defects based on images, characterized in that, include: Acquire images of the target object at different exposures from the same viewpoint; The images of the target object acquired at different exposures from the same viewpoint are combined to form a pseudo-color image of the target object. Generate inference image data based on the pseudo-color image of the target detection object; Based on the inference image data, defect identification and classification are performed to obtain the appearance defect detection result of the target detection object; The step of combining images of the target object acquired at different exposures from the same viewpoint into a pseudo-color image of the target object includes: Denoising processing is performed on images of the target object acquired at different exposures from the same viewpoint to obtain denoised image data; Based on the exposure difference information of the denoised image data, three-channel image data is generated, and the three-channel image data is synthesized to obtain a pseudo-color image of the target detection object; The step of generating three-channel image data based on the exposure difference information of the denoised image data, and synthesizing the three-channel image data to obtain the pseudo-color image of the target detection object, includes: The first exposure dimension single-channel image in the denoised image data is taken as the first channel image data; The second exposure dimension single-channel image in the denoised image data is used as the second channel image data; Generate third channel image data based on the first channel image data and the second channel image data; A pseudo-color image of the target object is obtained by combining the first channel image data, the second channel image data, and the third channel image data.

2. The image-based appearance defect detection method according to claim 1, characterized in that, The step of generating third channel image data based on the first channel image data and the second channel image data includes: Calculate the first product value between the first channel image data and the scaling factor; Calculate the reference coefficient of the scaling factor, and calculate the second product value between the second channel image data and the reference coefficient; The sum of the first product value and the second product value is calculated to obtain the third channel image data.

3. The image-based appearance defect detection method according to claim 1, characterized in that, The step of generating inference image data based on the pseudo-color image of the target detection object includes: The image of the cropped region to be detected is obtained by cropping from the pseudo-color image of the target object; The inference image data is obtained by segmenting the image of the cropped region to be detected using a redundant cropping method.

4. The image-based appearance defect detection method according to claim 3, characterized in that, The step of cropping the image of the cropped region to be detected from the pseudo-color image of the target detection object includes: The pseudo-color image of the target detection object is reduced by a set ratio to obtain a scaled pseudo-color image; The scaled pseudo-color image is input into the detection region localization model to obtain the initial coordinates of the detection image region; The initial coordinates of the detected image region are magnified according to the set ratio to obtain the image of the cropped region to be detected.

5. The image-based appearance defect detection method according to claim 1, characterized in that, The step of performing defect identification and classification based on the inference image data to obtain the appearance defect detection result of the target detection object includes: Abnormal regions on the inference image data are identified and cropped using a defect identification model to obtain the image cropping region; The image cropping area is scaled to the target size to obtain a uniform image cropping area; The defect classification model is used to classify defects in the cropped region of the unified image to obtain the appearance defect detection result of the target object.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that is executed by the at least one processor to enable the at least one processor to perform the image-based appearance defect detection method according to any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image-based appearance defect detection method according to any one of claims 1-5.

8. A computer program product, characterized in that, Includes a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the image-based appearance defect detection method according to any one of claims 1-5.

Images