Image Defect Detection Method, Device, Macro Imaging Device and Storage Medium

Abstract

The present application discloses an image defect detection method, apparatus, macro imaging device, and storage medium, relating to image processing. The method includes: obtaining a plurality of 2.5-dimensional images corresponding to the image to be detected; obtaining image patches respectively corresponding to each 2.5-dimensional image, and performing defect detection on each image patch respectively based on a target detection model to obtain sub-defect results corresponding to each image patch; the target detection model is obtained by improving the model parameters of a basic detection model; performing splicing processing on the sub-defect results corresponding to each image patch to obtain a defect detection result of the image to be detected. The present application can improve the problem of low accuracy in defect feature recognition existing in the traditional solution, achieving the beneficial effects of improving the efficiency and accuracy of defect detection and meeting the requirements for high-precision defect detection in modern industrial production.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to an image defect detection method, apparatus, macro imaging device, and storage medium. Background Art

[0002] In modern industrial production and the manufacturing process of various products, it is crucial to detect defects on the product surface. Traditional defect detection methods, such as manual visual inspection, rely on manual experience, which not only has low efficiency but is also easily affected by the subjective factors of the inspectors, resulting in unstable detection accuracy and difficulty in meeting the requirements of large-scale and high-precision production.

[0003] With the development of computer vision technology, image-based defect detection methods have gradually emerged. Traditional image-based defect detection mainly relies on simple image processing algorithms, such as edge detection, threshold segmentation, etc. These methods have a certain detection ability for simple and obvious-featured defects. However, with the increasing complexity of product manufacturing processes, there are a variety of defect types and complex features on the product surface, and traditional image processing algorithms are difficult to accurately and quickly detect defect features. Summary of the Invention

[0004] This application provides an image defect detection method, apparatus, macro imaging device, and storage medium, which can improve the efficiency and accuracy of defect detection.

[0005] In a first aspect, this application provides an image defect detection method, including: obtaining a plurality of 2.5-dimensional images corresponding to the image to be detected; obtaining image blocks corresponding to each of the 2.5-dimensional images, and respectively performing defect detection on each of the image blocks based on a target detection model to obtain sub-defect results corresponding to each of the image blocks; the target detection model is obtained by improving the model parameters of a basic detection model; performing splicing processing on the sub-defect results corresponding to each of the image blocks to obtain a defect detection result of the image to be detected.

[0006] Optionally, the obtaining image blocks corresponding to each of the 2.5-dimensional images includes: obtaining slicing parameters, where the slicing parameters include a slicing length and a slicing width; performing slicing processing on each of the 2.5-dimensional images according to the slicing length and the slicing width to obtain a plurality of image blocks corresponding to each of the 2.5-dimensional images.

[0007] Optionally, before defect detection is performed on each of the image patches based on the target detection model, the method further includes: performing foreground and background separation processing on each of the image patches to obtain separation features corresponding to each of the image patches; determining the image patches containing defect features in the separation features as candidate image patches, and performing gray gradient calculation on each of the candidate image patches to obtain target image patches.

[0008] Correspondingly, performing defect detection on each of the image patches based on the target detection model to obtain sub-defect results corresponding to each of the image patches includes: performing detection on each of the target image patches based on the target detection model to obtain sub-defect results corresponding to each of the target image patches.

[0009] Optionally, the method for performing splicing processing on the sub-defect results corresponding to each of the image patches to obtain the defect detection result of the image to be detected includes: obtaining an image identifier corresponding to each of the image patches; performing splicing processing on the sub-defect results corresponding to each of the image patches according to the image identifier to obtain an initial detection result of the image to be detected; and performing fusion processing on the initial detection result to obtain the defect detection result of the image to be detected.

[0010] Optionally, obtaining a plurality of 2.5-dimensional images corresponding to the image to be detected includes: obtaining an original image of a target object, performing image preprocessing on the original image to obtain the image to be detected; and performing transformation processing on the image to be detected based on a preset algorithm to obtain a plurality of the 2.5-dimensional images.

[0011] Optionally, the basic detection model is obtained in the following manner: obtaining a basic training sample set, where the basic training sample set includes a plurality of basic training samples, and each of the basic training samples includes a defect label; determining a loss function according to the type of defect to be detected; inputting the plurality of basic training samples into a deep learning network to obtain a prediction result corresponding to each of the basic training samples; determining the value of the loss function according to the prediction result and the defect label corresponding to each of the basic training samples; and obtaining the basic detection model when the value of the loss function reaches a preset value.

[0012] Optionally, the basic detection model includes frozen parameters and adaptation parameters; the target detection model is obtained by the following method: performing label verification on the basic training samples based on a confidence learning method to obtain optimized training samples; inputting the optimized training samples into the basic detection model, so that the frozen parameters in the basic detection model remain unchanged, training the adaptation parameters, and obtaining an intermediate detection model when the training result reaches the convergence condition; obtaining a distillation loss function and a defect type loss function; training the intermediate detection model based on the optimized training samples, and obtaining the target detection model when the sum of the distillation loss function and the defect type loss function reaches a preset threshold.

[0013] In a second aspect, the present application provides an image defect detection device, which includes: an image acquisition module for acquiring a plurality of 2.5-dimensional images corresponding to the image to be detected; a defect detection module for acquiring image blocks corresponding to each of the 2.5-dimensional images, and performing defect detection on each of the image blocks based on a target detection model to obtain sub-defect results corresponding to each of the image blocks; the target detection model is obtained by improving the model parameters of a basic detection model; a defect processing module for performing stitching processing on the sub-defect results corresponding to each of the image blocks to obtain a defect detection result of the image to be detected.

[0014] In a third aspect, the present application further provides a macro imaging device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the image defect detection method according to any embodiment of the present application.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores computer instructions for causing a processor to implement the image defect detection method according to any embodiment of the present application when executed.

[0016] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the image defect detection method according to any embodiment of the present application when executed by a processor.

[0017] The image defect detection solution provided by the embodiments of this application first obtains multiple 2.5D images corresponding to the image to be detected, and displays the multi-dimensional information of the image to be detected through the 2.5D images; then, based on the object detection model, defect detection is performed on the image blocks corresponding to each 2.5D image respectively, and sub-defect results corresponding to each image block are obtained, so that the model can analyze the local features of the image more carefully in a block-by-block processing manner; in this embodiment, the object detection model is obtained by improving the model parameters of the basic detection model; this improvement method can be optimized for specific 2.5D image data and defect detection tasks, so that the model can better learn and extract defect features in 2.5D images; finally, by splicing the sub-defect results corresponding to each image block to obtain the defect detection result of the image to be detected, a complete and accurate defect detection result can be obtained. This improves the problem of low accuracy in defect feature recognition in the traditional solution, achieves the beneficial effects of improving the efficiency and accuracy of defect detection, and meets the requirements of high-precision defect detection in modern industrial production.

[0018] It should be noted that the above computer instructions can be stored in whole or in part on a computer-readable storage medium. Among them, the computer-readable storage medium can be packaged together with the processor of the image defect detection device, or can be packaged separately from the processor of the image defect detection device. This application does not make any limitations in this regard.

[0019] The descriptions of the second, third, fourth, and fifth aspects in this application can refer to the detailed description of the first aspect; and, for the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects, reference can be made to the analysis of the beneficial effects of the first aspect, which will not be elaborated here.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of this application, nor is it used to limit the scope of this application. Other features of this application will become easily understood through the following description.

[0021] It can be understood that before using the technical solutions disclosed in the embodiments of this application, the types, usage scopes, usage scenarios, etc. of the personal information involved in this application should be informed to users and the authorization of users should be obtained in an appropriate manner in accordance with relevant laws and regulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of this application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic flowchart of an image defect detection method provided by an embodiment of the present application.

[0024] Figure 2 It is a schematic structural diagram of an image defect detection device provided by an embodiment of the present application.

[0025] Figure 3 It is a schematic structural diagram of a macro imaging device provided by an embodiment of the present application. Detailed implementation manners

[0026] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the present application will be clearly and completely described below in conjunction with the accompanying drawings in this embodiment. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limiting the present application. Additionally, it should be noted that for the sake of convenience of description, only parts related to the present application rather than all the structures are shown in the accompanying drawings.

[0029] Figure 1 It is a schematic flowchart of an image defect detection method provided by an embodiment of the present application. This embodiment is applicable to the situation of defect detection of an object image captured by a macro imaging device. This method can be executed by an image defect detection device, and this device can be implemented in the form of hardware and / or software and integrated in the macro imaging device that executes this method.

[0030] Refer to Figure 1, the image defect detection method of this embodiment includes but is not limited to the following steps:

[0031] S110. Obtain multiple 2.5D images corresponding to the image to be detected.

[0032] The image to be detected is an image that needs to be subjected to defect detection. It can come from various image acquisition devices, such as cameras, scanners, or macro imaging devices, etc.; the image to be detected can also cover different application scenarios, such as multiple fields including lithium batteries, 3C semiconductors, and automotive manufacturing, etc., taking the collected image data as the source of the basic training samples.

[0033] Preferably, the solution provided in this embodiment is integrated in the scenario of defect detection based on a macro imaging device. The reason for this is that the traditional method of image defect detection based on a macro camera generally is to first obtain image data at different angles by shooting an object with a macro camera, then transmit the image data to a personal computer (abbreviated as PC) terminal, and finally use the defect detection algorithm software integrated in the PC to perform defect recognition. This processing method is limited by the PC hardware configuration and interface bandwidth, and it is difficult to process a large amount of high-resolution image data in real time, resulting in low data processing efficiency. In view of this, by integrating the image defect detection method provided in this embodiment in the macro imaging device, this device can achieve efficient image data processing, quickly obtain 2.5D imaging sequence images, and achieve rapid analysis of defects through the target detection model carried, achieving the beneficial effects of one-key real-time high-precision defect detection, improving the automation level, and reducing the burden on the operator.

[0034] A 2.5D image is an image representation form between two-dimensional (2D) and three-dimensional (3D). Compared with traditional 2D images, 2.5D images not only contain the planar texture information of the object, but also can reflect the depth or height information of the object to a certain extent. This multi-dimensional information can more comprehensively describe the condition of the object surface, enabling the target detection model to obtain richer features. For example, for some objects with defects such as concavities, convexities, and holes on the surface, 2.5D images can more clearly present the three-dimensional forms of these defects, enabling the model to more accurately identify and locate the defects, effectively avoiding the situation of missed or misdetected defects caused by only using 2D images, and greatly improving the accuracy of defect recognition.

[0035] In this embodiment, the method for obtaining multiple 2.5D images corresponding to the image to be detected may be as follows: obtaining the original image of the target object, performing image preprocessing on the original image to obtain the image to be detected; and performing transformation processing on the image to be detected based on a preset algorithm to obtain multiple 2.5D images. By performing image preprocessing on the original image in this embodiment, noise can be removed, contrast can be adjusted, etc., so that the quality of the image to be detected is significantly improved. This processing method helps to improve the accuracy of subsequent analysis and reduce misjudgment and missed detection caused by image quality problems; and based on the 2.5D image, the depth information of the image is increased, which helps to understand the surface characteristics of the target object. For some objects with complex surface structures, such as objects with defects such as concavities, convexities, and holes, the 2.5D image can present these characteristics more clearly, thereby improving the accuracy and reliability of defect detection.

[0036] Among them, the above-mentioned target object refers to the specific object that needs to perform operations such as defect detection and feature analysis. Such as components in industrial production (such as automobile engine parts, electronic chips, etc.); the original image refers to the initial image data obtained by shooting or scanning the target object through an image acquisition device. These images may contain various problems such as noise, uneven light, and blurring, and their quality and format may vary depending on the acquisition device and environment.

[0037] In this embodiment, when performing image preprocessing on the original image to obtain the image to be detected, the current image to be detected may be an image obtained by processing the original image using a filtering algorithm, a grayscale processing algorithm, etc. The purpose of doing this is to facilitate highlighting the defect features in the image and facilitating subsequent processing operations.

[0038] Furthermore, in this embodiment, the method for performing transformation processing on the image to be detected based on a preset algorithm to obtain multiple 2.5D images may be as follows: using a structured light projector to project a known pattern (such as stripes, grids, etc.) onto the target object, and a macro imaging device to capture the pattern modulated by the object surface. By analyzing the captured image, calculating the deformation of the pattern, and further obtaining the depth information of the object surface. For example, Phase Measuring Profilometry (PMP) calculates the phase of the stripe pattern to obtain the height information of the object surface and generates a 2.5D image, etc.

[0039] In this embodiment, there are multiple 2.5D images, such as the shape map, texture map, height (depth) map, and glossiness comparison map generated for the image to be detected, etc.

[0040] S120. Obtain the image blocks corresponding to each 2.5D image respectively, and perform defect detection on each image block based on the target detection model to obtain the sub-defect results corresponding to each image block.

[0041] An image block is an image area obtained by dividing a 2.5D image according to certain rules. The selected small image areas after division correspond to the image blocks of the 2.5D image respectively. Each image block contains a part of the information of the original 2.5D image, such as planar texture information and corresponding depth information.

[0042] For the solution provided in this embodiment, the method for obtaining the image blocks corresponding to each 2.5D image can be: obtaining cutting parameters, where the cutting parameters include the cutting length and the cutting width; performing cutting processing on each 2.5D image according to the cutting length and the cutting width to obtain multiple image blocks corresponding to each 2.5D image respectively.

[0043] Specifically, first, according to factors such as actual detection requirements, the characteristics of the target detection model, and the pixel parameters of the 2.5D image, appropriate cutting parameters, that is, the cutting length and the cutting width, are determined. For example, if the target detection model has better processing effects for smaller-sized image blocks and the defects to be detected are usually tiny defects, then smaller cutting lengths and widths can be set, such as 50×50 pixels; conversely, if the defects in the image are larger and the model can process larger-sized image blocks, then the size of the cuts can be appropriately increased, such as 200×200 pixels. The specific limitation of the cutting parameters is not restricted here.

[0044] During the cutting process based on the cutting parameters, to ensure that the cutting window completely covers the image area, if the cutting window exceeds the image boundary at the edge of the image, a filling method (such as filling with 0 values or the mean value of the image, etc.) can be used to supplement the missing part so that the cutting process can be completed completely. Each time a cut is made to obtain an image block, it is saved until the entire 2.5D image is divided into multiple qualified image blocks. In this way, multiple image blocks corresponding to each 2.5D image can be obtained.

[0045] In this embodiment, by setting the cutting parameters, the size of the image blocks can be flexibly adjusted according to different application scenarios and detection requirements; and reasonable cutting processing can improve the detection efficiency to a certain extent. For example, when performing feature extraction and analysis on smaller image blocks, the computational load is relatively small, which can speed up the inference speed of the model; and cutting processing according to fixed cutting parameters can ensure that all parts of the image are included in the image blocks for analysis, avoiding the omission of important information caused by unreasonable cutting methods. At the same time, by setting appropriate cutting parameters, while ensuring coverage of the image, redundant information between image blocks can be reduced, improving the efficiency of data processing.

[0046] Furthermore, in this embodiment, by using a target detection model to perform defect detection on each image patch separately to obtain sub-defect results corresponding to each image patch, the presence or absence of defects and related defect information (such as defect type, location, size, etc.) in the corresponding image patch are described by the defect results. These sub-defect results are the basis for subsequent defect analysis of the entire image to be detected.

[0047] In this embodiment, the target detection model is obtained by improving the model parameters of the basic detection model. Specifically, for the target detection model pre-trained by the basic detection model, during the training process, it adjusts the model parameters according to a large number of image patches with accurate defect annotations, so that the model can accurately identify and classify various defects in the image patches. After training, the model is evaluated and optimized to ensure that its performance meets the requirements of defect detection. Finally, the divided image patches are sequentially input into the target detection model, and the model extracts and analyzes the features of the image patches, and judges whether there are defects in the image patches according to the learned image features. If there are defects, further determine information such as the type of defect (such as scratches, cracks, holes, etc.), location (coordinate position in the image patch), and size (such as defect area, length, etc.), and generate sub-defect results corresponding to each image patch.

[0048] Furthermore, since not necessarily every image patch contains defect features, therefore, in the solution provided in this embodiment, before performing defect detection on each image patch based on the target detection model, the following operations can also be performed: perform foreground and background separation processing on each image patch separately to obtain separation features corresponding to each image patch; determine the image patches containing defect features in the separation features as candidate image patches, and perform gray gradient calculation on each candidate image patch to obtain target image patches. In this embodiment, by separating the foreground and background and screening candidate image patches, a large number of image patches that do not contain defect features are excluded, reducing the amount of data to be input into the target detection model and improving the detection efficiency; further performing gray gradient calculation on the candidate image patches, and selecting the image patches with obvious gray changes as target image patches. These target image patches are more likely to contain defects, can provide more valuable inputs for the target detection model, help the model more accurately identify and locate defects, thereby improving the accuracy of defect detection and reducing the cases of false detection and missed detection.

[0049] The purpose of performing foreground and background separation processing on each image block is that in the defect detection of a macro camera, defects are usually located in the foreground part. Through foreground and background separation, defects can be highlighted from the complex background, making the features of the target defects more obvious, such as features like shape, texture, color, etc., which is convenient for subsequent analysis and processing. For example, for the tiny cracks on a circuit board, the trend and width of the cracks can be seen more clearly after separation; and it can remove some irrelevant information in the background, such as the outer shell of the device, the surrounding environmental light and shadow, etc., which may interfere with defect detection. Therefore, after the separation processing, the separated features corresponding to each image block can be obtained, and the current separated features can be defect features.

[0050] Furthermore, for the image blocks whose separated features do not contain defects, no subsequent processing operations need to be performed to reduce the processing pressure on the target detection model. The image blocks containing defect features are used as candidate image blocks, and thus the gray gradient is calculated for each candidate image block respectively. According to the calculated gray gradient amplitude, a suitable threshold is set, and the candidate image blocks with a gradient amplitude greater than the threshold are determined as target image blocks. These target image blocks are considered to have a high possibility of containing defects and will be used for subsequent defect detection based on the target detection model.

[0051] Preferably, in this embodiment, the specific implementation manner of performing defect detection on each image block based on the target detection model to obtain the sub-defect results corresponding to each image block can be: performing detection on each target image block based on the target detection model to obtain the sub-defect results corresponding to each target image block. Thus, the purpose of improving the defect detection efficiency is achieved.

[0052] Another preferred implementation manner is that the basic detection model in this embodiment is obtained through the following steps:

[0053] a) Obtain a basic training sample set, which contains multiple basic training samples, and each basic training sample includes a defect label.

[0054] Collect image data related to the defects to be detected from multiple different sources. These sources can include product images on the actual production line, images in historical detection records, images generated by simulation experiments, etc. For example, in the defect detection of industrial parts, images of parts in different production stages and different production batches can be collected to ensure the diversity and representativeness of the data; further add defect labels to each collected basic training sample. Accurately describe information such as the type of defect (such as scratches, cracks, holes, etc.), location (which can be marked using bounding boxes, pixel coordinates, etc.), and severity through the defect labels; finally, organize the labeled samples into a basic training sample set.

[0055] b) Determine the loss function according to the type of defect to be detected.

[0056] Before defect detection, clarify the types of defects to be detected and their characteristics. In this embodiment, different types of defects may require different loss functions for optimization. For example, for classification tasks (judging the type of defect), the cross-entropy loss function is usually used; for regression tasks (predicting continuous values such as the location and size of defects), the mean squared error loss function is commonly used.

[0057] c) Input multiple basic training samples into the deep learning network to obtain the prediction results corresponding to each basic training sample.

[0058] According to the characteristics and requirements of the defect detection task, select a suitable deep learning network architecture. The deep learning network in this embodiment can be a convolutional neural network.

[0059] d) Determine the value of the loss function according to the prediction results corresponding to each basic training sample and the defect labels.

[0060] For each basic training sample, substitute its prediction result and the corresponding defect label into the previously determined loss function to calculate the loss value of the sample. For example, when using the cross-entropy loss function for a classification task, calculate the cross-entropy loss of a single sample according to the predicted class probability distribution and the true label.

[0061] e) When the value of the loss function reaches a preset value, obtain the basic detection model.

[0062] Use an optimization algorithm (such as the stochastic gradient descent algorithm) to update the parameters of the deep learning network according to the gradient information of the loss function. The optimization algorithm will continuously adjust the parameters of the network to gradually reduce the value of the loss function. When the value of the loss function reaches the preset value, it is considered that the model has converged to a better state. At this time, stop the training and use the current deep learning network as the basic detection model.

[0063] Among them, the above preset value can be determined according to experience or through multiple experiments, and can also be combined with evaluation metrics (such as accuracy, recall, etc.) on the validation set to comprehensively judge whether the model has achieved satisfactory performance.

[0064] Furthermore, on the basis of obtaining the basic detection model, the basic detection model provided in this embodiment includes frozen parameters and adaptable parameters; among them, the frozen parameters are the parameters that are not updated and adjusted during the training stage in the basic detection model. Since the parameters usually acquire relatively general and stable feature representation capabilities through learning a large amount of data during the early training of the basic model, freezing can prevent these learned useful features from being overly changed during subsequent training, reducing the number of training parameters and computational complexity; the adaptable parameters are the adjustable parameters in the basic detection model, and the adaptable parameters are used to enable the model to better adapt to specific tasks or data. During the subsequent training process, by optimizing the adaptable parameters, the model can learn specific features and patterns related to the current task, thereby improving the performance of the model on this task.

[0065] Specifically, the object detection model provided in this embodiment is obtained in the following manner:

[0066] f) Perform label verification on the basic training samples based on the confidence learning method to obtain optimized training samples.

[0067] The method of performing label verification on the basic training samples based on the confidence learning method can be: by comparing and analyzing the prediction results of the model on the basic training samples with the original labels, calculate the confidence score of each sample label. The confidence score can reflect the reliability of the sample label, so that more reliable samples can be selected for subsequent training to improve the quality of the training data; by setting a confidence threshold, filter out the basic training samples with confidence scores lower than the confidence threshold to obtain optimized training samples.

[0068] g) Input the optimized training samples into the basic detection model, so that the frozen parameters in the basic detection model remain unchanged, train the adaptable parameters, and when the training result reaches the convergence condition, obtain an intermediate detection model.

[0069] Input the optimized training samples into the basic detection model. During the training process, calculate the loss function according to the prediction results and true labels of the optimized training samples, and adjust the adaptable parameters according to the gradient information of the loss function. During the training process, when it is monitored that the performance metrics of the model do not improve for multiple consecutive rounds, or the value of the loss function drops to a small range and tends to be stable, it is considered that the training result reaches the convergence condition, and at this time, an intermediate detection model is obtained.

[0070] h) Obtain the distillation loss function and the defect type loss function.

[0071] The distillation loss function is used to measure the output difference between the teacher model and the student model, prompting the student model to learn the knowledge and behavior of the teacher model, so as to achieve better performance with a smaller model size. Select an appropriate form of the distillation loss function according to the structure and training objectives of the intermediate detection model. According to the specific situation, a single distillation loss function can be selected or multiple distillation loss functions can be combined, and appropriate weights can be set for each loss function term.

[0072] The defect type loss function is a loss function defined according to the defect types to be detected. It is used to measure the difference between the prediction results of the model for different defect types and the true labels, guiding the model to learn the ability to accurately distinguish different defect types, so as to improve the accuracy of defect detection. For example, for a classification task, a multi-class cross-entropy loss function can be used; for a regression task, a mean squared error loss function can be used.

[0073] i) Train the intermediate detection model based on the optimized training samples. When the sum of the distillation loss function and the defect type loss function reaches a preset threshold, obtain the target detection model.

[0074] Input the optimized training samples into the intermediate detection model again. In each training batch, calculate the values of the distillation loss function and the defect type loss function simultaneously, and add them together to obtain the total loss function value; use an optimization algorithm to update the parameters of the intermediate detection model (which may include some frozen parameters and adaptation parameters that can be fine-tuned, depending on the settings) according to the gradient information of the total loss function to minimize the total loss function; during the training process, continuously monitor the sum of the distillation loss function and the defect type loss function. When this sum value reaches the preset threshold (the preset threshold can be determined through experiments or experience), it is considered that the model training has achieved the expected effect, and at this time, stop the training to obtain the target detection model.

[0075] The target detection model provided in this embodiment freezes some parameters in the basic detection model, reducing the number of parameters to be updated during the training process, reducing the computational complexity, and accelerating the training speed; and verifies the labels of the basic training samples through a confidence learning method, screening out optimized training samples with high confidence, reducing the interference of mislabeled or unreliable samples on model training, and improving the quality of training data; using the distillation loss function, it realizes the transfer of knowledge from the larger basic detection model to the intermediate detection model, which helps to maintain better performance with a smaller model size. At the same time, training in combination with the defect type loss function further optimizes the model's detection ability for defect types, improving the accuracy and reliability of defect detection.

[0076] S130. Stitch the sub-defect results corresponding to each image patch to obtain the defect detection result of the image to be detected.

[0077] For each sub - defect result corresponding to each obtained image patch, perform stitching according to certain rules to obtain the defect detection result corresponding to the original 2.5D image, that is, the defect detection result of the image to be detected. The current defect detection result at least includes: defect type, defect location, defect size, etc.

[0078] In a preferred implementation, the above - mentioned step S130 in the solution provided in this embodiment can be implemented in the following manner: Obtain the image identifier corresponding to each image patch; perform stitching processing on the sub - defect results corresponding to each image patch according to the image identifier to obtain the initial detection result of the image to be detected; perform fusion processing on the initial detection result to obtain the defect detection result of the image to be detected. Stitching the sub - defect results accurately through the image identifier avoids the detection result error caused by the disorder of the image patch positions. At the same time, operations such as removing duplicate detections and merging adjacent defects in the fusion processing can effectively eliminate false detections and duplicate annotations, making the detection result more accurately reflect the true defect situation in the image to be detected and reducing the probability of missed detections and false detections. Performing fusion processing on the initial detection result, especially merging adjacent defects and edge smoothing processing, makes the presentation of the defect area more coherent and natural, which helps the detection personnel more intuitively understand the distribution and shape of the defects and improves the ability to analyze and judge the defects.

[0079] When splitting the image to be detected into multiple image patches, assign a unique image identifier to each image patch. The current image identifier can be a digital number, arranged in sequence according to the position order of the image patches in the original image (for example, from left to right, from top to bottom); it can also be an encoding containing the coordinate information of the image patch (such as the upper - left corner coordinates) to accurately record the position of the image patch in the original image. After the object detection model performs defect detection on each image patch, it will obtain the sub - defect result corresponding to each image patch. These sub - defect results contain information about the defects within the image patch, such as the type, location, size, etc. of the defects.

[0080] According to the image identifier of each image patch, stitch their corresponding sub - defect results in the position order in the original image. Create a blank image with the same size as the original image to be detected as the basis for stitching. For the sub - defect result of each image patch, determine its position in the blank image according to its image identifier, and then fill the defect information in the sub - defect result (such as marking the defect area with a specific color or symbol) into the corresponding position of the blank image. If there is an overlapping part between the image patches, it is necessary to process the sub - defect results in the overlapping area according to the specific situation. For example, you can choose to retain the detection result with a higher confidence level, or comprehensively judge and merge the defect information in the overlapping area.

[0081] The methods for further fusing the initial detection results at least include smoothing the edges, merging adjacent defects, and smoothing the edges, etc., so as to obtain the defect detection result of the image to be detected. This result can be an image with defect annotations, or a data file containing defect information (such as the location, type, size, etc. of the defects), for subsequent analysis and decision-making.

[0082] The image defect detection method provided in this embodiment first obtains multiple 2.5D images corresponding to the image to be detected, and displays the multi-dimensional information of the image to be detected through the 2.5D images; then, based on the object detection model, defect detection is respectively performed on the image blocks corresponding to each 2.5D image to obtain the sub-defect results corresponding to each image block, so that the model can more carefully analyze the local features of the image in a block-by-block manner; in this embodiment, the object detection model is obtained by improving the model parameters of the basic detection model; this improvement method can be optimized for specific 2.5D image data and defect detection tasks, enabling the model to better learn and extract the defect features in the 2.5D images; finally, by splicing the sub-defect results corresponding to each image block to obtain the defect detection result of the image to be detected, a complete and accurate defect detection result can be obtained. This improves the problem of low accuracy in defect feature recognition in the traditional solution, achieving the beneficial effects of improving the efficiency and accuracy of defect detection and meeting the requirements for high-precision defect detection in modern industrial production.

[0083] Figure 2 It is a structural schematic diagram of an image defect detection device provided in an embodiment of the present application, and this device is applicable to execute the image defect detection method provided in the embodiment of the present application. As Figure 2 shown, this device may specifically include: an image acquisition module 210, a defect detection module 220, and a defect processing module 230.

[0084] Among them, the image acquisition module 210 is used to acquire multiple 2.5D images corresponding to the image to be detected.

[0085] The defect detection module 220 is used to acquire the image blocks corresponding to each of the 2.5D images, and respectively perform defect detection on each of the image blocks based on the object detection model to obtain the sub-defect results corresponding to each of the image blocks; the object detection model is obtained by improving the model parameters of the basic detection model.

[0086] The defect processing module 230 is used to splice the sub-defect results corresponding to each of the image blocks to obtain the defect detection result of the image to be detected.

[0087] The image defect detection device provided by the embodiment of the present application first obtains a plurality of 2.5-dimensional images corresponding to the image to be detected, and displays the multi-dimensional information of the image to be detected through the 2.5-dimensional images; then, based on the target detection model, defect detection is respectively performed on the image blocks corresponding to each 2.5-dimensional image to obtain sub-defect results corresponding to each image block, so that the model can more carefully analyze the local features of the image in a block-by-block processing manner; in this embodiment, the target detection model is obtained by improving the model parameters of the basic detection model; this improvement method can be optimized for specific 2.5-dimensional image data and defect detection tasks, so that the model can better learn and extract defect features in 2.5-dimensional images; finally, by splicing the sub-defect results corresponding to each image block, a defect detection result of the image to be detected can be obtained, and a complete and accurate defect detection result can be obtained. This improves the problem of low accuracy in defect feature recognition in the traditional solution, and achieves the beneficial effects of improving the efficiency and accuracy of defect detection and meeting the requirements of high-precision defect detection in modern industrial production.

[0088] In one embodiment, the defect detection module 220 includes a parameter acquisition unit and a block processing unit.

[0089] Among them, the parameter acquisition unit is used to acquire block parameters, and the block parameters include block length and block width.

[0090] The block processing unit is used to perform block processing on each 2.5-dimensional image according to the block length and the block width to obtain a plurality of image blocks corresponding to each 2.5-dimensional image respectively.

[0091] In one embodiment, the defect detection module 220 further includes a separation processing unit, a gradient calculation unit, and a defect detection unit.

[0092] Among them, the separation processing unit is used to perform foreground and background separation processing on each image block respectively to obtain separation features corresponding to each image block respectively.

[0093] The gradient calculation unit is used to determine the image blocks containing defect features in the separation features as candidate image blocks, and perform gray gradient calculation on each candidate image block respectively to obtain target image blocks.

[0094] The defect detection unit is used to perform detection on each target image block respectively based on the target detection model to obtain sub-defect results corresponding to each target image block.

[0095] In one embodiment, the defect processing module 230 is specifically configured to obtain the image identifier corresponding to each of the image blocks; splice the sub-defect results corresponding to each of the image blocks according to the image identifier to obtain an initial detection result of the image to be detected; and perform a fusion process on the initial detection result to obtain a defect detection result of the image to be detected.

[0096] In one embodiment, the image acquisition module 210 is specifically configured to obtain an original image of a target object, perform image preprocessing on the original image to obtain the image to be detected; and perform a conversion process on the image to be detected based on a preset algorithm to obtain a plurality of the 2.5-dimensional images.

[0097] In one embodiment, the basic detection model is obtained in the following manner: obtain a basic training sample set, the basic training sample set includes a plurality of basic training samples, and each of the basic training samples includes a defect label; determine a loss function according to the type of defect to be detected; input the plurality of basic training samples into a deep learning network to obtain a prediction result corresponding to each of the basic training samples; determine the value of the loss function according to the prediction result corresponding to each of the basic training samples and the defect label; and when the value of the loss function reaches a preset value, obtain the basic detection model.

[0098] In one embodiment, the basic detection model includes frozen parameters and adaptation parameters; the target detection model is obtained in the following manner: perform label verification on the basic training samples based on a confidence learning method to obtain optimized training samples; input the optimized training samples into the basic detection model so that the frozen parameters in the basic detection model remain unchanged, train the adaptation parameters, and when the training result reaches a convergence condition, obtain an intermediate detection model; obtain a distillation loss function and a defect type loss function; train the intermediate detection model based on the optimized training samples, and when the sum of the distillation loss function and the defect type loss function reaches a preset threshold, obtain the target detection model.

[0099] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the above-described functional modules can refer to the corresponding process in the foregoing method embodiments and will not be elaborated herein.

[0100] An embodiment of the present application also provides a macro imaging device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the image defect detection method according to any embodiment of the present application.

[0101] An embodiment of the present application also provides a computer-readable medium, and the computer-readable storage medium stores computer instructions for implementing the image defect detection method according to any embodiment of the present application when the computer instructions are executed by a processor.

[0102] Refer to the following Figure 3 , Figure 3 which is a schematic structural diagram of the macro imaging device provided by an embodiment of the present application. It shows a schematic structural diagram of a computer system 500 suitable for implementing the macro imaging device of the embodiment of the present application. Figure 3 The shown macro imaging device is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0103] As Figure 3 shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage section 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0104] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as required. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as required so that a computer program read from it can be installed into the storage section 508 as required.

[0105] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above functions defined in the system of the present application are executed.

[0106] It should be noted that the computer-readable medium shown in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. And in the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program codes. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program codes contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, and optical cable, etc., or any suitable combination of the above.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in a different order than that marked in the accompanying drawings. For example, two consecutive boxes shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as combinations of boxes in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0108] The modules and / or units involved in the embodiments described in the present application can be implemented in software or in hardware. The described modules and / or units can also be provided in a processor. For example, it can be described as: a processor includes an image acquisition module, a defect detection module, and a defect processing module. Among them, the names of these modules do not constitute a limitation to the module itself in some cases.

[0109] On the other hand, the present application also provides a computer-readable medium, which can be included in the device described in the above embodiments; or it can exist separately without being assembled into the device. The above computer-readable medium carries one or more programs. When the one or more programs are executed by the device, the device includes: obtaining a plurality of 2.5-dimensional images corresponding to the image to be detected; obtaining image patches corresponding to each of the 2.5-dimensional images, and respectively performing defect detection on each of the image patches based on a target detection model to obtain sub-defect results corresponding to each of the image patches; the target detection model is obtained by improving the model parameters of a basic detection model; performing stitching processing on the sub-defect results corresponding to each of the image patches to obtain a defect detection result of the image to be detected.

[0110] According to the technical solution of this embodiment, the problem of low accuracy in defect feature recognition in the traditional solution is improved, and the beneficial effects of improving the efficiency and accuracy of defect detection and meeting the high-precision defect detection requirements in modern industrial production are achieved.

[0111] The above specific embodiments do not constitute a limitation on the protection scope of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. An image defect detection method, characterized in that: include: Obtain multiple 2.5-dimensional images corresponding to the image to be detected; Obtaining image blocks corresponding to each of the 2.5-dimensional images, performing foreground and background separation processing on each of the image blocks, and obtaining separation features corresponding to each of the image blocks; Determine the image blocks containing defect features in the separation features as candidate image blocks, and perform grayscale gradient calculation on each of the candidate image blocks to obtain a target image block; Detect each of the target image blocks based on the target detection model to obtain a sub-defect result corresponding to each of the target image blocks; The target detection model is obtained by improving the model parameters of the basic detection model; Performing splicing processing on the sub-defect results corresponding to each of the image blocks to obtain a defect detection result of the image to be detected; The step of obtaining the image blocks corresponding to each of the 2.5-dimensional images comprises: Obtaining cutting parameters, wherein the cutting parameters include cutting length and cutting width; Each of the 2.5-dimensional images is sliced ​​according to the slice length and the slice width to obtain a plurality of image blocks corresponding to each of the 2.5-dimensional images.

2. The image defect detection method according to claim 1, characterized in that: The step of performing splicing processing on the sub-defect results corresponding to each of the image blocks to obtain the defect detection result of the image to be detected includes: Obtaining an image identifier corresponding to each of the image blocks; According to the image identifier, the sub-defect results corresponding to each of the image blocks are spliced ​​to obtain an initial detection result of the image to be detected; The initial detection results are fused to obtain defect detection results of the image to be detected.

3. The image defect detection method according to claim 1, characterized in that: The step of obtaining a plurality of 2.5-dimensional images corresponding to the image to be detected includes: Acquire an original image of the target object, perform image preprocessing on the original image, and obtain the image to be detected; The image to be detected is transformed based on a preset algorithm to obtain a plurality of 2.5-dimensional images.

4. The image defect detection method according to claim 1, characterized in that: The basic detection model is obtained in the following way: Acquire a basic training sample set, wherein the basic training sample set includes a plurality of basic training samples, and each of the basic training samples includes a defect label; Determine a loss function based on the type of defect to be detected; Inputting the plurality of basic training samples into a deep learning network to obtain a prediction result corresponding to each of the basic training samples; Determine the value of the loss function according to the prediction result and defect label corresponding to each of the basic training samples; When the value of the loss function reaches a preset value, the basic detection model is obtained.

5. The image defect detection method according to claim 4, characterized in that: The basic detection model includes frozen parameters and adapted parameters; The target detection model is obtained in the following way: Performing label verification on the basic training samples based on a confidence learning method to obtain optimized training samples; Inputting the optimized training samples into the basic detection model so that the frozen parameters in the basic detection model remain unchanged, training the adaptation parameters, and obtaining an intermediate detection model when the training results reach a convergence condition; Get the distillation loss function and defect type loss function; The intermediate detection model is trained based on the optimized training samples, and the target detection model is obtained when the sum of the distillation loss function and the defect type loss function reaches a preset threshold.

6. An image defect detection device, characterized in that: include: An image acquisition module is used to acquire multiple 2.5-dimensional images corresponding to the image to be detected; A defect detection module is used to obtain image blocks corresponding to each of the 2.5-dimensional images, perform foreground and background separation processing on each of the image blocks, and obtain separation features corresponding to each of the image blocks; Determine the image blocks containing defect features in the separation features as candidate image blocks, and perform grayscale gradient calculation on each of the candidate image blocks to obtain a target image block; Detect each of the target image blocks based on the target detection model to obtain a sub-defect result corresponding to each of the target image blocks; The target detection model is obtained by improving the model parameters of the basic detection model; A defect processing module, used for performing splicing processing on the sub-defect results corresponding to each of the image blocks to obtain a defect detection result of the image to be detected; The defect detection module includes a parameter acquisition unit and a block processing unit; The parameter acquisition unit is used to acquire the slicing parameters, wherein the slicing parameters include slicing length and slicing width; The slicing processing unit is used to perform slicing processing on each of the 2.5-dimensional images according to the slicing length and the slicing width to obtain a plurality of image blocks corresponding to each of the 2.5-dimensional images.

7. A macro imaging device, characterized in that: The macro imaging device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image defect detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image defect detection method as described in any one of claims 1 to 5 is implemented.

Images