Image defect detection method and device, macro imaging equipment and storage medium

Abstract

The invention discloses an image defect detection method and device, microspur imaging equipment and a storage medium, and relates to image processing, and the method comprises the steps: obtaining a plurality of 2.5-dimensional images corresponding to a to-be-detected image; acquiring an image block corresponding to each 2.5-dimensional image, and performing defect detection on each image block based on the target detection model to obtain a sub-defect result corresponding to each image block; the target detection model is obtained by improving model parameters of the basic detection model; and splicing the sub-defect result corresponding to each image block to obtain a defect detection result of the to-be-detected image. According to the invention, the problem of low defect feature recognition accuracy in a traditional scheme can be improved, and the beneficial effects of improving the defect detection efficiency and precision and meeting the high-precision defect detection requirement in modern industrial production are achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image defect detection method, device, macro imaging equipment and storage medium. Background Art

[0002] In modern industrial production and the manufacturing process of various products, it is very important to detect defects on the product surface. Traditional defect detection methods, such as manual visual inspection, rely on manual experience, which is not only inefficient, but also easily affected by the subjective factors of the inspectors, resulting in unstable detection accuracy and difficulty in meeting the needs of large-scale, high-precision production.

[0003] With the development of computer vision technology, image-based defect detection methods have gradually emerged. Traditional image-based defect detection mainly relies on simple image processing algorithms, such as edge detection and threshold segmentation. These methods have certain detection capabilities for simple defects with obvious features. However, with the increasing complexity of product manufacturing processes, the defects on the product surface are of various types and complex features, and traditional image processing algorithms are difficult to accurately and quickly detect defect features. Summary of the invention

[0004] The present application provides an image defect detection method, device, macro imaging equipment and storage medium, which can improve the efficiency and accuracy of defect detection.

[0005] In a first aspect, the present application provides an image defect detection method, comprising: obtaining multiple 2.5-dimensional images corresponding to an image to be detected; obtaining image blocks corresponding to each of the 2.5-dimensional images, performing defect detection on each of the image blocks based on a target detection model, and obtaining a sub-defect result corresponding to each of the image blocks; the target detection model is obtained by improving the model parameters of a basic detection model; and splicing the sub-defect results corresponding to each of the image blocks to obtain a defect detection result of the image to be detected.

[0006] Optionally, obtaining the image blocks corresponding to each of the 2.5-dimensional images includes: obtaining slicing parameters, the slicing parameters including slicing length and slicing width; slicing each of the 2.5-dimensional images according to the slicing length and the slicing width to obtain multiple image blocks corresponding to each of the 2.5-dimensional images.

[0007] Optionally, before performing defect detection on each of the image blocks based on the target detection model, the method further includes: performing foreground and background separation processing on each of the image blocks to obtain separation features corresponding to each of the image blocks; determining an image block containing a defect feature in the separation feature as a candidate image block, and performing grayscale gradient calculation on each of the candidate image blocks to obtain a target image block.

[0008] Correspondingly, the defect detection is performed on each of the image blocks based on the target detection model to obtain a sub-defect result corresponding to each of the image blocks, including: detecting each of the target image blocks based on the target detection model to obtain a sub-defect result corresponding to each of the target image blocks.

[0009] Optionally, the sub-defect results corresponding to each of the image blocks are stitched together to obtain the defect detection result of the image to be detected, including: obtaining the image identifier corresponding to each of the image blocks; stitching the sub-defect results corresponding to each of the image blocks according to the image identifier to obtain the initial detection result of the image to be detected; and fusing the initial detection results to obtain the defect detection result of the image to be detected.

[0010] Optionally, the acquiring of multiple 2.5-dimensional images corresponding to the image to be detected includes: acquiring an original image of the target object, performing image preprocessing on the original image to acquire the image to be detected; and performing transformation processing on the image to be detected based on a preset algorithm to acquire the multiple 2.5-dimensional images.

[0011] Optionally, the basic detection model is obtained in the following manner: obtaining a basic training sample set, the basic training sample set comprising multiple basic training samples, each of the basic training samples comprising a defect label; determining a loss function according to the type of defect to be detected; inputting the multiple basic training samples into a deep learning network to obtain a prediction result corresponding to each of the basic training samples; determining a value of the loss function according to the prediction result and defect label corresponding to each of the basic training samples; and obtaining the basic detection model when the value of the loss function reaches a preset value.

[0012] Optionally, the basic detection model includes frozen parameters and adaptation parameters; the target detection model is obtained in the following manner: label verification is performed on the basic training samples based on a confidence learning method to obtain optimized training samples; the optimized training samples are input into the basic detection model so that the frozen parameters in the basic detection model remain unchanged, the adaptation parameters are trained, and when the training results reach convergence conditions, an intermediate detection model is obtained; a distillation loss function and a defect type loss function are obtained; the intermediate detection model is trained based on the optimized training samples, and when the sum of the distillation loss function and the defect type loss function reaches a preset threshold, the target detection model is obtained.

[0013] In the second aspect, the present application provides an image defect detection device, which includes: an image acquisition module, used to acquire multiple 2.5-dimensional images corresponding to the image to be detected; a defect detection module, used to acquire image blocks corresponding to each of the 2.5-dimensional images, and perform defect detection on each of the image blocks based on a target detection model to obtain a sub-defect result corresponding to each of the image blocks; the target detection model is obtained by improving the model parameters of the basic detection model; a defect processing module, used to splice the sub-defect results corresponding to each of the image blocks to obtain the defect detection result of the image to be detected.

[0014] In a third aspect, the present application also provides a macro imaging device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image defect detection method described in any embodiment of the present application.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image defect detection method described in any embodiment of the present application when executed.

[0016] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the image defect detection method described in any embodiment of the present application.

[0017] The image defect detection scheme provided in the embodiment of the present application first obtains multiple 2.5-dimensional images corresponding to the image to be detected, and displays the multi-dimensional information of the image to be detected through the 2.5-dimensional images; then, based on the target detection model, defect detection is performed on the image blocks corresponding to each 2.5-dimensional image, and the sub-defect results corresponding to each image block are obtained, so that the model can analyze the local features of the image more carefully through block processing; in this embodiment, the target detection model is obtained by improving the model parameters of the basic detection model; this improvement method can be optimized for specific 2.5-dimensional image data and defect detection tasks, so that the model can better learn and extract defect features in 2.5-dimensional images; finally, by splicing the sub-defect results corresponding to each image block, the defect detection results of the image to be detected are obtained, and a complete and accurate defect detection result can be obtained. The problem of low defect feature recognition accuracy in the traditional scheme is improved, and the efficiency and accuracy of defect detection are improved, which meets the beneficial effect of high-precision defect detection requirements in modern industrial production.

[0018] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the image defect detection device, or may be packaged separately from the processor of the image defect detection device, and this application does not limit this.

[0019] The description of the second, third, fourth and fifth aspects of the present application can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description.

[0021] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, and usage scenarios of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a flow chart of the image defect detection method provided in the embodiment of the present application.

[0024] Figure 2 It is a structural schematic diagram of the image defect detection device provided in an embodiment of the present application.

[0025] Figure 3 It is a structural schematic diagram of the macro imaging device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the present application will be clearly and completely described below in conjunction with the drawings in the present embodiment. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0029] Figure 1 A flow chart of an image defect detection method provided in an embodiment of the present application. This embodiment is applicable to the situation where defect detection is performed on an image of an object captured by a macro imaging device. The method can be performed by an image defect detection device, which can be implemented in the form of hardware and / or software and integrated in the macro imaging device that executes the method.

[0030] refer to Figure 1The image defect detection method of this embodiment includes but is not limited to the following steps:

[0031] S110, obtaining a plurality of 2.5-dimensional images corresponding to the image to be detected.

[0032] The image to be inspected is an image that needs to be inspected for defects. It can come from various image acquisition devices, such as cameras, scanners, or macro imaging devices; the image to be inspected can also cover different application scenarios, such as lithium batteries, 3C semiconductors, and automobile manufacturing, etc., so that the collected image data can be used as the source of basic training samples.

[0033] Preferably, the solution provided in this embodiment is integrated in the scenario of defect detection based on a macro imaging device. The reason for this is that the traditional method of image defect detection based on a macro camera is generally to first shoot the object with a macro camera to obtain image data at different angles, then transmit the image data to a personal computer (PC) terminal, and finally use the defect detection algorithm software integrated in the PC to identify defects. This processing method is limited by the PC hardware configuration and interface bandwidth, and it is difficult to process a large amount of high-resolution image data in real time, resulting in low data processing efficiency. In view of this, by integrating the image defect detection method provided by this embodiment into a macro imaging device, the device can realize efficient image data processing, quickly obtain 2.5D imaging sequence images, and realize rapid analysis of defects through the onboard target detection model, achieving the beneficial effects of one-click real-time high-precision defect detection, improving the level of automation, and reducing the burden on operators.

[0034] 2.5D images are a form of image representation between two-dimensional (2D) and three-dimensional (3D). Compared with traditional 2D images, 2.5D images not only contain the plane texture information of the object, but also reflect the depth or height information of the object to a certain extent. This multi-dimensional information can more comprehensively describe the surface condition of the object, allowing the target detection model to obtain richer features. For example, for some objects with defects such as bumps, holes, etc. on the surface, 2.5D images can more clearly present the three-dimensional form of these defects, allowing the model to more accurately identify and locate defects, effectively avoiding the omission or misdetection of defects caused by using only 2D images, and greatly improving the accuracy of defect recognition.

[0035] In this embodiment, the method of obtaining multiple 2.5-dimensional images corresponding to the image to be detected can be: obtaining the original image of the target object, performing image preprocessing on the original image, and obtaining the image to be detected; converting the image to be detected based on a preset algorithm to obtain multiple 2.5-dimensional images. This embodiment can remove noise, adjust contrast, etc. by performing image preprocessing on the original image, so that the quality of the image to be detected is significantly improved. This processing method helps to improve the accuracy of subsequent analysis and reduce misjudgments and missed detections caused by image quality problems; and based on the 2.5D image, the depth information of the image is increased, which helps to understand the surface features of the target object. For some objects with complex surface structures, such as objects with defects such as bumps, holes, etc., 2.5D images can present these features more clearly, thereby improving the accuracy and reliability of defect detection.

[0036] The target objects mentioned above refer to specific objects that need to be subjected to defect detection, feature analysis and other operations, such as parts in industrial production (such as automobile engine parts, electronic chips, etc.); the original image refers to the initial image data obtained by photographing or scanning the target object through an image acquisition device. These images may contain various noises, uneven light, blur, and other problems, and their quality and format may vary depending on the acquisition equipment and environment.

[0037] In this embodiment, when the original image is preprocessed to obtain the image to be detected, the current image to be detected can be an image obtained by processing the original image using a filtering algorithm and a grayscale processing algorithm, etc. The purpose of this is to highlight the defect features in the image and facilitate subsequent processing operations.

[0038] Furthermore, this embodiment converts the image to be detected based on a preset algorithm, and the method of obtaining multiple 2.5-dimensional images can be: using a structured light projector to project a known pattern (such as stripes, grids, etc.) onto the target object, and a macro imaging device to capture the pattern modulated by the surface of the object. By analyzing the captured image, the deformation of the pattern is calculated, and then the depth information of the object surface is obtained. For example, phase measurement profilometry (PMP) obtains the height information of the object surface by solving the phase of the stripe pattern, generating a 2.5D image, etc.

[0039] In this embodiment, the 2.5D image includes multiple images, such as a shape image, a texture image, a height (depth) image, and a glossiness comparison image generated about the image to be detected.

[0040] S120, obtaining image blocks corresponding to each 2.5-dimensional image, performing defect detection on each image block based on the target detection model, and obtaining sub-defect results corresponding to each image block.

[0041] The image block is the image area divided according to certain rules of the 2.5D image, and each image block contains part of the original 2.5D image information, such as plane texture information and corresponding depth information.

[0042] The solution provided in this embodiment can obtain the image blocks corresponding to each 2.5-dimensional image by: obtaining the slicing parameters, the slicing parameters including the slicing length and the slicing width; slicing each 2.5-dimensional image according to the slicing length and the slicing width, to obtain multiple image blocks corresponding to each 2.5-dimensional image.

[0043] Specifically, first determine the appropriate block parameters, i.e., block length and block width, based on actual detection requirements, the characteristics of the target detection model, and the pixel parameters of the 2.5D image. For example, if the target detection model has a better processing effect on smaller image blocks, and the defects to be detected are usually small defects, then a smaller block length and width can be set, such as 50×50 pixels; conversely, if the defects in the image are large and the model can process larger image blocks, then the size of the block can be appropriately increased, such as 200×200 pixels. The specific block parameters are not limited here.

[0044] In the process of slicing based on slicing parameters, to ensure that the slicing window completely covers the image area, if the slicing window exceeds the image boundary at the edge of the image, the missing part can be supplemented by filling (such as filling with 0 value or the mean value of the image, etc.) so that the slicing process can be completed. Each time an image block is obtained by cutting, it is saved until the entire 2.5-dimensional image is divided into multiple image blocks that meet the requirements. In this way, multiple image blocks corresponding to each 2.5-dimensional image can be obtained.

[0045] This embodiment sets the block cutting parameters to flexibly adjust the size of the image block according to different application scenarios and detection requirements; and reasonable block cutting can improve the detection efficiency to a certain extent. For example, when performing feature extraction and analysis on smaller image blocks, the amount of calculation is relatively small, which can speed up the reasoning speed of the model; and block cutting according to fixed block cutting parameters can ensure that all parts of the image are included in the image block for analysis, avoiding the omission of important information due to unreasonable block cutting methods. At the same time, by setting appropriate block cutting parameters, while ensuring the coverage of the image, the redundant information between image blocks can be reduced, thereby improving the efficiency of data processing.

[0046] Furthermore, this embodiment uses the target detection model to perform defect detection on each image block to obtain a sub-defect result corresponding to each image block, and describes whether there is a defect in the corresponding image block and related information of the defect (such as the type, location, size, etc. of the defect) through the value defect result. These sub-defect results are the basis for subsequent defect analysis of the entire image to be detected.

[0047] In this embodiment, the target detection model is obtained by improving the model parameters of the basic detection model. Specifically, for the target detection model that has been pre-trained by the basic detection model, the model parameters will be adjusted according to a large number of image blocks with accurate defect annotations during the training process, so that the model can accurately identify and classify various defects in the image blocks. After the training is completed, the model is evaluated and optimized to ensure that its performance meets the requirements of defect detection. Finally, the divided image blocks are input into the target detection model in sequence, and the model extracts and analyzes the image blocks, and determines whether there are defects in the image blocks based on the learned image features. If there are defects, the type of defects (such as scratches, cracks, holes, etc.), location (coordinate position in the image block) and size (such as the area and length of the defect) and other information are further determined to generate sub-defect results corresponding to each image block.

[0048] Furthermore, since not every image block contains defect features, the solution provided in this embodiment can also perform the following operations before performing defect detection on each image block based on the target detection model: performing foreground and background separation processing on each image block to obtain separation features corresponding to each image block; determining the image block containing defect features in the separation features as a candidate image block, and performing grayscale gradient calculation on each candidate image block to obtain a target image block. This embodiment excludes a large number of image blocks that do not contain defect features by separating the foreground and background and screening candidate image blocks, thereby reducing the amount of data that needs to be input into the target detection model and improving detection efficiency; further performing grayscale gradient calculation on the candidate image blocks, and selecting image blocks with obvious grayscale changes as target image blocks. These target image blocks are more likely to contain defects, and can provide more valuable input for the target detection model, which helps the model to more accurately identify and locate defects, thereby improving the accuracy of defect detection and reducing false detection and missed detection.

[0049] The purpose of performing foreground and background separation processing on each image block is that in the defect detection of the macro camera, the defect is usually located in the foreground. By separating the foreground and background, the defect can be highlighted from the complex background, making the characteristics of the target defect more obvious, such as shape, texture, color and other features, which is convenient for subsequent analysis and processing. For example, after separation, the tiny cracks on the circuit board can more clearly see the details such as the direction and width of the cracks; and some irrelevant information in the background can be removed, such as the shell of the device, the surrounding ambient light and shadow, etc., which may interfere with defect detection. Therefore, after the separation process, the separation features corresponding to each image block can be obtained, and the current separation features can be defect features.

[0050] Furthermore, for image blocks that do not contain defects in the separation features, subsequent processing operations may not be performed to reduce the processing pressure of the target detection model. For image blocks that contain defect features, grayscale gradient calculations are performed on the candidate image blocks respectively, and a suitable threshold is set according to the calculated grayscale gradient amplitude, and the candidate image blocks with gradient amplitudes greater than the threshold are determined as target image blocks. These target image blocks are considered to have a high probability of containing defects and will be used for subsequent defect detection based on the target detection model.

[0051] Preferably, in this embodiment, the specific implementation method of performing defect detection on each image block based on the target detection model and obtaining the sub-defect result corresponding to each image block can be: performing defect detection on each target image block based on the target detection model and obtaining the sub-defect result corresponding to each target image block, thereby achieving the purpose of improving defect detection efficiency.

[0052] In another preferred implementation, the basic detection model in this embodiment is obtained by the following steps:

[0053] a) Obtain a basic training sample set, where the basic training sample set includes multiple basic training samples, and each basic training sample includes a defect label.

[0054] Collect image data related to the defects to be detected from multiple different sources. These sources can include product images on the actual production line, images in historical inspection records, images generated by simulation experiments, etc. For example, in industrial parts defect detection, images of parts at different production stages and different production batches can be collected to ensure that the data is diverse and representative; further add defect labels to each basic training sample collected. Defect labels accurately describe information such as the type of defect (such as scratches, cracks, holes, etc.), location (can be marked using bounding boxes, pixel coordinates, etc.), and severity; finally, the labeled samples are organized into a basic training sample set.

[0055] b) Determine the loss function based on the type of defect to be detected.

[0056] Before defect detection, the defect type and its characteristics to be detected are clearly defined. In this embodiment, different types of defects may require different loss functions for optimization. For example, for classification tasks (determining the type of defect), the cross entropy loss function is usually used; for regression tasks (predicting continuous values ​​such as the location and size of the defect), the mean square error loss function is often used.

[0057] c) Input multiple basic training samples into the deep learning network to obtain the prediction results corresponding to each basic training sample.

[0058] According to the characteristics and requirements of the defect detection task, a suitable deep learning network architecture is selected. In this embodiment, the deep learning network can be a convolutional neural network.

[0059] d) Determine the value of the loss function based on the prediction results and defect labels corresponding to each basic training sample.

[0060] For each basic training sample, its prediction result and the corresponding defect label are substituted into the previously determined loss function to calculate the loss value of the sample. For example, when using the cross entropy loss function for classification tasks, the cross entropy loss of a single sample is calculated based on the predicted class probability distribution and the true label.

[0061] e) When the value of the loss function reaches a preset value, a basic detection model is obtained.

[0062] Use an optimization algorithm (such as the stochastic gradient descent algorithm) to update the parameters of the deep learning network based on the gradient information of the loss function. The optimization algorithm will continuously adjust the parameters of the network so that the value of the loss function gradually decreases. When the value of the loss function reaches the preset value, it is considered that the model has converged to a better state. At this time, training is stopped and the current deep learning network is used as the basic detection model.

[0063] The above preset values ​​can be determined based on experience or through multiple experiments, and can also be combined with evaluation indicators on the validation set (such as accuracy, recall, etc.) to comprehensively judge whether the model has achieved satisfactory performance.

[0064] Furthermore, on the basis of obtaining the basic detection model, the basic detection model provided in this embodiment includes frozen parameters and adaptive parameters; wherein, the frozen parameters are parameters that are not updated or adjusted during the training phase of the basic detection model. Since the parameters usually have acquired relatively general and stable feature representation capabilities through learning a large amount of data during the early training of the basic model, freezing can prevent excessive changes in these learned useful features in subsequent training, thereby reducing the number of training parameters and computational complexity; the adaptive parameters are adjustable parameters in the basic detection model, and the adaptive parameters are used to enable the model to better adapt to specific tasks or data. In the subsequent training process, by optimizing the adaptive parameters, the model can learn specific features and patterns related to the current task, thereby improving the performance of the model on the task.

[0065] Specifically, the target detection model provided in this embodiment is obtained in the following manner:

[0066] f) Based on the confidence learning method, the labels of the basic training samples are verified to obtain the optimized training samples.

[0067] The label verification method for basic training samples based on the confidence learning method can be as follows: by comparing and analyzing the prediction results of the model on the basic training samples with the original labels, the confidence score of each sample label is calculated. The confidence score can reflect the reliability of the sample label, so that more reliable samples can be screened out for subsequent training and the quality of training data can be improved; by setting the confidence threshold, the basic training samples with confidence scores lower than the confidence threshold are filtered out to obtain optimized training samples.

[0068] g) Inputting the optimized training samples into the basic detection model so that the frozen parameters in the basic detection model remain unchanged, training the adaptation parameters, and obtaining the intermediate detection model when the training results reach the convergence condition.

[0069] The optimized training samples are input into the basic detection model. During the training process, the loss function is calculated based on the prediction results of the optimized training samples and the true labels, and the adaptation parameters are adjusted based on the gradient information of the loss function. During the training process, if the performance indicators of the model are monitored to no longer improve for multiple consecutive rounds, or the loss function value drops to a smaller range and tends to be stable, the training results are considered to have reached the convergence condition, and the intermediate detection model is obtained at this time.

[0070] h) Obtain the distillation loss function and the defect type loss function.

[0071] The distillation loss function is used to measure the output difference between the teacher model and the student model, prompting the student model to learn the knowledge and behavior of the teacher model, thereby achieving better performance at a smaller model scale. According to the structure and training objectives of the intermediate detection model, select the appropriate form of the distillation loss function. Depending on the specific situation, you can choose a single distillation loss function or combine multiple distillation loss functions, and set appropriate weights for each loss function item.

[0072] The defect type loss function is defined according to the defect type to be detected. It is used to measure the difference between the model's prediction results for different defect types and the true labels, and guide the model to learn the ability to accurately distinguish different defect types to improve the accuracy of defect detection. For example, for classification tasks, the multi-classification cross entropy loss function can be used; for regression tasks, the mean square error loss function can be used.

[0073] i) The intermediate detection model is trained based on the optimized training samples, and when the sum of the distillation loss function and the defect type loss function reaches a preset threshold, the target detection model is obtained.

[0074] The optimized training samples are input into the intermediate detection model again. In each training batch, the values ​​of the distillation loss function and the defect type loss function are calculated at the same time, and they are added to obtain the total loss function value. The optimization algorithm is used to update the parameters of the intermediate detection model (which may include some fine-tunable freezing parameters and adaptation parameters, depending on the settings) according to the gradient information of the total loss function to minimize the total loss function. During the training process, the sum of the distillation loss function and the defect type loss function is continuously monitored. When the sum reaches the preset threshold (the preset threshold can be determined by experiment or experience), it is considered that the model training has achieved the expected effect. At this time, the training is stopped to obtain the target detection model.

[0075] The target detection model provided in this embodiment reduces the number of parameters that need to be updated during the training process by freezing some parameters in the basic detection model, reduces the computational complexity, and speeds up the training speed; and verifies the labels of the basic training samples through the confidence learning method, screens out the optimized training samples with high confidence, reduces the interference of wrong labels or unreliable samples on the model training, and improves the quality of the training data; uses the distillation loss function to realize the transfer of knowledge from the larger basic detection model to the intermediate detection model, which helps to maintain good performance at a smaller model scale. At the same time, combined with the defect type loss function for training, the model's detection ability for defect types is further optimized, and the accuracy and reliability of defect detection are improved.

[0076] S130, performing splicing processing on the sub-defect results corresponding to each image block to obtain a defect detection result of the image to be detected.

[0077] The sub-defect results corresponding to each image block are spliced ​​according to certain rules to obtain the defect detection result corresponding to the original 2.5D image, that is, the defect detection result of the image to be detected. The current defect detection result at least includes: defect type, defect location, and defect size.

[0078] A preferred implementation method, the above step S130 of the scheme provided in this embodiment can be implemented in the following manner: obtain the image identifier corresponding to each image block; splice the sub-defect results corresponding to each image block according to the image identifier to obtain the initial detection result of the image to be detected; fuse the initial detection result to obtain the defect detection result of the image to be detected. Accurately splicing the sub-defect results through image identifiers avoids erroneous detection results caused by the disorder of image block positions. At the same time, the removal of duplicate detection and merging of adjacent defects in the fusion process can effectively eliminate false detections and duplicate annotations, so that the detection results more accurately reflect the actual defect situation in the image to be detected, and reduce the probability of missed detection and false detection. The initial detection results are fused, especially the merging of adjacent defects and edge smoothing, so that the presentation of the defect area is more coherent and natural, which helps the inspectors to more intuitively understand the distribution and morphology of defects and improve the ability to analyze and judge defects.

[0079] When the image to be detected is divided into multiple image blocks, a unique image identifier is assigned to each image block. The current image identifier can be a digital number, which is arranged in the order of the position of the image blocks in the original image (for example, from left to right, from top to bottom); or it can be a code containing the coordinate information of the image block (such as the coordinates of the upper left corner) to accurately record the position of the image block in the original image; after the target detection model performs defect detection on each image block, it will obtain the sub-defect results corresponding to each image block. These sub-defect results contain information about the defects in the image block, such as the type, location, size, etc. of the defect.

[0080] Based on the image identifier of each image block, the corresponding sub-defect results are stitched in the order of their positions in the original image. Create a blank image of the same size as the original image to be detected as the basis for stitching. For the sub-defect results of each image block, determine the position in the blank image according to its image identifier, and then fill the defect information in the sub-defect results (such as marking the defect area with a specific color or symbol) into the corresponding position of the blank image. If there are overlapping parts between image blocks, it is necessary to process the sub-defect results of the overlapping area according to the specific situation. For example, you can choose to retain the detection results with higher confidence, or make a comprehensive judgment and merge the defect information in the overlapping area.

[0081] Further fusion processing of the initial detection results includes at least smoothing edge processing, merging adjacent defects and smoothing edge processing, so as to obtain the defect detection result of the image to be detected. The result can be an image with defect annotations or a data file containing defect information (such as the location, type, size, etc. of the defect) for subsequent analysis and decision-making.

[0082] The image defect detection method provided in this embodiment first obtains multiple 2.5-dimensional images corresponding to the image to be detected, and displays the multi-dimensional information of the image to be detected through the 2.5-dimensional images; then, based on the target detection model, the image blocks corresponding to each 2.5-dimensional image are respectively detected to obtain the sub-defect results corresponding to each image block, so that the model can analyze the local features of the image more carefully through block processing; in this embodiment, the target detection model is obtained by improving the model parameters of the basic detection model; this improvement method can be optimized for specific 2.5-dimensional image data and defect detection tasks, so that the model can better learn and extract defect features in 2.5-dimensional images; finally, by splicing the sub-defect results corresponding to each image block to obtain the defect detection results of the image to be detected, a complete and accurate defect detection result can be obtained. The problem of low defect feature recognition accuracy in the traditional solution is improved, and the efficiency and accuracy of defect detection are improved to meet the beneficial effect of high-precision defect detection requirements in modern industrial production.

[0083] Figure 2 is a structural schematic diagram of an image defect detection device provided in an embodiment of the present application, and the device is suitable for executing the image defect detection method provided in an embodiment of the present application. Figure 2 As shown, the device may specifically include: an image acquisition module 210 , a defect detection module 220 and a defect processing module 230 .

[0084] The image acquisition module 210 is used to acquire multiple 2.5-dimensional images corresponding to the image to be detected.

[0085] The defect detection module 220 is used to obtain the image blocks corresponding to each of the 2.5-dimensional images, perform defect detection on each of the image blocks based on the target detection model, and obtain the sub-defect results corresponding to each of the image blocks; the target detection model is obtained by improving the model parameters of the basic detection model.

[0086] The defect processing module 230 is used to perform splicing processing on the sub-defect results corresponding to each of the image blocks to obtain the defect detection result of the image to be detected.

[0087] The image defect detection device provided in the embodiment of the present application first obtains multiple 2.5-dimensional images corresponding to the image to be detected, and displays the multi-dimensional information of the image to be detected through the 2.5-dimensional images; then, based on the target detection model, the image blocks corresponding to each 2.5-dimensional image are respectively detected to obtain the sub-defect results corresponding to each image block, so that the model can analyze the local features of the image more carefully through block processing; in this embodiment, the target detection model is obtained by improving the model parameters of the basic detection model; this improvement method can be optimized for specific 2.5-dimensional image data and defect detection tasks, so that the model can better learn and extract defect features in 2.5-dimensional images; finally, by splicing the sub-defect results corresponding to each image block, the defect detection result of the image to be detected is obtained, and a complete and accurate defect detection result can be obtained. The problem of low defect feature recognition accuracy in the traditional solution is improved, and the efficiency and accuracy of defect detection are improved, which meets the beneficial effect of high-precision defect detection requirements in modern industrial production.

[0088] In one embodiment, the defect detection module 220 includes a parameter acquisition unit and a block processing unit.

[0089] The parameter acquisition unit is used to acquire the slicing parameters, and the slicing parameters include slicing length and slicing width.

[0090] The slicing processing unit is used to perform slicing processing on each of the 2.5-dimensional images according to the slicing length and the slicing width to obtain a plurality of image blocks corresponding to each of the 2.5-dimensional images.

[0091] In one embodiment, the defect detection module 220 further includes a separation processing unit, a gradient calculation unit and a defect detection unit.

[0092] The separation processing unit is used to perform foreground and background separation processing on each of the image blocks to obtain separation features corresponding to each of the image blocks.

[0093] The gradient calculation unit is used to determine the image blocks containing defect features in the separation features as candidate image blocks, and perform grayscale gradient calculation on each of the candidate image blocks to obtain a target image block.

[0094] The defect detection unit is used to detect each of the target image blocks based on the target detection model to obtain a sub-defect result corresponding to each of the target image blocks.

[0095] In one embodiment, the defect processing module 230 is specifically used to obtain the image identifier corresponding to each of the image blocks; splice the sub-defect results corresponding to each of the image blocks according to the image identifier to obtain the initial detection result of the image to be detected; and fuse the initial detection result to obtain the defect detection result of the image to be detected.

[0096] In one embodiment, the image acquisition module 210 is specifically used to acquire the original image of the target object, perform image preprocessing on the original image to obtain the image to be detected; and perform transformation processing on the image to be detected based on a preset algorithm to obtain multiple 2.5-dimensional images.

[0097] In one embodiment, the basic detection model is obtained in the following manner: obtaining a basic training sample set, the basic training sample set comprising multiple basic training samples, each of the basic training samples comprising a defect label; determining a loss function according to the type of defect to be detected; inputting the multiple basic training samples into a deep learning network to obtain a prediction result corresponding to each of the basic training samples; determining a value of the loss function according to the prediction result and defect label corresponding to each of the basic training samples; and obtaining the basic detection model when the value of the loss function reaches a preset value.

[0098] In one embodiment, the basic detection model includes frozen parameters and adaptive parameters; the target detection model is obtained in the following manner: label verification is performed on the basic training samples based on a confidence learning method to obtain optimized training samples; the optimized training samples are input into the basic detection model so that the frozen parameters in the basic detection model remain unchanged, the adaptive parameters are trained, and when the training results reach convergence conditions, an intermediate detection model is obtained; a distillation loss function and a defect type loss function are obtained; the intermediate detection model is trained based on the optimized training samples, and when the sum of the distillation loss function and the defect type loss function reaches a preset threshold, the target detection model is obtained.

[0099] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0100] An embodiment of the present application also provides a macro imaging device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image defect detection method described in any embodiment of the present application.

[0101] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image defect detection method described in any embodiment of the present application when executed.

[0102] Reference below Figure 3 , Figure 3 1 is a schematic diagram of the structure of the macro imaging device provided in the embodiment of the present application, which shows a schematic diagram of the structure of a computer system 500 suitable for implementing the macro imaging device in the embodiment of the present application. Figure 3 The macro imaging device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0103] like Figure 3 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0104] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0105] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present application are executed.

[0106] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, etc., or any suitable combination of the above.

[0107] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0108] The modules and / or units involved in the embodiments described in the present application may be implemented by software or hardware. The modules and / or units described may also be set in a processor, for example, it may be described as: a processor includes an image acquisition module, a defect detection module, and a defect processing module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.

[0109] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining multiple 2.5-dimensional images corresponding to the image to be detected; obtaining image blocks corresponding to each of the 2.5-dimensional images, performing defect detection on each of the image blocks based on the target detection model, and obtaining sub-defect results corresponding to each of the image blocks; the target detection model is obtained by improving the model parameters of the basic detection model; the sub-defect results corresponding to each of the image blocks are spliced ​​to obtain the defect detection results of the image to be detected.

[0110] According to the technical solution of this embodiment, the problem of low defect feature recognition accuracy in the traditional solution is improved, and the efficiency and accuracy of defect detection are improved, which has the beneficial effect of meeting the demand for high-precision defect detection in modern industrial production.

[0111] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. An image defect detection method, characterized in that: include: Obtain multiple 2.5-dimensional images corresponding to the image to be detected; Obtain image blocks corresponding to each of the 2.5-dimensional images, perform defect detection on each of the image blocks based on the target detection model, and obtain sub-defect results corresponding to each of the image blocks; The target detection model is obtained by improving the model parameters of the basic detection model; Performing splicing processing on the sub-defect results corresponding to each of the image blocks to obtain a defect detection result of the image to be detected; The step of obtaining the image blocks corresponding to each of the 2.5-dimensional images comprises: Obtaining cutting parameters, wherein the cutting parameters include cutting length and cutting width; Each of the 2.5-dimensional images is sliced ​​according to the slice length and the slice width to obtain a plurality of image blocks corresponding to each of the 2.5-dimensional images.

2. The image defect detection method according to claim 1, characterized in that: Before performing defect detection on each of the image blocks based on the target detection model, the method further includes: Performing foreground and background separation processing on each of the image blocks to obtain separation features corresponding to each of the image blocks; Determine the image blocks containing defect features in the separation features as candidate image blocks, and perform grayscale gradient calculation on each of the candidate image blocks to obtain a target image block; Correspondingly, the defect detection is performed on each of the image blocks based on the target detection model to obtain a sub-defect result corresponding to each of the image blocks, including: Each of the target image blocks is detected based on the target detection model to obtain a sub-defect result corresponding to each of the target image blocks.

3. The image defect detection method according to claim 1, characterized in that: The step of performing splicing processing on the sub-defect results corresponding to each of the image blocks to obtain the defect detection result of the image to be detected includes: Obtaining an image identifier corresponding to each of the image blocks; According to the image identifier, the sub-defect results corresponding to each of the image blocks are spliced ​​to obtain an initial detection result of the image to be detected; The initial detection results are fused to obtain defect detection results of the image to be detected.

4. The image defect detection method according to claim 1, characterized in that: The step of obtaining a plurality of 2.5-dimensional images corresponding to the image to be detected includes: Acquire an original image of the target object, perform image preprocessing on the original image, and obtain the image to be detected; The image to be detected is transformed based on a preset algorithm to obtain a plurality of 2.5-dimensional images.

5. The image defect detection method according to claim 1, characterized in that: The basic detection model is obtained in the following way: Acquire a basic training sample set, wherein the basic training sample set includes a plurality of basic training samples, and each of the basic training samples includes a defect label; Determine a loss function based on the type of defect to be detected; Inputting the plurality of basic training samples into a deep learning network to obtain a prediction result corresponding to each of the basic training samples; Determine the value of the loss function according to the prediction result and defect label corresponding to each of the basic training samples; When the value of the loss function reaches a preset value, the basic detection model is obtained.

6. The image defect detection method according to claim 5, characterized in that: The basic detection model includes frozen parameters and adapted parameters; The target detection model is obtained in the following way: Performing label verification on the basic training samples based on a confidence learning method to obtain optimized training samples; Inputting the optimized training samples into the basic detection model so that the frozen parameters in the basic detection model remain unchanged, training the adaptation parameters, and obtaining an intermediate detection model when the training results reach a convergence condition; Get the distillation loss function and defect type loss function; The intermediate detection model is trained based on the optimized training samples, and the target detection model is obtained when the sum of the distillation loss function and the defect type loss function reaches a preset threshold.

7. An image defect detection device, characterized in that: include: An image acquisition module is used to acquire multiple 2.5-dimensional images corresponding to the image to be detected; A defect detection module is used to obtain image blocks corresponding to each of the 2.5-dimensional images, perform defect detection on each of the image blocks based on the target detection model, and obtain sub-defect results corresponding to each of the image blocks; The target detection model is obtained by improving the model parameters of the basic detection model; A defect processing module, used for performing splicing processing on the sub-defect results corresponding to each of the image blocks to obtain a defect detection result of the image to be detected; The defect detection module includes a parameter acquisition unit and a block processing unit; The parameter acquisition unit is used to acquire the slicing parameters, wherein the slicing parameters include slicing length and slicing width; The slicing processing unit is used to perform slicing processing on each of the 2.5-dimensional images according to the slicing length and the slicing width to obtain a plurality of image blocks corresponding to each of the 2.5-dimensional images.

8. A macro imaging device, characterized in that: The macro imaging device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image defect detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image defect detection method as described in any one of claims 1 to 6 is implemented.

Images