Defect detection method and device
By fusing the initial images acquired under different lighting conditions into the image to be tested and obtaining the grayscale feature vectors of each pixel point in the image, the problem of the inability to accurately detect small foreign objects in the film in the prior art is solved, the detection accuracy and accuracy are improved, and the over-detection and leakage detection rates are reduced.
Patent Information
- Application Number
- CN202411966599.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-25
AI Technical Summary
The prior art cannot accurately detect the smaller foreign body characteristics in the film, resulting in low detection accuracy and accuracy, and high over-detection and leakage detection rates.
By fusing multiple initial images collected under different lighting conditions into the image to be tested, the grayscale changes around the defective features in the image to be tested are more obvious, the grayscale feature vectors of each pixel point are obtained, and multiple grayscale feature sets and target grayscale feature values of the defect area are obtained based on these feature vectors.
It improves the accuracy and accuracy of defect detection, effectively reduces the over-detection rate and missed detection rate during the automated detection process, and can accurately detect small foreign matter characteristics in the film.
Smart Images
Figure CN119888387A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of industrial detection technology, and in particular, relates to a defect detection method and device. Background Art
[0002] In the new energy lithium battery industry, the appearance quality inspection of the battery cell is very important, which is related to the safety of the battery and people. Foreign matter on the surface of the battery cell is the top priority in the entire inspection system. Foreign matter in the film will pierce the surface film, causing the outer shell to discharge, causing serious accidents such as battery fire. In the related technology, there are methods for defect detection through AI segmentation and classification algorithms and traditional algorithms, but this method cannot accurately detect the smaller foreign matter features in the film, the detection precision and accuracy are not high, and the over-inspection rate and missed detection rate are high. Summary of the invention
[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a defect detection method and device, which can accurately detect the characteristics of smaller foreign matter in the film, improve the precision and accuracy of defect detection, and effectively reduce the over-detection rate and missed detection rate in the automated detection process.
[0004] In a first aspect, the present application provides a defect detection method, the method comprising:
[0005] Processing the image to be tested corresponding to the acquired object to be tested, and obtaining the grayscale feature vector corresponding to each pixel in the image to be tested; the image to be tested is obtained by fusing multiple initial images of the object to be tested acquired under different lighting conditions and the same acquisition viewing angle;
[0006] Based on the grayscale feature vectors corresponding to each of the pixel points and the target grayscale threshold, a first grayscale feature set and a second grayscale feature set corresponding to the defect area in the image to be tested are obtained; each grayscale feature vector in the first grayscale feature set is greater than the target grayscale threshold, and each grayscale feature vector in the second grayscale feature set is less than the target grayscale threshold;
[0007] Based on each grayscale feature vector in the first grayscale feature set and each grayscale feature vector in the second grayscale feature set, a target grayscale feature value corresponding to the defect area is obtained; the target grayscale feature value is used to perform defect detection on the defect features in the defect area.
[0008] According to the defect detection method provided in the embodiment of the present application, by fusing multiple initial images captured at the same acquisition angle under multiple different lighting conditions into an image to be tested, the grayscale change around the defect feature in the image to be tested is made more obvious, so that multiple grayscale feature sets corresponding to the defect area can be obtained by using the grayscale vector features of each pixel point in the image to be tested, and the target grayscale feature value corresponding to the defect area is obtained based on the larger grayscale feature vector and the smaller grayscale feature vector in the imaging of the defect area, so that the target grayscale feature value can more obviously characterize the tiny defects on the surface of the object to be tested, so that smaller foreign matter features in the thin film can be accurately detected, thereby improving the precision and accuracy of defect detection, and effectively reducing the over-detection rate and missed detection rate in the automated detection process.
[0009] In a defect detection method according to an embodiment of the present application, obtaining a target grayscale feature value corresponding to the defective area based on each grayscale feature vector in the first grayscale feature set and each grayscale feature vector in the second grayscale feature set includes:
[0010] Based on each grayscale feature vector in the first grayscale feature set, obtaining a first mean value corresponding to the first grayscale feature set; based on each grayscale feature vector in the second grayscale feature set, obtaining a second mean value corresponding to the second grayscale feature set;
[0011] Based on the difference between the first mean value and the second mean value, a target grayscale characteristic value corresponding to the defective area is obtained.
[0012] A defect detection method according to an embodiment of the present application,
[0013] The obtaining, based on each grayscale feature vector in the first grayscale feature set, a first mean corresponding to the first grayscale feature set includes:
[0014] Acquire multiple grayscale feature vectors greater than a first grayscale threshold from the first grayscale feature set; the first grayscale threshold is greater than the smallest grayscale feature vector in the first grayscale feature set and less than the largest grayscale feature vector in the first grayscale feature set;
[0015] Determine the mean corresponding to a plurality of grayscale feature vectors greater than the first grayscale threshold in the first grayscale feature set as the first mean;
[0016] The obtaining, based on each grayscale feature vector in the second grayscale feature set, a second mean corresponding to the second grayscale feature set includes:
[0017] Acquire multiple grayscale feature vectors less than a second grayscale threshold from the second grayscale feature set; the second grayscale threshold is greater than the smallest grayscale feature vector in the second grayscale feature set and less than the largest grayscale feature vector in the second grayscale feature set;
[0018] The mean corresponding to a plurality of grayscale feature vectors in the second grayscale feature set that are smaller than the second grayscale threshold is determined as the second mean.
[0019] A defect detection method according to an embodiment of the present application, wherein the image to be tested corresponding to the acquired object to be tested is processed to obtain a grayscale feature vector corresponding to each pixel in the image to be tested, comprises:
[0020] The image to be tested is processed by using a photometric stereo method to obtain an image grayscale matrix corresponding to the image to be tested; the number of rows of the image grayscale matrix is the number of multiple pixel points in the image to be tested, and the number of columns of the image grayscale matrix is the number of frames of the multiple initial images;
[0021] The image grayscale matrix is processed to obtain a grayscale feature vector corresponding to each pixel in the image to be tested.
[0022] In a defect detection method according to an embodiment of the present application, the grayscale matrix of the image is processed to obtain a grayscale feature vector corresponding to each pixel in the image to be tested, including:
[0023] Performing singular value decomposition processing on the image grayscale matrix to obtain a first matrix corresponding to the image grayscale matrix; the number of rows of the first matrix is the same as the number of rows of the image grayscale matrix, and the number of columns of the first matrix is the same as the number of columns of the image grayscale matrix;
[0024] The first matrix is normalized to obtain a grayscale feature vector corresponding to each pixel in the image to be tested.
[0025] In a defect detection method according to an embodiment of the present application, the grayscale feature vector includes the grayscale feature components corresponding to the pixel points under each image channel; and based on the grayscale feature vectors corresponding to each pixel point and the target grayscale threshold, obtaining the first grayscale feature set and the second grayscale feature set corresponding to the defect area in the image to be tested includes:
[0026] Obtaining the grayscale feature component corresponding to each pixel point in the target image channel among the multiple image channels;
[0027] All grayscale feature components whose grayscale feature components corresponding to the pixels in the defect area are greater than the target grayscale threshold are determined as the first grayscale feature set; all grayscale feature components whose grayscale feature components corresponding to the pixels in the defect area are less than the target grayscale threshold are determined as the second grayscale feature set.
[0028] In a second aspect, the present application provides a defect detection device, the device comprising:
[0029] A first processing module is used to process the image to be tested corresponding to the acquired object to be tested, and obtain the grayscale feature vector corresponding to each pixel in the image to be tested; the image to be tested is obtained by fusing multiple initial images of the object to be tested acquired under different lighting conditions and the same acquisition angle;
[0030] A second processing module is used to obtain a first grayscale feature set and a second grayscale feature set corresponding to the defect area in the image to be tested based on the grayscale feature vector corresponding to each pixel point and the target grayscale threshold; each grayscale feature vector in the first grayscale feature set is greater than the target grayscale threshold, and each grayscale feature vector in the second grayscale feature set is less than the target grayscale threshold;
[0031] The third processing module is used to obtain the target grayscale feature value corresponding to the defect area based on each grayscale feature vector in the first grayscale feature set and each grayscale feature vector in the second grayscale feature set; the target grayscale feature value is used to perform defect detection on the defect features in the defect area.
[0032] According to the defect detection device provided in the embodiment of the present application, by fusing multiple initial images captured at the same capture angle under multiple different lighting conditions into an image to be tested, the grayscale change around the defect feature in the image to be tested is more obvious, so that multiple grayscale feature sets corresponding to the defect area can be obtained with the grayscale vector features of each pixel point in the image to be tested, and the target grayscale feature value corresponding to the defect area is obtained based on the larger grayscale feature vector and the smaller grayscale feature vector in the imaging of the defect area, so that the target grayscale feature value can more obviously characterize the tiny defects on the surface of the object to be tested, so that smaller foreign matter features in the thin film can be accurately detected, thereby improving the precision and accuracy of defect detection, and effectively reducing the over-detection rate and missed detection rate in the automated detection process.
[0033] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the defect detection method as described in the first aspect above when executing the computer program.
[0034] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the defect detection method as described in the first aspect above.
[0035] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the defect detection method as described in the first aspect above.
[0036] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:
[0037] By fusing multiple initial images acquired at the same acquisition angle under multiple different lighting conditions into the image to be tested, the grayscale changes around the defect features in the image to be tested are more obvious, so that multiple grayscale feature sets corresponding to the defect area can be obtained with the grayscale vector features of each pixel in the image to be tested, and the target grayscale feature value corresponding to the defect area is obtained based on the larger grayscale feature vector and the smaller grayscale feature vector in the defect area imaging, so that the target grayscale feature value can more obviously characterize the tiny defects on the surface of the object to be tested, so that smaller foreign matter features in the thin film can be accurately detected, thereby improving the precision and accuracy of defect detection and effectively reducing the over-detection rate and missed detection rate in the automated detection process.
[0038] Furthermore, by obtaining the target grayscale threshold in the imaging corresponding to the flat area, a first grayscale feature set greater than the target grayscale threshold and a second grayscale feature set less than the target grayscale threshold are obtained corresponding to the defective area. Then, the first mean corresponding to the first grayscale feature set and the second mean corresponding to the second grayscale feature set are respectively obtained by the percentage mean method. The target grayscale feature value corresponding to the defective area is calculated by the relative extreme point method, which can effectively filter out the influence of interference points close to the target grayscale threshold, so that tiny defects can be more clearly characterized by the target grayscale feature value, thereby improving the accuracy and precision of defect detection.
[0039] Furthermore, by adopting the photometric stereo method to obtain the image grayscale matrix corresponding to the image to be tested, and then performing singular value decomposition processing on the image grayscale matrix to normalize the obtained first matrix, the grayscale feature vector of each pixel point in the image to be tested can be normalized to a uniform interval, which is convenient for solving the feature description of the defect area according to the grayscale feature vector, thereby improving the accuracy of defect detection.
[0040] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0042] Figure 1 It is a flowchart of a defect detection method provided in an embodiment of the present application;
[0043] Figure 2 It is one of the detection schematic diagrams of the defect detection method provided in the embodiment of the present application;
[0044] Figure 3 This is the second detection schematic diagram of the defect detection method provided in the embodiment of the present application;
[0045] Figure 4 This is the third detection schematic diagram of the defect detection method provided in the embodiment of the present application;
[0046] Figure 5 This is the fourth detection schematic diagram of the defect detection method provided in the embodiment of the present application;
[0047] Figure 6 is a schematic diagram of the structure of a defect detection device provided in an embodiment of the present application;
[0048] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0050] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0051] The defect detection method, defect detection device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0052] The defect detection method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.
[0053] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).
[0054] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.
[0055] The defect detection method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the defect detection method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The defect detection method provided in the embodiment of the present application is described below using the electronic device as an example of the execution subject.
[0056] like Figure 1 As shown, the defect detection method includes: step 110, step 120 and step 130.
[0057] Step 110: Process the image to be tested corresponding to the acquired object to be tested, and obtain the grayscale feature vector corresponding to each pixel in the image to be tested;
[0058] In this step, the object to be tested may be a workpiece to be tested in an industrial testing process, for example, the object to be tested may include a cell in a battery.
[0059] There is a film attached to the surface of the battery cell, which is usually a colored film, such as a blue film or a black film of 90um, 110um or above 110um. There may be defects with concave and convex characteristics such as foreign matter on the surface of the battery cell, and the defects have high information.
[0060] For example, defects can also include pits, bubbles, and wrinkles, which are highly informative.
[0061] The image to be tested is obtained by fusing multiple initial images of the object to be tested that are collected under different lighting conditions and the same collection angle of view.
[0062] A plurality of light source devices may be provided, each of which is installed at a different position, that is, a plurality of light source directions may be used to illuminate the object to be measured.
[0063] Each light source device can emit parallel light of equal cross-section. The light source device may include a telecentric light source. The light emitted by the light source device has uniform intensity. The light source device may also be set as a long-distance point light source. The selection can be based on user needs and is not limited in this application.
[0064] An image sensor can be used to capture images of the object to be measured. The image sensor can be fixedly installed above the area to be measured. The image sensor can be perpendicular to the plane where the object to be measured is located. The object to be measured can be placed in the center of the image sensor's capture field of view.
[0065] The image sensor may include a telecentric lens or a telephoto lens so that there is no distortion of the object to be measured in the initial image.
[0066] When performing image acquisition, any one of the multiple light source devices can be controlled to be turned on, and the other light source devices can be turned off to provide different lighting conditions.
[0067] Multiple initial images can be registered so that they are aligned in the same coordinate system, and then an image fusion algorithm (such as weighted average, maximum fusion or multi-resolution fusion) can be used to merge the registered images into one image to be tested.
[0068] In the present application, by providing multiple different lighting conditions and acquiring multiple initial images at the same acquisition angle of view, and then merging the multiple initial images into an image to be tested, the grayscale changes around the defects in the imaging of the object to be tested are more obvious, thereby enabling the defects to be accurately detected.
[0069] For each initial image, a grayscale feature vector of each pixel can be calculated, and the grayscale feature vector is used to characterize the reflection of the object to be measured under different lighting conditions.
[0070] The grayscale feature vector may include the grayscale value or normal vector of a pixel point.
[0071] Step 120: Based on the grayscale feature vector corresponding to each pixel point and the target grayscale threshold, a first grayscale feature set and a second grayscale feature set corresponding to the defective area in the image to be tested are obtained;
[0072] In this step, the grayscale feature vectors corresponding to the plurality of pixels corresponding to the defective area may be divided into two sets based on the size relationship between the grayscale feature vector corresponding to each pixel and the target grayscale threshold.
[0073] Among them, the defect area can be the mask area corresponding to the defect feature, such as Figure 2 An example of a defect characteristic is shown below. Figure 4 The mask area corresponding to the defect feature is illustrated; Figure 3 Another defect characteristic is illustrated, Figure 5 The mask area corresponding to the defect feature is illustrated.
[0074] The grayscale feature vector corresponding to each pixel can be normalized to [0, 255], and the target grayscale threshold can be 127.
[0075] Each grayscale feature vector in the first grayscale feature set is greater than the target grayscale threshold, and each grayscale feature vector in the second grayscale feature set is less than the target grayscale threshold.
[0076] During the research and development process, the inventors discovered that in a flat area without height changes, the grayscale feature vector corresponding to the pixel points in the image to be tested is close to the target grayscale threshold, and in an area with concave-convex defects, the imaging of the area presents a yin-yang distribution, that is, the grayscale feature vectors of the pixel points in half of the area are greater than the target grayscale threshold, and the grayscale feature vectors of the pixel points in the other half of the area are less than the target grayscale threshold. According to the size relationship between the grayscale threshold of the pixel points and the target grayscale threshold, the first grayscale feature set (that is, the set of grayscale feature vectors greater than the target grayscale threshold) and the second grayscale feature set (that is, the set of grayscale feature vectors less than the target grayscale threshold) corresponding to the defective area can be obtained.
[0077] Step 130: based on each grayscale feature vector in the first grayscale feature set and each grayscale feature vector in the second grayscale feature set, obtain a target grayscale feature value corresponding to the defect area;
[0078] In this step, the target grayscale feature value corresponding to the defective area can be used to characterize the severity of the defect feature in the defective area. For example, when the target grayscale feature value is larger, it indicates that the defect feature in the defective area is more severe.
[0079] The method of relatively taking extreme points is used to obtain the target grayscale feature value corresponding to the defect area, which can more clearly characterize tiny defects.
[0080] Based on the calculated target grayscale feature value corresponding to the defect area, defect detection can be performed on the defect features in the defect area. For example, the defect severity of the defect features in the defect area can be directly graded according to the target grayscale feature value, or the target grayscale feature value and traditional feature description, such as the area, length, width or height of the defect feature, can be combined to grade the defect features in the defect area.
[0081] For example, the user can set a threshold for defect level classification based on the detection requirements. For example, the threshold can be set to 70 or 80, or other values. The threshold can be set to 70, and when the target grayscale feature value corresponding to the defect area is 255, the object to be tested can be determined as an NG product. When the target grayscale feature value corresponding to the defect area is 20, the object to be tested can be determined as a good product.
[0082] According to the defect detection method provided in the embodiment of the present application, by fusing multiple initial images captured at the same acquisition angle under multiple different lighting conditions into an image to be tested, the grayscale change around the defect feature in the image to be tested is made more obvious, so that multiple grayscale feature sets corresponding to the defect area can be obtained by using the grayscale vector features of each pixel point in the image to be tested, and the target grayscale feature value corresponding to the defect area is obtained based on the larger grayscale feature vector and the smaller grayscale feature vector in the imaging of the defect area, so that the target grayscale feature value can more obviously characterize the tiny defects on the surface of the object to be tested, so that smaller foreign matter features in the thin film can be accurately detected, thereby improving the precision and accuracy of defect detection, and effectively reducing the over-detection rate and missed detection rate in the automated detection process.
[0083] In some embodiments, step 130 may include:
[0084] Based on each grayscale feature vector in the first grayscale feature set, obtaining a first mean value corresponding to the first grayscale feature set; based on each grayscale feature vector in the second grayscale feature set, obtaining a second mean value corresponding to the second grayscale feature set;
[0085] Based on the difference between the first mean and the second mean, a target grayscale characteristic value corresponding to the defective area is obtained.
[0086] In this embodiment, the first mean value is used to characterize the grayscale information corresponding to the brighter area in the imaging of the defect area.
[0087] The second mean value is used to characterize the grayscale information corresponding to the darker area in the imaging of the defect area.
[0088] The difference between the first mean and the second mean may be determined as the target grayscale characteristic value corresponding to the defective area.
[0089] In some embodiments, obtaining a first mean value corresponding to the first grayscale feature set based on each grayscale feature vector in the first grayscale feature set may include:
[0090] Acquire a plurality of grayscale feature vectors greater than a first grayscale threshold from the first grayscale feature set;
[0091] The mean corresponding to a plurality of grayscale feature vectors greater than the first grayscale threshold in the first grayscale feature set is determined as a first mean.
[0092] In this embodiment, the grayscale feature vectors in the first grayscale feature set are all greater than the target grayscale threshold, and the first grayscale threshold can be determined according to the number of multiple grayscale feature vectors required.
[0093] The first grayscale threshold is greater than the smallest grayscale feature vector in the first grayscale feature set, and less than the largest grayscale feature vector in the first grayscale feature set.
[0094] For example, the first grayscale threshold may be determined according to the size distribution of the grayscale feature vectors in the first grayscale feature set, and then a target number of grayscale feature vectors greater than the first grayscale threshold may be obtained.
[0095] In some embodiments, the grayscale feature vectors in the first grayscale feature set can be sorted in a target order based on the size of each grayscale feature vector in the first grayscale feature set, and multiple grayscale feature vectors greater than the first grayscale threshold value are obtained in the sorted first grayscale feature set.
[0096] For example, the grayscale feature vectors in the first grayscale feature set can be sorted in descending order, and then the first n% grayscale feature vectors can be obtained from the sorted first grayscale feature set, and the mean of the first n% grayscale feature vectors can be determined as the first mean.
[0097] In the actual execution process, the first grayscale feature set in which the grayscale feature vectors are all greater than the target grayscale threshold in the mask area corresponding to the defect feature is N. max =[N m0 …N mm ], the first grayscale feature set includes m grayscale feature vectors, and then N max Sort in descending order and calculate N max The mean of the first MK (MK∈[1,m]) points is determined as the first mean, denoted by V max .
[0098] In some embodiments, obtaining a second mean corresponding to the second grayscale feature set based on each grayscale feature vector in the second grayscale feature set may include:
[0099] Acquire a plurality of grayscale feature vectors less than a second grayscale threshold from the second grayscale feature set;
[0100] The mean corresponding to a plurality of grayscale feature vectors in the second grayscale feature set that are smaller than the second grayscale threshold is determined as the second mean.
[0101] In this embodiment, the grayscale feature vectors in the second grayscale feature set are all smaller than the target grayscale threshold, and the second grayscale threshold can be determined according to the number of the plurality of grayscale feature vectors required.
[0102] The second grayscale threshold is greater than the smallest grayscale feature vector in the second grayscale feature set, and less than the largest grayscale feature vector in the second grayscale feature set.
[0103] For example, the second grayscale threshold may be determined according to the size distribution of the grayscale feature vectors in the second grayscale feature set, and then a target number of grayscale feature vectors smaller than the first grayscale threshold may be obtained.
[0104] In some embodiments, the grayscale feature vectors in the second grayscale feature set can be sorted in a target order based on the size of each grayscale feature vector in the second grayscale feature set, and multiple grayscale feature vectors less than the second grayscale threshold value in the sorted second grayscale feature set are obtained.
[0105] For example, the grayscale feature vectors in the second grayscale feature set can be sorted in ascending order, and then the first n% grayscale feature vectors can be obtained from the sorted second grayscale feature set, and the mean of the first n% grayscale feature vectors can be determined as the second mean.
[0106] In the actual execution process, the second grayscale feature set in which the grayscale feature vectors are all smaller than the target grayscale threshold in the mask area corresponding to the defect feature is N. min =[N n0 …N nn ], the second grayscale feature set includes n grayscale feature vectors, and then N min Sort in ascending order and calculate N min The mean of the first NK (NK∈[1,n]) points of is determined as the second mean, denoted by V min .
[0107] After calculating the first mean and the second mean, the target grayscale feature value corresponding to the defect area can be obtained:
[0108] DN Norm =V max -V min
[0109] Among them, DN Norm is the target grayscale eigenvalue, V max is the first mean, V min is the second mean.
[0110] According to the defect detection method provided in the embodiment of the present application, by obtaining the target grayscale threshold in the imaging corresponding to the flat area, a first grayscale feature set greater than the target grayscale threshold and a second grayscale feature set less than the target grayscale threshold corresponding to the defective area are obtained, and then the first mean corresponding to the first grayscale feature set and the second mean corresponding to the second grayscale feature set are respectively obtained by the percentage mean method. The target grayscale feature value corresponding to the defective area is calculated by the relative extreme point method, which can effectively filter out the influence of interference points close to the target grayscale threshold, so that tiny defects can be more clearly characterized by the target grayscale feature value, thereby improving the accuracy and precision of defect detection.
[0111] In some embodiments, step 110 may include:
[0112] The photometric stereo method is used to process the image to be tested, and the image grayscale matrix corresponding to the image to be tested is obtained;
[0113] The image grayscale matrix is processed to obtain the grayscale feature vector corresponding to each pixel in the image to be tested.
[0114] In this embodiment, the photometric stereo method is a method of acquiring a two-dimensional image sequence of the surface of the object to be measured at the same viewing angle under different lighting conditions by transforming light sources in different directions, calculating the surface normal vector from the light and dark changes of the two-dimensional image, then obtaining the directional gradients in the X and Y directions, and finally reconstructing the surface shape based on the gradients.
[0115] Based on the principle of photometric stereo imaging, we know that:
[0116] I=SL
[0117] Among them, S represents the weighted amplitude of the surface normal vector, L is the light source direction vector, and I is the image grayscale vector (i.e., the image grayscale matrix).
[0118] in, p is the number of pixels in the image to be tested, and f is the number of multiple initial images.
[0119] The number of rows of the image gray matrix is p, p is the number of pixels in the image to be tested, the number of columns of the image gray matrix is f, f is the number of the multiple initial images, and f≥3.
[0120] By processing the image grayscale matrix, the grayscale feature vector corresponding to each pixel in the image to be tested can be obtained.
[0121] In some embodiments, processing the image grayscale matrix to obtain the grayscale feature vector corresponding to each pixel in the image to be tested may include:
[0122] Performing singular value decomposition processing on the image grayscale matrix to obtain a first matrix corresponding to the image grayscale matrix;
[0123] The first matrix is normalized to obtain the grayscale feature vector corresponding to each pixel in the image to be tested.
[0124] In this embodiment, the image grayscale matrix is subjected to singular value decomposition (SVD) processing, and the image grayscale matrix can be decomposed into three specific matrix products: a left singular matrix, an eigenvalue diagonal matrix, and the transpose of a right singular matrix.
[0125] The first matrix may be a left singular matrix.
[0126] The number of rows of the first matrix is the same as the number of rows of the image grayscale matrix, and the number of columns of the first matrix is the same as the number of columns of the image grayscale matrix.
[0127] For example, by performing SVD decomposition on the image grayscale matrix I, we can obtain: I = UWV T , where the size of the left singular matrix U is [p, f], the size of the eigenvalue diagonal matrix W is [f, f], and the size of the right singular matrix V is [f, f].
[0128] We can define S=U, L=WV T .
[0129] The same pixel positions in each initial image have common points. For example, the grayscale values corresponding to the same pixel position in different initial images may be larger or smaller (that is, the imaging of the area in different initial images is brighter or darker). By performing SVD decomposition on the image grayscale matrix, the most representative features of each pixel point can be concentrated in the first few columns of the decomposed image grayscale matrix, and then the grayscale feature vector corresponding to each pixel point in the image to be tested can be obtained based on the features of the first few columns.
[0130] In the actual execution process, the first three columns of the first matrix U can be taken and normalized according to the rows to obtain the normalized surface normal vector:
[0131]
[0132] Wherein, p is the number of pixels in the image to be tested. Before normalization, the value range of the surface normal vector is [-1, 1]. By normalizing the first matrix, the Nx and Ny components of the surface normal vector can be normalized to [0, 255]. When the surface normal vector approaches 0 before normalization, the grayscale feature vector after normalization is 127, that is, in the area where there is no height change on the surface of the object to be tested, the grayscale feature vector corresponding to the pixel in the corresponding imaging is close to 127;
[0133] When the surface normal vector is positive before normalization, the grayscale feature vector after normalization is (127, 255]; when the surface normal vector is negative before normalization, the grayscale feature vector after normalization is [0, 127).
[0134] like Figure 2 and Figure 3 As shown, the imaging corresponding to the area with concave-convex defects on the surface of the object to be tested presents a positive-negative distribution, that is, the grayscale feature vector corresponding to the pixels in half of the area is greater than 127, the grayscale feature vector corresponding to the pixels in the other half of the area is less than 127, and the grayscale feature vector corresponding to the pixels in the background area is around 127.
[0135] When the height of the raised defect on the surface of the object to be tested is higher and the foreign matter is sharper, the defect feature is more obvious and the target grayscale feature value corresponding to the defect area obtained after detection is larger. Figure 2 The target grayscale feature value corresponding to the defect feature is greater than Figure 3 The target grayscale feature value corresponding to the defect feature.
[0136] According to the defect detection method provided in the embodiment of the present application, the image grayscale matrix corresponding to the image to be tested is obtained by adopting the photometric stereo method, and then the image grayscale matrix is subjected to singular value decomposition processing to normalize the obtained first matrix, so that the grayscale feature vector of each pixel point in the image to be tested can be normalized to a uniform interval, which is convenient for solving the feature description of the defect area according to the grayscale feature vector, thereby improving the accuracy of defect detection.
[0137] In some embodiments, step 120 may include:
[0138] Obtain the grayscale feature components corresponding to each pixel in the target image channel among multiple image channels;
[0139] All grayscale feature components corresponding to the pixel points in the defect area that are greater than the target grayscale threshold are determined as the first grayscale feature set; all grayscale feature components corresponding to the pixel points in the defect area that are less than the target grayscale threshold are determined as the second grayscale feature set.
[0140] In this embodiment, the grayscale feature vector includes grayscale feature components corresponding to the pixel points in each image channel.
[0141] The multiple image channels may include an R channel, a G channel, and a B channel, and the target image channel is any channel among the multiple image channels, that is, the grayscale feature vector includes Nz, Nx, and Ny.
[0142] For example, after normalizing the Nx and Ny components of the surface normal vector to [0, 255], Nx or Ny can be determined as the grayscale feature component corresponding to each pixel in the target image channel. For example, the grayscale feature component can be defined as: N = [N1 ...N p ], where p is the number of multiple pixels in the image to be tested.
[0143] For example, the G channel can be determined as the target image channel, all grayscale feature components in Nx whose grayscale feature components are greater than the target grayscale threshold are determined as the first grayscale feature set, and all grayscale feature components in Nx whose grayscale feature components are less than the target grayscale threshold are determined as the second grayscale feature set.
[0144] In the present application, the grayscale feature vector of each pixel in the image to be tested is calculated by using the photometric stereo method to generate a feature descriptor (target grayscale feature value) corresponding to the defect area. Compared with traditional 2D feature descriptors (such as geometric features or texture features), the feature descriptor proposed in the present application can provide "3D" information of defect features without increasing the equipment cost and algorithm complexity, and endow the 2D image with 3D information representation, providing a more diversified information representation, and the representation of defect features is more accurate and comprehensive, thereby improving the detection precision and accuracy of the detection equipment.
[0145] The defect detection device provided in the present application is described below. The defect detection device described below and the defect detection method described above can be referenced to each other.
[0146] The defect detection method provided in the embodiment of the present application can be executed by a defect detection device. In the embodiment of the present application, the defect detection device provided in the embodiment of the present application is described by taking the defect detection method executed by the defect detection device as an example.
[0147] The embodiment of the present application also provides a defect detection device.
[0148] like Figure 6 As shown, the defect detection device includes: a first processing module 610, a second processing module 620 and a third processing module 630.
[0149] The first processing module 610 is used to process the image to be tested corresponding to the acquired object to be tested, and obtain the grayscale feature vector corresponding to each pixel in the image to be tested; the image to be tested is obtained by fusing multiple initial images of the object to be tested acquired under different lighting conditions and the same acquisition angle;
[0150] The second processing module 620 is used to obtain a first grayscale feature set and a second grayscale feature set corresponding to the defect area in the image to be tested based on the grayscale feature vector corresponding to each pixel point and the target grayscale threshold; each grayscale feature vector in the first grayscale feature set is greater than the target grayscale threshold, and each grayscale feature vector in the second grayscale feature set is less than the target grayscale threshold;
[0151] The third processing module 630 is used to obtain the target grayscale feature value corresponding to the defect area based on each grayscale feature vector in the first grayscale feature set and each grayscale feature vector in the second grayscale feature set; the target grayscale feature value is used to perform defect detection on the defect features in the defect area.
[0152] According to the defect detection device provided in the embodiment of the present application, by fusing multiple initial images captured at the same capture angle under multiple different lighting conditions into an image to be tested, the grayscale change around the defect feature in the image to be tested is more obvious, so that multiple grayscale feature sets corresponding to the defect area can be obtained with the grayscale vector features of each pixel point in the image to be tested, and the target grayscale feature value corresponding to the defect area is obtained based on the larger grayscale feature vector and the smaller grayscale feature vector in the imaging of the defect area, so that the target grayscale feature value can more obviously characterize the tiny defects on the surface of the object to be tested, so that smaller foreign matter features in the thin film can be accurately detected, thereby improving the precision and accuracy of defect detection, and effectively reducing the over-detection rate and missed detection rate in the automated detection process.
[0153] In some embodiments, the third processing module 630 may also be used to:
[0154] Based on each grayscale feature vector in the first grayscale feature set, obtaining a first mean value corresponding to the first grayscale feature set; based on each grayscale feature vector in the second grayscale feature set, obtaining a second mean value corresponding to the second grayscale feature set;
[0155] Based on the difference between the first mean and the second mean, a target grayscale characteristic value corresponding to the defective area is obtained.
[0156] In some embodiments, the third processing module 630 may also be used to:
[0157] Acquire multiple grayscale feature vectors greater than a first grayscale threshold from the first grayscale feature set; the first grayscale threshold is greater than the smallest grayscale feature vector in the first grayscale feature set and less than the largest grayscale feature vector in the first grayscale feature set;
[0158] Determine the mean corresponding to a plurality of grayscale feature vectors greater than the first grayscale threshold in the first grayscale feature set as a first mean;
[0159] Acquire multiple grayscale feature vectors less than a second grayscale threshold from the second grayscale feature set; the second grayscale threshold is greater than the smallest grayscale feature vector in the second grayscale feature set and less than the largest grayscale feature vector in the second grayscale feature set;
[0160] The mean corresponding to a plurality of grayscale feature vectors in the second grayscale feature set that are smaller than the second grayscale threshold is determined as the second mean.
[0161] In some embodiments, the first processing module 610 may also be used to:
[0162] The image to be tested is processed by using the photometric stereo method to obtain an image grayscale matrix corresponding to the image to be tested; the number of rows of the image grayscale matrix is the number of multiple pixel points in the image to be tested, and the number of columns of the image grayscale matrix is the number of frames of the multiple initial images;
[0163] The image grayscale matrix is processed to obtain the grayscale feature vector corresponding to each pixel in the image to be tested.
[0164] In some embodiments, the first processing module 610 may also be used to:
[0165] Performing singular value decomposition processing on the image grayscale matrix to obtain a first matrix corresponding to the image grayscale matrix; the number of rows of the first matrix is the same as the number of rows of the image grayscale matrix, and the number of columns of the first matrix is the same as the number of columns of the image grayscale matrix;
[0166] The first matrix is normalized to obtain the grayscale feature vector corresponding to each pixel in the image to be tested.
[0167] In some embodiments, the grayscale feature vector includes grayscale feature components corresponding to the pixel points under each image channel; the second processing module 620 may also be used for:
[0168] Obtain the grayscale feature components corresponding to each pixel in the target image channel among multiple image channels;
[0169] All grayscale feature components corresponding to the pixel points in the defect area that are greater than the target grayscale threshold are determined as the first grayscale feature set; all grayscale feature components corresponding to the pixel points in the defect area that are less than the target grayscale threshold are determined as the second grayscale feature set.
[0170] The defect detection device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0171] The defect detection device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0172] The defect detection device provided in the embodiment of the present application can achieve Figures 1 to 5 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0173] In some embodiments, Figure 7 As shown, an embodiment of the present application also provides an electronic device 700, including a processor 701, a memory 702, and a computer program stored in the memory 702 and executable on the processor 701. When the program is executed by the processor 701, each process of the above-mentioned defect detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0174] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0175] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the various processes of the above-mentioned defect detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0176] On the other hand, the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it is implemented to perform the various processes of the above-mentioned defect detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0177] On the other hand, an embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned defect detection method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0178] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0179] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.
[0180] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can be essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.
[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A defect detection method, characterized in that: include: Processing the acquired image to be tested corresponding to the object to be tested to obtain a grayscale feature vector corresponding to each pixel in the image to be tested; The image to be measured is obtained by fusing multiple initial images of the object to be measured collected under different lighting conditions and the same collection angle of view; Based on the grayscale feature vector corresponding to each of the pixel points and the target grayscale threshold, obtaining a first grayscale feature set and a second grayscale feature set corresponding to the defect area in the image to be tested; Each grayscale feature vector in the first grayscale feature set is greater than the target grayscale threshold, and each grayscale feature vector in the second grayscale feature set is less than the target grayscale threshold; Based on each grayscale feature vector in the first grayscale feature set and each grayscale feature vector in the second grayscale feature set, a target grayscale feature value corresponding to the defect area is obtained; the target grayscale feature value is used to perform defect detection on the defect features in the defect area.
2. The defect detection method according to claim 1, characterized in that: The step of acquiring a target grayscale feature value corresponding to the defective area based on each grayscale feature vector in the first grayscale feature set and each grayscale feature vector in the second grayscale feature set includes: Based on each grayscale feature vector in the first grayscale feature set, obtaining a first mean value corresponding to the first grayscale feature set; based on each grayscale feature vector in the second grayscale feature set, obtaining a second mean value corresponding to the second grayscale feature set; Based on the difference between the first mean value and the second mean value, a target grayscale characteristic value corresponding to the defective area is obtained.
3. The defect detection method according to claim 2, characterized in that: The obtaining, based on each grayscale feature vector in the first grayscale feature set, a first mean corresponding to the first grayscale feature set includes: Acquire multiple grayscale feature vectors greater than a first grayscale threshold from the first grayscale feature set; the first grayscale threshold is greater than the smallest grayscale feature vector in the first grayscale feature set and less than the largest grayscale feature vector in the first grayscale feature set; Determine the mean corresponding to a plurality of grayscale feature vectors greater than the first grayscale threshold in the first grayscale feature set as the first mean; The obtaining, based on each grayscale feature vector in the second grayscale feature set, a second mean corresponding to the second grayscale feature set includes: Acquire multiple grayscale feature vectors less than a second grayscale threshold from the second grayscale feature set; the second grayscale threshold is greater than the smallest grayscale feature vector in the second grayscale feature set and less than the largest grayscale feature vector in the second grayscale feature set; The mean corresponding to a plurality of grayscale feature vectors in the second grayscale feature set that are smaller than the second grayscale threshold is determined as the second mean.
4. The defect detection method according to any one of claims 1 to 3, characterized in that: The step of processing the image to be tested corresponding to the acquired object to be tested to obtain a grayscale feature vector corresponding to each pixel in the image to be tested includes: The image to be tested is processed by using a photometric stereo method to obtain an image grayscale matrix corresponding to the image to be tested; the number of rows of the image grayscale matrix is the number of multiple pixel points in the image to be tested, and the number of columns of the image grayscale matrix is the number of frames of the multiple initial images; The image grayscale matrix is processed to obtain a grayscale feature vector corresponding to each pixel in the image to be tested.
5. The defect detection method according to claim 4, characterized in that: The processing of the image grayscale matrix to obtain the grayscale feature vector corresponding to each pixel in the image to be tested includes: Performing singular value decomposition processing on the image grayscale matrix to obtain a first matrix corresponding to the image grayscale matrix; the number of rows of the first matrix is the same as the number of rows of the image grayscale matrix, and the number of columns of the first matrix is the same as the number of columns of the image grayscale matrix; The first matrix is normalized to obtain a grayscale feature vector corresponding to each pixel in the image to be tested.
6. The defect detection method according to any one of claims 1 to 3, characterized in that: The grayscale feature vector includes the grayscale feature components corresponding to the pixel points under each image channel; the step of obtaining the first grayscale feature set and the second grayscale feature set corresponding to the defective area in the image to be tested based on the grayscale feature vectors corresponding to each pixel point and the target grayscale threshold value includes: Obtaining the grayscale feature component corresponding to each pixel point in the target image channel among the multiple image channels; All grayscale feature components whose grayscale feature components corresponding to the pixels in the defect area are greater than the target grayscale threshold are determined as the first grayscale feature set; all grayscale feature components whose grayscale feature components corresponding to the pixels in the defect area are less than the target grayscale threshold are determined as the second grayscale feature set.
7. A defect detection device, characterized in that: include: A first processing module is used to process the image to be tested corresponding to the acquired object to be tested, and obtain the grayscale feature vector corresponding to each pixel point in the image to be tested; The image to be measured is obtained by fusing multiple initial images of the object to be measured collected under different lighting conditions and the same collection angle of view; A second processing module, configured to obtain a first grayscale feature set and a second grayscale feature set corresponding to the defective area in the image to be tested based on the grayscale feature vector corresponding to each of the pixel points and a target grayscale threshold; Each grayscale feature vector in the first grayscale feature set is greater than the target grayscale threshold, and each grayscale feature vector in the second grayscale feature set is less than the target grayscale threshold; The third processing module is used to obtain the target grayscale feature value corresponding to the defect area based on each grayscale feature vector in the first grayscale feature set and each grayscale feature vector in the second grayscale feature set; the target grayscale feature value is used to perform defect detection on the defect features in the defect area.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the defect detection method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the defect detection method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the defect detection method according to any one of claims 1 to 6 is implemented.