Defect detection method and apparatus, device, and storage medium

By using an image acquisition device set with a symmetric light source in industrial production, the image of the object surface is collected and the height distribution information of the target defect area is determined, which solves the problem of low accuracy in object surface defect detection and achieves more efficient defect detection.

WO2024239630A1PCT designated stage expired Publication Date: 2024-11-28CONTEMPORARY AMPEREX TECHNOLOGY CO LTD

Patent Information

Application Number
PCT/CN2023/140259
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-23
Filing Date
2023-12-20
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

In industrial production, the accuracy of object surface defect detection is not high, especially due to the variable locations and different shapes of defects, which makes it difficult for the prior art to effectively detect.

Method used

The image acquisition device is used to collect images under the illumination of the first light source and the second light source, and the symmetric light source settings are used to better obtain the morphological changes of the object surface, determine the height distribution information of the target defect area, and perform defect detection based on this information. Improve the accuracy and efficiency of detection.

Benefits of technology

Through the use of symmetric light source settings and the utilization of height distribution information, the accuracy and efficiency of defect detection are significantly improved, and the online detection needs of high-speed production beats can be better met.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023140259_28112024_PF_FP_ABST
    Figure CN2023140259_28112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided are a defect detection method and apparatus, a device, and a storage medium. The defect detection method comprises: obtaining a first image and a second image acquired by an image acquisition device (11) from an object surface to be detected respectively under the irradiation of a first light source (12) and a second light source (13), wherein emitted light (r1) of the first light source and emitted light (r2) of the second light source are symmetrical with respect to an optical axis (ro) of the image acquisition device; on the basis of the first image and the second image, performing defect detection on the object surface to obtain a target defect area of the object surface; on the basis of the first image and the second image, determining height distribution information of the target defect area surface; and on the basis of the height distribution information, performing defect detection on the target defect area to obtain a detection result. The accuracy and efficiency of defect detection of object surfaces can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Defect detection method, device and equipment, and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on May 23, 2023, with application number 202310587622.8 and application name “Defect Detection Method, Device, Equipment and Readable Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to, but is not limited to, the field of image processing, and in particular to a defect detection method, apparatus and equipment, and storage medium. Background Art

[0003] In industrial production, defect detection on the surfaces of products or equipment is crucial. For example, in lithium battery production, defect detection of battery cells can be used to identify defective products, thereby improving product yield. However, surface defects often vary in location and form, resulting in limited accuracy in related defect detection technologies.

[0004] Summary of the Invention

[0005] In view of this, the embodiments of the present application provide at least one defect detection method, apparatus and equipment, and storage medium, which can improve the accuracy and efficiency of surface defect detection of objects.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The present invention provides a defect detection method, which includes:

[0008] Acquire a first image and a second image of the surface of the object to be inspected, captured by an image acquisition device under illumination of a first light source and a second light source, respectively; wherein the light emitted by the first light source and the light emitted by the second light source are symmetrical with respect to an optical axis of the image acquisition device;

[0009] performing defect detection on the surface of the object based on the first image and the second image to obtain a target defect area on the surface of the object;

[0010] determining height distribution information of a surface of the target defect area based on the first image and the second image;

[0011] Based on the height distribution information, defect detection is performed on the target defect area to obtain a detection result.

[0012] The present invention provides a defect detection device, comprising:

[0013] a first acquisition module, configured to acquire a first image and a second image of the surface of the object to be inspected, captured by an image acquisition device under illumination of a first light source and a second light source, respectively; wherein the light emitted by the first light source and the light emitted by the second light source are symmetrical with respect to an optical axis of the image acquisition device;

[0014] a first detection module, configured to perform defect detection on the surface of the object based on the first image and the second image to obtain a target defect area on the surface of the object;

[0015] A first determining module, configured to determine height distribution information of a surface of the target defect area based on the first image and the second image;

[0016] The second detection module is used to perform defect detection on the target defect area based on the height distribution information to obtain a detection result.

[0017] An embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0018] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements some or all of the steps in the above method when executed by a processor.

[0019] An embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, the steps in the above method are implemented.

[0020] In an embodiment of the present application, an image acquisition device acquires first and second images of a surface of an object to be inspected, respectively, under illumination from a first light source and a second light source; wherein the light emitted by the first light source and the light emitted by the second light source are symmetrical with respect to the optical axis of the image acquisition device; based on the first and second images, defect detection is performed on the surface of the object to obtain a target defect region on the surface of the object; based on the first and second images, height distribution information of the surface of the target defect region is determined; and based on the height distribution information, defect detection is performed on the target defect region to obtain a detection result. Thus, on the one hand, because the light emitted by the first and second light sources is symmetrical with respect to the optical axis of the image acquisition device, the morphological changes of the surface of the object can be better obtained based on the first and second images, thereby more accurately determining the target defect region and the height distribution information of the surface of the target defect region, and then re-defect detection is performed on the target defect region based on the height distribution information, thereby improving the accuracy of defect detection; on the other hand, because only the height distribution information of the surface of the target defect region can be determined without calculating the height distribution information of the entire surface of the object, the amount of calculation can be reduced, thereby improving the efficiency of defect detection, and better meeting the requirements of online inspection at high production rates.

[0021] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0023] FIG1A is a schematic diagram of an implementation flow of a defect detection method provided in an embodiment of the present application;

[0024] FIG1B is a schematic diagram illustrating the arrangement of a first light source and a second light source in a defect detection method provided in an embodiment of the present application;

[0025] FIG1C is a schematic diagram of a first image and a second image acquired in a defect detection method provided in an embodiment of the present application;

[0026] FIG2A is a schematic diagram of an implementation flow of a defect detection method provided in an embodiment of the present application;

[0027] FIG2B is a schematic diagram showing the distribution of linear regions in a target defect area provided by an embodiment of the present application;

[0028] FIG2C is a schematic diagram of a first sub-image and a second sub-image provided in an embodiment of the present application;

[0029] FIG2D is a schematic diagram illustrating the filtering effect of the normal vector angles of each sub-region block within a linear region in a defect detection method provided by an embodiment of the present application;

[0030] FIG2E is a schematic diagram showing the filtering effect of the heights of sub-region blocks within a linear region in a defect detection method provided by an embodiment of the present application;

[0031] FIG3A is a schematic diagram of an implementation flow of a defect detection method provided in an embodiment of the present application;

[0032] FIG3B is a schematic diagram illustrating the arrangement of a first light source, a second light source, and a third light source in a defect detection method provided in an embodiment of the present application;

[0033] FIG3C is a schematic diagram showing a comparison of a first image, a second image, and a third image obtained in a defect detection method provided in an embodiment of the present application;

[0034] FIG4A is a schematic diagram of an implementation flow of a defect detection method provided in an embodiment of the present application;

[0035] 4B and 4C are respectively a schematic three-dimensional view and a schematic top view of a three-dimensional height map provided in an embodiment of the present application;

[0036] FIG5A is a schematic diagram of the structure of a defect detection system provided in an embodiment of the present application;

[0037] FIG5B is a schematic structural diagram of a battery cell top patch provided in an embodiment of the present application;

[0038] FIG5C is a schematic diagram of an implementation flow of a defect detection method provided in an embodiment of the present application;

[0039] FIG5D is a line graph comparing the height of the target defect area surface estimated in a defect detection method provided by an embodiment of the present application and the height of the target defect area surface captured by a three-dimensional camera;

[0040] FIG6 is a schematic diagram of the structure of a defect detection device provided in an embodiment of the present application;

[0041] FIG7 is a schematic diagram of a hardware entity of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions of this application are further elaborated in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0043] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0044] The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" can be interchanged with a specific order or sequence where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing this application only and are not intended to limit this application.

[0046] The present application provides a defect detection method that can be executed by a processor of a computer device. The computer device may include a server, laptop, tablet, desktop computer, industrial computer, smart TV, set-top box, mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device), or other device with data processing capabilities. FIG1A is a schematic diagram of an implementation flow of a defect detection method provided in an embodiment of the present application. As shown in FIG1A , the method may include the following steps S101 to S104:

[0047] Step S101, obtaining a first image and a second image of the surface of the object to be detected captured by an image acquisition device under the illumination of a first light source and a second light source respectively; wherein the emitted light of the first light source and the emitted light of the second light source are symmetrical with respect to the optical axis of the image acquisition device.

[0048] Here, the image acquisition device may include but is not limited to an industrial camera, a camera, a scanner, a charge coupled device (CCD) image array, or a complementary metal oxide semiconductor (CMOS) image array, or any other suitable device capable of acquiring image data.

[0049] The object to be inspected can be any suitable product or device to be inspected for surface defects, and the present invention is not limited thereto. For example, the object to be inspected can include, but is not limited to, at least one of a coated cell, a battery, an industrial component, a wafer, and the like.

[0050] In implementation, the first light source and the second light source may be any suitable light source, and the embodiments of the present application are not limited thereto. In some embodiments, the first light source may include a first white light source, the second light source may include a second white light source, and the object to be detected may include a battery cell coated with a blue film.

[0051] The first and second light sources can be disposed on either side of the image capture device, respectively, and the light emitted by the first and second light sources are symmetrical with respect to the optical axis of the image capture device. The light emitted by the first and second light sources form a first angle with the optical axis of the image capture device, and the light emitted by the second light source form a second angle with the optical axis of the image capture device, with both the first and second angles being acute. It should be noted that symmetry between the light emitted by the first and second light sources with respect to the optical axis of the image capture device means that the first and second angles are equal or approximately equal. For example, if the light emitted by the first and second light sources are absolutely symmetrical with respect to the optical axis of the image capture device, then the first and second angles are absolutely equal. For another example, the light emitted by the first and second light sources can be symmetrical with respect to the light source of the image capture device within a narrow error range, meaning that the first and second angles can be approximately equal, for example, the difference between the first and second angles can be within 1 degree.

[0052] When the first light source is used to illuminate the surface of the object, the image acquisition device can capture an image of the surface of the object to obtain a first image. When the second light source is used to illuminate the surface of the object, the image acquisition device can capture an image of the surface of the object to obtain a second image.

[0053] In some embodiments, the first image and the second image may be acquired separately through two consecutive scans. For example, the image acquisition device acquires the second image after acquiring the first image, or the image acquisition device acquires the first image after acquiring the second image.

[0054] In some embodiments, the image acquisition device is a line array camera, and the first and second images can be acquired using a time-sharing stroboscopic method. For example, a time-sharing stroboscopic controller can be used to switch the light source type each time the line array camera acquires a line of images, so that the images captured under the first and second light sources are arranged in an orderly, spaced relationship within the original image. After acquisition, the original image is split and reassembled to obtain the first image corresponding to the first light source and the second image corresponding to the second light source in a single scan. Using a time-sharing stroboscopic method, the first and second images can be acquired in a single scan, reducing image acquisition costs, improving image acquisition efficiency, and achieving better imaging results, thereby improving defect detection efficiency.

[0055] In some embodiments, as shown in FIG1B , the image capture device 11, the first light source 12, and the second light source 13 can all be positioned above the object 14 to be inspected. The first light source 12 and the second light source 13 can be symmetrically positioned on either side of the image capture device. That is, the light beam r1 emitted by the first light source 12 and the light beam r2 emitted by the second light source 13 are symmetrical with respect to the optical axis R0 of the image capture device 11. The emitted light beam r1 forms a first angle a1 with the optical axis R0, and the emitted light beam r2 forms a second angle a2 with the optical axis R0. The first angle a1 and the second angle a2 are both acute angles, and the first angle a1 and the second angle a2 are equal or approximately equal. The emitted light beams r1 and r2 lie in the same plane as the optical axis R0 and intersect the surface of the object 14. It will be appreciated that the surface of the object to be inspected has diffuse reflective properties, and thus the symmetrical positioning of the first and second light sources can better highlight slight morphological changes on the object's surface. Consequently, the combined first and second images can minimize reflections and overexposure, thereby better presenting defects on the surface of the object to be inspected.

[0056] Figure 1C is a schematic diagram of the first image and the second image obtained in a defect detection method provided in an embodiment of the present application. As shown in Figure 1C, both the first image p1 and the second image p2 can better highlight slight morphological changes on the surface of the object, and can suppress reflections and reduce overexposure to the greatest extent.

[0057] Step S102: performing defect detection on the surface of the object based on the first image and the second image to obtain a target defect area on the surface of the object.

[0058] Here, based on the first image and the second image, any suitable machine vision-based defect detection method can be used to perform defect detection on the object surface to obtain at least one target defect area on the object surface where a defect may exist. The defects on the object surface may include, but are not limited to, at least one of a pit defect (such as a scratch, a dent, etc.), a protrusion defect (such as a foreign object, an air bubble, etc.), and the like.

[0059] In some embodiments, feature extraction may be performed on the first image and the second image respectively to obtain extracted first image features and second image features, and then based on the first image features and the second image features, defects on the surface of the object may be detected to obtain a target defect area.

[0060] In some embodiments, a morphological image with enhanced morphological features can be obtained based on the first image and the second image, and features can be extracted from the morphological image to obtain extracted third image features. Based on the third image features, defects on the surface of the object can be detected to obtain a target defect area.

[0061] In some embodiments, a pre-trained first neural network model may be used to perform defect detection on the surface of an object based on the first image and the second image to obtain a target defect area on the surface of the object.

[0062] Step S103 : determining height distribution information of the surface of the target defect area based on the first image and the second image.

[0063] Here, the height distribution information of the target defect area surface may include, but is not limited to, the height of each location on the target defect area surface relative to a preset reference plane. The preset reference plane may be set based on actual conditions, and this embodiment of the present application does not limit this. For example, the reference plane may be a pre-set plane located below the surface of the object, or a pre-set plane located above the surface of the object. In another example, the reference plane may be obtained by fitting the boundaries of the reference plane.

[0064] Because the surface of an object has diffuse reflection properties, and the intensity of the diffuse reflection is ultimately reflected in the brightness difference of the image, the brightness difference (i.e., the pixel value difference) between each pixel in the first image and the second image can reflect the intensity of the diffuse reflection on the surface of the target defect area. The intensity of the diffuse reflection on the surface of the target defect area can also reflect the surface morphology at each location on the surface of the target defect area. In this way, based on the first image and the second image, the surface morphology at each location on the surface of the target defect area can be determined, and thus the height distribution information of the surface of the target defect area can be determined.

[0065] In some embodiments, the normal vector angle at at least one location on the surface of the target defect region can be estimated based on the first image and the second image to obtain an estimated normal vector angle distribution for the target defect region surface. Based on the normal vector angle distribution for the target defect region surface, the height at at least one location on the surface of the target defect region can be estimated to obtain estimated height distribution information for the target defect region surface. Here, the normal vector angle is the angle between the normal vector at the corresponding location and the optical axis of the image acquisition device.

[0066] In some embodiments, a pre-trained second neural network model can be used to estimate the height distribution information of the target defect area surface based on the first image and the second image to obtain the estimated height distribution information of the target defect area surface.

[0067] Step S104 : performing defect detection on the target defect area based on the height distribution information to obtain a detection result.

[0068] Here, after determining the target defect area, the target defect area can be re-detected for defects based on the height distribution information of the surface of the target defect area to obtain a detection result. The detection result may include, but is not limited to, at least one of a first result indicating that a defect exists in the target defect area, a second result indicating that no defect exists in the target defect area, and the like.

[0069] During implementation, those skilled in the art may use any appropriate method to perform defect detection on the target defect area based on the height distribution information according to actual conditions, and the embodiments of the present application are not limited to this.

[0070] In some embodiments, the highest point and / or lowest point on the surface of the target defect region can be determined based on the height distribution information; and a detection result for the target defect region can be determined based on the height of the highest point and / or lowest point. For example, if the height of the highest point and / or lowest point exceeds a preset height threshold, the detection result can be determined as a first result indicating that a defect exists in the target defect region. For another example, if the height of neither the highest point nor the lowest point exceeds a preset height threshold, the detection result can be determined as a second result indicating that no defect exists in the target defect region.

[0071] In some embodiments, the variance of the surface height of the target defect region can be determined based on the height distribution information, and the detection result of the target defect region can be determined based on the variance of the surface height of the target defect region. For example, if the variance of the surface height of the target defect region is greater than a preset variance threshold, the detection result can be determined as a first result indicating that a defect exists in the target defect region. For another example, if the variance of the surface height of the target defect region is not greater than a preset variance threshold, the detection result can be determined as a second result indicating that no defect exists in the target defect region.

[0072] In an embodiment of the present application, an image acquisition device acquires first and second images of a surface of an object to be inspected, respectively, under illumination from a first light source and a second light source; wherein the light emitted by the first light source and the light emitted by the second light source are symmetrical with respect to the optical axis of the image acquisition device; based on the first and second images, defect detection is performed on the surface of the object to obtain a target defect region on the surface of the object; based on the first and second images, height distribution information of the surface of the target defect region is determined; and based on the height distribution information, defect detection is performed on the target defect region to obtain a detection result. Thus, on the one hand, because the light emitted by the first and second light sources is symmetrical with respect to the optical axis of the image acquisition device, the morphological changes of the surface of the object can be better obtained based on the first and second images, thereby more accurately determining the target defect region and the height distribution information of the surface of the target defect region, and then re-defect detection is performed on the target defect region based on the height distribution information, thereby improving the accuracy of defect detection; on the other hand, because only the height distribution information of the surface of the target defect region can be determined without calculating the height distribution information of the entire surface of the object, the amount of calculation can be reduced, thereby improving the efficiency of defect detection, and better meeting the requirements of online inspection at high production rates.

[0073] In some embodiments, the above step S104 may include the following steps S111 to S112:

[0074] Step S111 : determining the height of each sub-region block in the target defect region based on the height distribution information.

[0075] Step S112 : when the absolute value of the height of at least one of the sub-region blocks is greater than a preset height threshold, determining that the detection result is a first result indicating that a defect exists in the target defect region.

[0076] Here, the target defect area may include a plurality of sub-area blocks.

[0077] In some embodiments, the height distribution information of the target defect region surface may include the height of each sub-region relative to a reference plane. The reference plane is determined based on the boundary of the target defect region. It is understood that the height of the sub-region above the reference plane is greater than 0, the height of the sub-region below the reference plane is less than 0, and the height of the sub-region within the reference plane is 0.

[0078] The absolute value of the height of the sub-region block may be the height or depth of the sub-region block relative to the reference plane. If the absolute value of the height of at least one sub-region block is greater than a preset height threshold, it indicates that a sub-region block having a height or depth greater than the preset height threshold exists in the target defect region. Therefore, it can be determined that a relatively obvious pit or protrusion exists in the target defect region, and the detection result can be determined as a first result indicating the presence of a defect in the target defect region.

[0079] During implementation, the preset height threshold may be pre-set by those skilled in the art according to actual conditions, and the present application embodiment does not limit this. For example, the preset height threshold may be 100 microns.

[0080] In some implementations, the above step S104 may further include the following step S121:

[0081] Step S121 : when the absolute value of the height of each sub-region block is not greater than a preset height threshold, determining that the detection result is a second result indicating that there is no defect in the target defect region.

[0082] In the above embodiment, the height of each sub-region within the target defect region is determined based on the height distribution information of the surface of the target defect region. If the absolute value of the height of at least one sub-region is greater than a preset height threshold, the detection result is determined to be a first result indicating the presence of a defect in the target defect region. In this way, the detection result of the target defect region can be controlled by the preset height threshold, allowing for rapid and accurate defect re-assessment of the target defect region, thereby improving the accuracy of defect detection and reducing the number of missed defects.

[0083] The present invention provides a defect detection method that can be executed by a processor of a computer device. FIG2A is a schematic diagram of an implementation flow of a defect detection method provided by an embodiment of the present invention. As shown in FIG2A , the method can include the following steps S201 to S204:

[0084] Step S201, obtaining a first image and a second image captured by an image acquisition device of the surface of the coating film covered on the target object to be inspected under the illumination of a first white light source and a second white light source, respectively; wherein the emitted light of the first white light source and the emitted light of the second white light source are symmetrical with respect to the optical axis of the image acquisition device.

[0085] Here, the surface of the target object to be detected can be covered with any suitable covering film. In implementation, the covering film can include but is not limited to at least one of a blue film, a yellow film, a green film, etc., and the embodiment of the present application is not limited to this.

[0086] In some embodiments, the target object may be a battery cell, and the coating may be a blue film covering the surface of the battery cell. A battery cell refers to an electrochemical cell containing positive and negative electrodes and is generally not used directly. The battery cell is generally provided with an external casing, typically made of aluminum, and is wrapped with a layer of plastic film. This plastic film is generally blue, hence the name blue film.

[0087] In some embodiments, the battery cell may include a lithium battery cell.

[0088] Step S202 : performing defect detection on the surface of the coating film based on the first image and the second image to obtain a target defect area on the surface of the coating film.

[0089] Step S203 : determining height distribution information of the surface of the target defect area based on the first image and the second image.

[0090] Step S204: performing defect detection on the target defect area based on the height distribution information to obtain a detection result.

[0091] Here, steps S202 to S204 correspond to the aforementioned steps S102 to S104 respectively, and the implementation of the aforementioned steps S102 to S104 may be referred to.

[0092] In an embodiment of the present application, an image acquisition device acquires a first image and a second image of the surface of a coating film covering a target object to be inspected, respectively, under the illumination of a first white light source and a second white light source; wherein the light emitted by the first white light source and the light emitted by the second white light source are symmetrical with respect to the optical axis of the image acquisition device; based on the first image and the second image, defect detection is performed on the surface of the coating film to obtain a target defect area on the surface of the coating film; based on the first image and the second image, height distribution information of the surface of the target defect area is determined; based on the height distribution information, defect detection is performed on the target defect area to obtain a detection result. It can be understood that under appropriate illumination intensity, the white light source can reduce interference from the background on the surface of the coating film and can also achieve a good imaging effect for defects with slight morphological changes on the surface of the coating film; and the symmetrical arrangement of the first white light source and the second white light source can greatly highlight the morphological changes of defects on the surface of the coating film, and is conducive to determining the height distribution information of the surface of the target defect area, thereby improving the accuracy of defect detection.

[0093] In some embodiments, the above step S203 may include the following steps S211 to S212:

[0094] Step S211 : determining a target sub-image corresponding to the target defect area based on the first image and the second image.

[0095] Here, the target sub-image may be an image for presenting a target defect area.

[0096] In some embodiments, the target sub-image may include a partial image corresponding to the target defect area in the first image and a partial image corresponding to the target defect area in the second image.

[0097] In some embodiments, the target sub-image may include a difference image between a partial image corresponding to the target defect area in the first image and a partial image corresponding to the target defect area in the second image.

[0098] In some embodiments, the target sub-image may include a local image corresponding to the target defect area in the difference image between the first image and the second image.

[0099] Step S212 : Based on the pixel value variation between each pixel point in the target sub-image, height modeling is performed on the target defect area to obtain height distribution information of the surface of the target defect area.

[0100] Here, any suitable height modeling algorithm can be used to perform height modeling on the target defect area based on the pixel value change between each pixel point in the target sub-image to obtain height distribution information on the surface of the target defect area. The embodiments of the present application are not limited to this.

[0101] In some embodiments, a pre-trained third neural network model can be used to perform height modeling on the target defect area based on the pixel value variation between each pixel point in the target sub-image to obtain height distribution information on the surface of the target defect area.

[0102] In some embodiments, the normal vector angle at at least one position on the surface of the target defect area can be estimated based on the pixel value change between each pixel point in the target sub-image to obtain the estimated normal vector angle distribution of the target defect area surface; based on the normal vector angle distribution of the target defect area surface, the height at at least one position on the surface of the target defect area can be estimated to obtain the estimated height distribution information of the target defect area surface.

[0103] In the embodiment of the present application, a target sub-image corresponding to the target defect area is determined based on the first image and the second image. Based on the pixel value variations between each pixel in the target sub-image, a height model of the target defect area is generated to obtain height distribution information on the surface of the target defect area. In this way, the height distribution information on the surface of the target defect area can be quickly and accurately determined by combining the first image and the second image.

[0104] In some embodiments, the orthographic projection of the light emitted by the first light source on the surface of the coating extends along a first direction, the target defect area includes multiple linear areas distributed side by side along a second direction, each of the linear areas includes multiple sub-region blocks distributed sequentially along the first direction, the second direction is perpendicular to the first direction, and the height distribution information of the surface of the target defect area includes the height of each sub-region block relative to a reference plane, and the reference plane is determined based on the boundary of the target defect area. The above-mentioned step S212 may include the following steps S221 to S222:

[0105] Step S221, based on the pixel value change between each pixel point in the target sub-image, the normal vector angle of each sub-region block is estimated to obtain the estimated normal vector angle of each sub-region block; wherein the normal vector angle of each sub-region block is the angle between the normal vector of the sub-region block and the optical axis.

[0106] Here, since the emitted light of the first light source and the emitted light of the second light source are symmetrical relative to the optical axis of the image acquisition device, the positive projection of the emitted light of the first light source on the surface of the coating film extends along the first direction, and the positive projection of the emitted light of the second light source on the surface of the coating film extends in the opposite direction of the first direction.

[0107] Figure 2B is a schematic diagram illustrating the distribution of linear regions within a target defect region according to an embodiment of the present application. As shown in Figure 2B , a first direction X represents the direction of the orthographic projection of light emitted by a first light source onto the surface of the coating film. A second direction Y is perpendicular to the first direction X. The target defect region 200 may include multiple linear regions 210 arranged side by side along the second direction Y. Each linear region 210 includes multiple sub-region blocks 211 arranged sequentially along the first direction Y.

[0108] It can be understood that, in the target sub-image, each sub-region block may correspond to a pixel point in the image, and each linear region may correspond to a row of pixel points along the first direction in the image.

[0109] Since white light irradiates an object, diffuse reflection occurs. The intensity of diffuse reflection is ultimately reflected in the brightness difference of the corresponding image. For example, the diffuse reflection intensity can be calculated as shown in the following formula 2-1:

[0110] Among them, L d is the diffuse reflection intensity, K dis the absorption coefficient, I is the light energy, r is the distance between the incident light source and the reflecting plane, and n·l is the dot product of the surface normal n and the incident direction l of the light source. It can be seen that for surfaces with morphological changes, the diffuse reflection intensity is directly reflected in the angular change of the surface normal. In other words, the angular change of the object surface can be determined based on the brightness difference of the object surface image (i.e., the change in the pixel value of each pixel in the object surface image). Therefore, based on the pixel value of each pixel in the target sub-image, the normal vector angle of each sub-region block in the target defect area can be estimated to obtain the estimated normal vector angle of each sub-region block.

[0111] During implementation, those skilled in the art can estimate the normal vector angle of each sub-region block based on the change in pixel value between each pixel point in the target image in any appropriate manner to obtain the estimated normal vector angle of each sub-region block. The embodiments of the present application are not limited to this.

[0112] Step S222 : estimating the height of each sub-region block based on the normal vector angle of each sub-region block to obtain an estimated height of each sub-region block.

[0113] Here, since the normal vector angle of the sub-region block can reflect the slope of the target defect area surface at the sub-region block, the height of each sub-region block can be estimated according to the normal vector angle of each sub-region block.

[0114] During implementation, those skilled in the art may estimate the height of each sub-region block based on the normal vector angle of each sub-region block in an appropriate manner according to actual circumstances, and the embodiments of the present application are not limited to this.

[0115] In some embodiments, the slope of the target defect area surface at each sub-region block can be determined based on the normal vector angle of each sub-region block, and the height change of each sub-region block can be determined based on the slope corresponding to each sub-region block; and the height of each sub-region block can be determined based on the height change of each sub-region block.

[0116] In some embodiments, a pre-trained fourth neural network model may be used to estimate the height of each sub-region block based on the normal vector angle of each sub-region block to obtain the estimated height of each sub-region block.

[0117] In the above embodiment, the normal vector angle of each sub-region block is estimated based on the pixel value variation between each pixel point in the target sub-image, thereby obtaining the estimated normal vector angle of each sub-region block. Based on the normal vector angle of each sub-region block, the height of each sub-region block is estimated to obtain the estimated height of each sub-region block. In this way, the principle of diffuse reflection can be utilized to quickly and accurately estimate the height of each sub-region block based on the target sub-image.

[0118] In some embodiments, the target sub-image includes a first sub-image and a second sub-image. The above step S211 may include the following steps S231 to S232:

[0119] Step S231 : determining a first image area corresponding to the target defect area in the first image as the first sub-image.

[0120] Here, a first image region corresponding to the target defect region may be segmented from the first image as a first sub-image.

[0121] Step S232 : determining a second image region corresponding to the target defect region in the second image as the second sub-image.

[0122] Here, the second image region corresponding to the target defect region may be segmented from the second image as the first sub-image.

[0123] Figure 2C is a schematic diagram of a first sub-image and a second sub-image provided in an embodiment of the present application, wherein the first sub-image a is a first image area corresponding to the target defect area in the first image, and the second sub-image b is a second image area corresponding to the target defect area in the second image; since the first sub-image a and the second sub-image b both correspond to the target defect area, there is a one-to-one correspondence between the pixels in the first sub-image a and the second sub-image b.

[0124] In some embodiments, the above step S221 may include the following steps S241 to S244:

[0125] Step S241 : determining a pixel value variation of each pixel point in the first sub-image in the first direction.

[0126] Here, those skilled in the art may determine the pixel value variation in the first direction of each pixel point in the first sub-image in any appropriate manner according to actual conditions.

[0127] In some embodiments, the pixel value change of a certain pixel point in the first direction may be the change in the pixel value of the pixel point relative to the pixel value of the pixel point immediately preceding the pixel point in the first direction. For example, the difference between the pixel value of a certain pixel point in the first sub-image and the pixel value of the pixel point immediately preceding the pixel point in the first direction may be determined as the pixel value change of the pixel point in the first direction. For another example, the difference between the logarithm of the pixel value of a certain pixel point in the first sub-image and the logarithm of the pixel value of the pixel point immediately preceding the pixel point in the first direction may be determined as the pixel value change of the pixel point in the first direction.

[0128] In some embodiments, the pixel value change of a certain pixel in the first direction may be the change in the pixel value of the next pixel in the first direction relative to the pixel value of the pixel. For example, the difference between the pixel value of the next pixel in the first direction of a certain pixel in the first sub-image and the pixel value of the pixel can be determined as the pixel value change of the pixel in the first direction. For another example, the difference between the logarithm of the next pixel value in the first direction of a certain pixel in the first sub-image and the logarithm of the pixel value of the pixel in the first direction can be determined as the pixel value change of the pixel in the first direction.

[0129] For example, the first sub-image I can be determined by the following formula 2-2: A Pixel x in i The pixel value change ΔI in the first direction A,R (x i ): ΔI A,R (x i )=log I A,R (x i +Δx i )-log I A,R (x i ) (2-2);

[0130] Among them, I A,R (x i ) is the first sub-image I A Middle pixel x i The pixel value at I A,R (x i +Δx i ) is the first sub-image I A Middle pixel x i The next pixel point x in the first direction i +Δx i The pixel value at .

[0131] It should be noted that the next pixel point in the first direction of a pixel point corresponding to a certain sub-region block in the first sub-image may be the pixel point corresponding to the next sub-region block in the first direction of the certain sub-region block in the first sub-image; the previous pixel point in the first direction of a pixel point corresponding to a certain sub-region block in the first sub-image may be the pixel point corresponding to the previous sub-region block in the first direction of the certain sub-region block in the first sub-image.

[0132] Step S242: Determine the pixel value variation of each pixel point in the second sub-image in the first direction.

[0133] Here, the pixel value variation of each pixel point in the second sub-image in the first direction may be determined by referring to the method of determining the pixel value variation of each pixel point in the first sub-image in the first direction in the aforementioned step S241 .

[0134] For example, the second sub-image I can be determined by the following formula 2-3: B Pixel x in i The pixel value change ΔI in the first direction B,R (x i ): ΔI B,R (x i )=log I B,R (x i +Δx i )-log I B,R (x i ) (2-3);

[0135] Among them, I B,R (x i ) is the second sub-image I B Middle pixel x i The pixel value at I B,R (x i +Δx i ) is the second sub-image I B Middle pixel x i The next pixel point x in the first direction i +Δx i The pixel value at .

[0136] Step S243: For each of the sub-region blocks, determine the target pixel value change corresponding to the sub-region block based on the difference between the pixel value change of the pixel point corresponding to the sub-region block in the first sub-image in the first direction and the pixel value change of the pixel point corresponding to the sub-region block in the second sub-image in the first direction.

[0137] In step S244 , for each sub-region block, a normal vector angle of the sub-region block is determined based on target pixel value changes corresponding to the starting sub-region block and the sub-region block distributed along the first direction in the linear region where the sub-region block is located.

[0138] Here, since the first white light source and the second white light source are usually far away from the object to be detected, the absorption coefficient K in the above formula 2-1 in the local range is d , light energy I, and the distance r between the incident light source and the reflecting plane can be considered constants. The change in the normal angle of each sub-region on the surface of the coating can be expressed as the discrete line integral of the normal angle of each sub-region along the first direction. In other words, the normal angle of a sub-region can be obtained by accumulating the changes in the normal vectors corresponding to the starting sub-region distributed along the first direction in the linear region where the sub-region is located to the sub-region. The change in the normal vector corresponding to the sub-region can be determined based on the change in the target pixel value corresponding to the sub-region.

[0139] In some embodiments, the first sub-image I may be converted into A Pixel x in i In the linear region where the corresponding sub-region block is located, the normal vector changes corresponding to the starting sub-region block distributed along the first direction to the sub-region block are accumulated to obtain the normal vector θ of the sub-region block:

[0140] Among them, Δθ(x i ) is the normal vector change corresponding to the sub-region block.

[0141] In some embodiments, the first sub-image I may be determined by the following formula 2-5: A Pixel x in i The normal vector change Δθ(x i ): Δθ(x i )=(-sin(2θ A ) / 2)*[ΔI A,R (x i )-ΔI B,R (x i )] (2-5);

[0142] Among them, θ A is the first angle between the light emitted by the first light source and the optical axis of the image acquisition device, ΔI A,R (x i )-ΔI B,R (x i ) is the target pixel value change corresponding to the sub-region block.

[0143] In the above embodiment, for each sub-region block, a target pixel value change corresponding to the sub-region block is determined based on the difference between the pixel value change in the first direction for the pixel corresponding to the sub-region block in the first sub-image and the pixel value change in the first direction for the pixel corresponding to the sub-region block in the second sub-image. Furthermore, a normal vector angle for the sub-region block is determined based on the target pixel value changes corresponding to the starting sub-region block and the sub-region block distributed along the first direction within the linear region where the sub-region block is located. Thus, in determining the normal vector angle for each sub-region block, only the target pixel value change corresponding to each sub-region block within the linear region where the sub-region block is located needs to be considered, thereby reducing the amount of computation and improving the efficiency of defect detection. Moreover, the target pixel value change corresponding to the sub-region block is determined based on the difference between the pixel value change in the first direction of the pixel point corresponding to the sub-region block in the first sub-image and the pixel value change in the first direction of the pixel point corresponding to the sub-region block in the second sub-image. Since the first sub-image and the second sub-image are collected under the illumination of two symmetrically arranged white light sources, the difference between the pixel value change in the first direction of the pixel point corresponding to the sub-region block in the first sub-image and the pixel value change in the first direction of the pixel point corresponding to the sub-region block in the second sub-image can more accurately reflect the morphological changes of the surface of the target defect area, thereby improving the accuracy of the estimated normal vector angles of each sub-region block.

[0144] In some embodiments, the target sub-image includes a third sub-image. The above step S211 may include the following steps S251 or S252:

[0145] Step S251, determining a first image area corresponding to the target defect area in the first image, and determining a second image area corresponding to the target defect area in the second image; and determining the difference between the first image area and the second image area as the third sub-image.

[0146] Here, the third sub-image may be obtained by performing pixel-by-pixel subtraction on the first image area and the second image area.

[0147] Step S252 : determining a first difference image between the first image and the second image, and determining a third image region corresponding to the target defect region in the first difference image as the third sub-image.

[0148] Here, the first image and the second image may be subtracted pixel by pixel to obtain a first difference image.

[0149] In the above embodiment, the third sub-image used for height modeling can be determined simply and quickly. Furthermore, because the third sub-image is determined based on the difference between the target defect area in the first and second images, the surface concave and convex morphology of the target defect area can be better highlighted, enhancing the morphological features of the target defect area in the third sub-image. This helps improve the accuracy of height modeling and, in turn, the accuracy of defect detection.

[0150] In some embodiments, the above step S221 may include the following steps S261 to S262:

[0151] Step S261 : determining a pixel value variation of each pixel point in the third sub-image in the first direction.

[0152] Here, the pixel value variation of each pixel point in the third sub-image in the first direction may be determined by referring to the method of determining the pixel value variation of each pixel point in the first sub-image in the first direction in the aforementioned step S241 .

[0153] For example, the third sub-image I can be determined by the following formula 2-6: C Pixel x in i The pixel value change ΔI in the first direction C,R (x i ): ΔI C,R (x i )=log I C,R (x i +Δx i )-log I C,R (x i ) (2-6);

[0154] Among them, I C,R (x i ) is the third sub-image I C Middle pixel x i The pixel value at I C,R (x i +Δx i ) is the third sub-image I C Middle pixel x i The next pixel point x in the first direction i +Δx i The pixel value at .

[0155] In step S262, for each of the sub-region blocks, the normal vector angle of the sub-region block is determined based on the pixel value change from the starting sub-region block distributed along the first direction in the linear area where the sub-region block is located to the corresponding pixel points in the third sub-image.

[0156] Here, the method of determining the normal vector angle of the sub-region block based on the target pixel value changes from the starting sub-region block distributed along the first direction in the linear region where the sub-region block is located to the sub-region block in step S244 can be referred to. The normal vector angle of the sub-region block can be determined based on the pixel value changes of the pixels corresponding to the sub-region block in the third sub-image from the starting sub-region block distributed along the first direction in the linear region where the sub-region block is located. The pixel value changes of the pixels corresponding to the sub-region block in the third sub-image can correspond to the target pixel value changes corresponding to the sub-region block in step S244.

[0157] For example, the third sub-image I can be transformed into C Pixel x in i In the linear region where the corresponding sub-region block is located, the normal vector changes corresponding to the starting sub-region block distributed along the first direction to the sub-region block are accumulated to obtain the normal vector θ of the sub-region block:

[0158] Among them, θ A is the first angle between the light emitted by the first light source and the optical axis of the image acquisition device, ΔI C,R (x i ) is the pixel value change of the pixel point corresponding to the sub-region block in the third sub-image.

[0159] In the above embodiment, the pixel value change in the first direction for each pixel in the third sub-image is determined. For each sub-region block, the normal vector angle of the sub-region block is determined based on the pixel value change from the starting sub-region block distributed along the first direction to the corresponding pixel points in the third sub-image within the linear region where the sub-region block is located. In this way, because the concavo-convex morphological features of the target defect area surface in the third sub-image are enhanced, the normal vector angle of the sub-region block can be quickly and accurately determined based on the pixel value change from the starting sub-region block distributed along the first direction to the corresponding pixel points in the third sub-image within the linear region where the sub-region block is located, thereby improving the accuracy of height modeling and, in turn, the accuracy of defect detection.

[0160] In some embodiments, after determining the normal vector angle of each sub-region block, the normal vector angle of each sub-region block can be post-processed in any appropriate manner to make the normal vector angle of each sub-region block after processing smoother, so as to be close to the actual normal vector angle of each sub-region block.

[0161] In some embodiments, before the above step S222, the method further includes the following step S271:

[0162] Step S271: According to the first constraint condition, the normal vector angle of each sub-region block is filtered along the first direction to obtain the normal vector angle of each sub-region block after filtering; wherein, the first constraint condition includes that the normal vector angle of each sub-region block located at the boundary of the target defect area is 0.

[0163] The above step S222 may include the following step S272:

[0164] Step S272 : estimating the height of each sub-region block based on the filtered normal vector angle of each sub-region block to obtain an estimated height of each sub-region block.

[0165] Here, those skilled in the art may use any appropriate filter according to actual conditions, and filter the normal vector angles of each sub-region block along the first direction according to the first constraint condition, and the embodiments of the present application are not limited to this.

[0166] In some embodiments, a first partial differential equation filter can be used to filter the normal vector angle of each sub-region block along a first direction according to a first constraint condition to obtain a filtered normal vector angle for each sub-region block. The first partial differential equation filter can be pre-set based on actual conditions. The differential equation used in the first partial differential equation filter can be a Poisson equation or other partial differential equation, which is not limited in this embodiment of the present application.

[0167] During implementation, a one-dimensional partial differential equation filter and / or a two-dimensional partial differential equation filter may be used to filter the normal vector angle of each sub-region block according to the first constraint condition to obtain the normal vector angle of each sub-region block after filtering.

[0168] In some embodiments, a one-dimensional partial differential equation filter can be first used to filter the normal vector angle of each sub-region block according to a first constraint condition to obtain the normal vector angle of each sub-region block after the one-dimensional partial differential equation filtering; then a two-dimensional partial differential equation filter can be used to filter the normal vector angle of each sub-region block after the one-dimensional partial differential equation filtering according to the first constraint condition to obtain the normal vector angle of each sub-region block after the two-dimensional partial differential equation filtering.

[0169] FIG2D is a schematic diagram illustrating the filtering effect of the normal vector angles of each sub-region block within a linear region in a defect detection method provided by an embodiment of the present application. As shown in FIG2D , for each sub-region block within the same linear region, compared to the change curve 21 of the normal vector angles of each sub-region block before filtering by the one-dimensional partial differential equation, the change curve 22 of the normal vector angles of each sub-region block after filtering by the one-dimensional partial differential equation is smoother; and compared to the change curve 22 of the normal vector angles of each sub-region block after filtering by the one-dimensional partial differential equation, the change curve 23 of the normal vector angles of each sub-region block after filtering by the two-dimensional partial differential equation is further smoothed.

[0170] In the above embodiment, the normal vector angle of each sub-region block is filtered along a first direction according to a first constraint condition to obtain a filtered normal vector angle for each sub-region block. The first constraint condition includes that the normal vector angle of each sub-region block located at the boundary of the target defect area is zero. This allows the normal vector angle of each sub-region block to change more smoothly after filtering, more closely approximating its true normal vector angle. Consequently, the height estimate of each sub-region block based on the filtered normal vector angle of each sub-region block is more accurate, further improving the accuracy of defect detection.

[0171] In some embodiments, the above step S222 may include the following steps S281 to S282:

[0172] Step S281 : determining a height variation of each sub-region block in the first direction based on a normal vector angle of each sub-region block.

[0173] In some embodiments, for each sub-region block, the height change of the sub-region block in the first direction can be determined based on the product of the tangent value of the normal vector angle of the sub-region block and the resolution of the image acquisition device in the first direction. For example, the following formula 2-8 can be used to determine the height change of the pixel point x in the target image: i The height change Δh(x i ): Δh(x i )=tan(θ(x i ))*Δx (2-8);

[0174] Among them, θ(x i ) is the normal vector angle of the sub-region block, and Δx is the resolution of the image acquisition device in the first direction.

[0175] Step S282 : For each of the sub-region blocks, determine the height of the sub-region block based on the height changes corresponding to the starting sub-region blocks distributed along the first direction in the linear region where the sub-region block is located and to the sub-region block.

[0176] Here, the change in height of each sub-region on the coating surface can be expressed as a discrete line integral of the height of each sub-region along the first direction. In other words, the height of a sub-region can be calculated by accumulating the corresponding height changes from the starting sub-region distributed along the first direction to the sub-region within the linear region where the sub-region is located.

[0177] For example, the following formula 2-9 can be used to calculate the height change Δh(x) corresponding to the starting sub-region block and the sub-region block in the linear area where the sub-region block corresponding to the pixel point in the target image is located. i ) is accumulated to obtain the height h of the sub-region block:

[0178] In the above embodiment, the height change of each sub-region block in the first direction is determined based on the normal vector angle of each sub-region block. For each sub-region block, the height of the sub-region block is determined based on the corresponding height changes from the starting sub-region block to the sub-region block distributed along the first direction within the linear region where the sub-region block is located. In this way, the height of each sub-region block can be quickly and accurately estimated based on the normal vector angle of each sub-region block.

[0179] In some embodiments, before the above step S204, the method may further include the following step S291:

[0180] Step S291: Filter the height estimates of each sub-region block along the first direction according to the second constraint condition to obtain the height of each sub-region block after filtering; wherein the second constraint condition includes that the height of each sub-region block located at the boundary of the target defect area is 0.

[0181] The above step S204 may further include the following step S292:

[0182] Step S292 : performing defect detection on the target defect area based on the height of each sub-region block after filtering to obtain a detection result.

[0183] Here, those skilled in the art may use any appropriate filter according to actual conditions, and perform filtering processing on the height estimation of each sub-region block along the first direction according to the second constraint condition, and the embodiments of the present application are not limited to this.

[0184] In some embodiments, a second partial differential equation filter can be used to filter the height estimates of each sub-region block along the first direction according to the first constraint to obtain the filtered height of each sub-region block. The second partial differential equation filter can be pre-set based on actual conditions. The differential equation used in the second partial differential equation filter can be a Poisson equation or other partial differential equation, which is not limited in this embodiment of the present application.

[0185] During implementation, a one-dimensional partial differential equation filter and / or a two-dimensional partial differential equation filter may be used to filter the height estimation of each sub-region block according to the second constraint condition to obtain the height of each sub-region block after filtering.

[0186] In some embodiments, a one-dimensional partial differential equation filter can be first used to filter the height of each sub-region block according to the second constraint condition to obtain the height of each sub-region block after the one-dimensional partial differential equation filtering; then a two-dimensional partial differential equation filter can be used to filter the height of each sub-region block after the one-dimensional partial differential equation filtering according to the second constraint condition to obtain the height of each sub-region block after the two-dimensional partial differential equation filtering.

[0187] Figure 2E is a schematic diagram illustrating the filtering effect of the height of each sub-region within a linear region in a defect detection method provided by an embodiment of the present application. As shown in Figure 2E , for each sub-region within the same linear region, the height change curve 25 of each sub-region after filtering by the one-dimensional partial differential equation, and the height change curve 26 of each sub-region after filtering by the two-dimensional partial differential equation, are smoother than the height change curve 24 of each sub-region before filtering by the one-dimensional partial differential equation.

[0188] In the above embodiment, the height of each sub-region block is filtered along the first direction according to the second constraint, thereby obtaining the filtered height of each sub-region block. The second constraint includes setting the height of each sub-region block located at the boundary of the target defect area to zero. This allows the height of each sub-region block to change more smoothly after filtering, more closely approximating its true height, thereby further improving the accuracy of defect detection.

[0189] The present invention provides a defect detection method that can be executed by a processor of a computer device. FIG3A is a schematic diagram of an implementation flow of a defect detection method provided by an embodiment of the present invention. As shown in FIG3A , the method may include the following steps S301 to S305:

[0190] Step S301, obtaining a first image and a second image captured by an image acquisition device of the surface of the coating film covered on the target object to be inspected under the illumination of a first white light source and a second white light source, respectively; wherein the emitted light of the first white light source and the emitted light of the second white light source are symmetrical with respect to the optical axis of the image acquisition device.

[0191] Here, the above step S301 corresponds to the above step S201, and the implementation method of the above step S201 can be referred to during implementation.

[0192] Step S302: performing difference processing on the first image and the second image.

[0193] Here, during the differential processing, the first image and the second image can be subtracted pixel by pixel to obtain a first difference image, or after obtaining the first difference image, the first difference image can be appropriately enhanced, such as at least one of contrast enhancement and noise reduction.

[0194] Step S303 : performing defect detection on the surface of the coating film based on the image after the difference processing to obtain a target defect area on the surface of the coating film.

[0195] Here, based on the image after the difference processing, any appropriate defect detection algorithm can be used to perform defect detection on the surface of the covering film to obtain at least one target defect area on the surface of the covering film that may contain defects.

[0196] In some embodiments, a pre-trained fifth neural network model may be used to perform defect detection on the surface of the coating film based on the image after differential processing to obtain a target defect area on the surface of the coating film.

[0197] Step S304: determining height distribution information of the surface of the target defect area based on the first image and the second image.

[0198] Step S305 : performing defect detection on the target defect area based on the height distribution information to obtain a detection result.

[0199] Here, steps S304 to S305 correspond to the aforementioned steps S103 to S104 respectively, and the implementation of the aforementioned steps S103 to S104 may be referred to.

[0200] In the embodiment of the present application, a differential process is performed on the first and second images, and based on the differentially processed images, defect detection is performed on the surface of the coating film to obtain a target defect area on the surface of the coating film. Thus, because the first and second images are captured under illumination from two symmetrically arranged white light sources, differential processing of the first and second images can better highlight the concave and convex morphology of the coating film surface, enhancing the morphological features of the coating film surface in the differentially processed images, thereby facilitating defect detection on the coating film surface and making the obtained target defect area on the coating film surface more accurate.

[0201] In some embodiments, the above step S302 may include the following steps S311 to S313:

[0202] Step S311: Determine a first difference image between the first image and the second image.

[0203] Step S312: determining the morphological distribution information of the surface of the coating film based on the first difference image.

[0204] Here, the morphological distribution information of the coating film surface may include the concave-convex morphology of the plurality of sub-region blocks of the coating film surface corresponding to each pixel point in the first difference image. For example, the concave-convex morphology may include, but is not limited to, at least one of a concave morphology, a convex morphology, and a flat morphology.

[0205] In some embodiments, the concave-convex shape of the sub-region block corresponding to each pixel point can be determined based on the pixel value of each pixel point in the first difference image. For example, when the pixel value of a certain pixel point is not less than the first pixel threshold, the concave-convex shape of the sub-region block corresponding to the pixel point is determined to be a convex shape; when the pixel value of a certain pixel point is less than the first pixel threshold and not less than the second pixel threshold, the concave-convex shape of the sub-region block corresponding to the pixel point is determined to be a flat shape; when the pixel value of a certain pixel point is less than the second pixel threshold, the concave-convex shape of the sub-region block corresponding to the pixel point is determined to be a concave shape. For another example, when the change in the pixel value of a certain pixel point along the first direction is 0, the concave-convex shape of the sub-region block corresponding to the pixel point is determined to be a flat shape; when the change in the pixel value of a certain pixel point along the first direction is greater than 0, the sub-region block corresponding to the pixel point is determined to be a convex shape; when the change in the pixel value of a certain pixel point along the first direction is less than 0, the concave-convex shape of the sub-region block corresponding to the pixel point is determined to be a concave shape.

[0206] Step S313: performing enhancement processing on the first difference image based on the morphological distribution information.

[0207] Here, any appropriate method may be used to enhance the first difference image according to the concave-convex shape of the sub-region block corresponding to each pixel point, and the embodiment of the present application is not limited to this.

[0208] In some embodiments, in order to enhance the contrast between pixels in the first difference image, the pixel values ​​of pixels whose corresponding sub-region blocks are concave can be reduced, and the pixel values ​​of pixels whose corresponding sub-region blocks are convex can be increased.

[0209] In the above embodiment, a first difference image is determined between the first image and the second image; morphological distribution information of the coating film surface is determined based on the first difference image; and enhancement processing is performed on the first difference image based on the morphological distribution information. This further highlights the concave and convex morphology of the coating film surface in the difference-processed image, thereby enhancing the morphological features of the coating film surface in the difference-processed image.

[0210] In some embodiments, the above step S303 may include the following steps S321 to S322:

[0211] Step S321 : Based on the differentially processed image, defect detection is performed on the surface of the coating film to obtain a first defect area on the surface of the coating film and an initial defect type of the first defect area; the initial defect type includes a pit defect or a convex defect.

[0212] Here, any appropriate defect detection algorithm may be used to detect the pit defects and / or convex defects on the surface of the coating film to obtain a first defect area on the surface of the coating film containing the pit defects and / or convex defects.

[0213] Step S322 : When the initial defect type is a pit defect, the first defect area is determined as a target defect area.

[0214] In the above embodiment, based on the image after differential processing, defect detection is performed on the coating film surface to obtain a first defect area on the coating film surface and the initial defect type of the first defect area. If the initial defect type is a pit defect, the first defect area is determined as the target defect area. In this way, the defect area where the pit defect on the coating film surface is located can be quantitatively height modeled to obtain height distribution information on the surface of the defect area. Based on this height distribution information, the pit defect corresponding to the defect area can be re-detected, thereby improving the accuracy of pit defect detection on the coating film surface and reducing the number of over-detection cases.

[0215] In some embodiments, the above method may further include the following steps S331 to S332:

[0216] Step S331 : obtaining a third image captured by the image acquisition device of the surface of the target object located under the covering film under the illumination of a third light source; wherein the third light source is an infrared light source.

[0217] Here, the third light source is an infrared light source. In implementation, the third light source can be set at any suitable position above the target object covered with the coating film, and the number of the third light sources can also be determined according to actual conditions, which is not limited in the embodiment of the present application.

[0218] It can be understood that since infrared light can penetrate the covering film (such as blue film, yellow film, green film, etc.), the image acquisition device can capture a third image of the battery cell surface located under the covering film when the third light source illuminates the battery cell.

[0219] In some embodiments, as shown in FIG3B , the image acquisition device 11, the first light source 12, the second light source 13, and the third light source 15 can all be positioned above the object 14 to be inspected, and the first light source 12 and the second light source 13 can be symmetrically positioned on either side of the image acquisition device. Specifically, the light beam r1 emitted by the first light source 12 and the light beam r2 emitted by the second light source 13 are symmetrical relative to the optical axis R0 of the image acquisition device 11. The light beam r1 and the optical axis R0 have a first angle a1, the light beam r2 and the optical axis R0 have a second angle a2, and the light beam r3 and the optical axis R0 have a third angle a3. The first angle a1, the second angle a2, and the third angle a3 are all acute angles, and the first angle a1 and the second angle a2 are equal or approximately equal. The light beams r1, r2, and r3 are located in the same plane as the optical axis R0 and intersect the surface of the object 14. The values ​​of the first angle a1, the second angle a2, and the third angle a3 can be any appropriate values ​​determined in practice. For example, the first angle a1 and the second angle a2 may be between 30 degrees and 45 degrees, and the third angle a3 may be between 50 degrees and 60 degrees.

[0220] Taking a battery cell coated with blue film as an example, Figure 3C is a schematic diagram comparing the first, second, and third images obtained in a defect detection method provided by an embodiment of the present application. As shown in Figure 3C, the pixels corresponding to various regions of the battery cell in the first image p1, the second image p2, and the third image p3 are aligned. For example, the same region of the battery cell corresponds to aligned pixel regions 31, 32, and 33 in the first image p1, the second image p2, and the third image p3, respectively. Both the first image p1 and the second image p2 can effectively highlight slight morphological changes on the battery cell's blue film surface, minimizing reflections and overexposure. The third image p3 can also reveal the morphology of the battery cell's surface beneath the blue film.

[0221] In some embodiments, the first image, the second image, and the third image may be acquired through three consecutive scans.

[0222] In some embodiments, the image acquisition device is a line array camera, and the first, second, and third images may be acquired using a time-sharing stroboscopic method. This time-sharing stroboscopic method allows the first, second, and third images to be acquired in a single scan, reducing image acquisition costs, improving image acquisition efficiency, and achieving better imaging results, thereby improving defect detection efficiency.

[0223] Step S332: classify the first defect area based on the third image to obtain a target defect type of the first defect area.

[0224] Here, since the third image is formed after the infrared light penetrates the covering film, the third image can show features such as bubbles and foreign matter under the covering film.

[0225] In some embodiments, the target defect type of the first defect region may include, but is not limited to, at least one of a bubble defect, a foreign matter defect, and the like.

[0226] By identifying bubbles and / or foreign matter in the third image, the target defect type of the first defect area can be determined. During implementation, any suitable target detection algorithm can be used to identify bubbles and / or foreign matter in the third image, and this embodiment of the application is not limited to this.

[0227] For example, a pre-trained sixth neural network model may be used to identify bubbles and / or foreign matter in the third image to obtain the target defect type of the first defect area.

[0228] The above step S303 may further include the following step S333:

[0229] Step S333 : When the initial defect type is a convex defect, a target defect area is determined based on the target defect type.

[0230] Here, if the initial defect type of the first defect region is a raised defect, it can be determined based on the target defect type whether further height modeling of the first defect region is required, so as to conduct a defect re-identification of the first defect region based on the height distribution information on the surface of the first defect region. If further defect re-identification of the first defect region is required, the first defect region can be determined as the target defect region; if further defect re-identification of the first defect region is not required, the first defect region does not need to be determined as the target defect region.

[0231] During implementation, those skilled in the art may determine the target defect area based on the target defect type in an appropriate manner according to the actual application scenario, and the embodiments of the present application are not limited to this.

[0232] In the above embodiment, a third image is captured by the image acquisition device under illumination from a third light source, which is an infrared light source, of the surface of the target object located beneath the coating. Based on the third image, the first defect region is classified to obtain a target defect type for the first defect region. If the initial defect type is a raised defect, the target defect region is determined based on the target defect type. In this way, the third image, captured under infrared light, can be used to assist in the classification of the first defect region, thereby identifying the first defect region requiring defect reassessment based on height distribution information and obtaining the target defect region. This allows for more detailed defect detection, making it suitable for more complex defect detection scenarios.

[0233] In some embodiments, the target defect type includes a bubble defect or a foreign body defect. The step S333 described above of determining the target defect area based on the target defect type may include the following step S341:

[0234] Step S341 : When the target defect type is a foreign body defect, the first defect area is determined as a target defect area.

[0235] Here, the bubble defect refers to a protruding defect caused by bubbles between the coating film and the surface of the target object, while the foreign matter defect refers to a protruding defect caused by foreign matter between the coating film and the surface of the target object.

[0236] For foreign body defects, the first defect area can be determined as the target defect area. In this way, the first defect area can be further height-modeled to re-judge the defects of the first defect area based on the height distribution information on the surface of the first defect area, thereby improving the accuracy of foreign body defect detection and reducing the problem of over-inspection of foreign body defects.

[0237] The present invention provides a defect detection method, which can be executed by a processor of a computer device. FIG4A is a schematic diagram of an implementation flow of a defect detection method provided by an embodiment of the present invention. As shown in FIG4A , the method may include the following steps S401 to S406:

[0238] Step S401: Acquire a first image and a second image of the surface of the object to be detected captured by an image acquisition device under the illumination of a first light source and a second light source, respectively; wherein the emitted light of the first light source and the emitted light of the second light source are symmetrical with respect to the optical axis of the image acquisition device.

[0239] Step S402 : performing defect detection on the surface of the object based on the first image and the second image to obtain a target defect area on the surface of the object.

[0240] Step S403: determining height distribution information of the surface of the target defect area based on the first image and the second image.

[0241] Step S404: performing defect detection on the target defect area based on the height distribution information to obtain a detection result.

[0242] Here, steps S401 to S404 correspond to steps S101 to S104 in the aforementioned embodiment, respectively. When implementing, reference may be made to the implementation of the aforementioned steps S101 to S104.

[0243] Step S405 : performing three-dimensional height restoration on the surface of the target defect area based on the height distribution information to obtain a three-dimensional height map of the surface of the target defect area.

[0244] Step S406: display the three-dimensional height map.

[0245] Here, any appropriate three-dimensional drawing tool can be used to perform three-dimensional height restoration on the target defect area surface based on the height distribution information of the target defect area surface, and obtain and display a three-dimensional height map of the target defect area surface.

[0246] It should be noted that the embodiment of the present application does not limit the display method of the three-dimensional height map.

[0247] Figures 4B and 4C are respectively a stereoscopic schematic diagram and a top-view schematic diagram of a three-dimensional height map provided in an embodiment of the present application. As shown in Figures 4B and 4C, the three-dimensional height map can present the height information of each sub-region block on the surface of the target defect area relative to the reference plane.

[0248] In the embodiment of the present application, based on the height distribution information, the surface of the target defect area is restored in three dimensions to obtain a three-dimensional height map of the target defect area surface, and the three-dimensional height map is displayed. This allows for a more intuitive presentation of the height distribution information of the target defect area surface, further assisting in defect detection and improving detection accuracy.

[0249] The following describes the application of the defect detection method provided in the embodiments of the present application in actual scenarios.

[0250] With the widespread application of batteries in various fields, their safety issues are receiving increasing attention from all parties. As the core component of batteries, battery cells are responsible for the core functions of batteries, such as converting chemical energy into electrical energy, storing electrical energy, and charging and discharging. Therefore, battery cell safety testing is a crucial part of battery safety.

[0251] Taking the scenario of defect detection on the surface of blue-filmed battery cells as an example, the blue-filmed battery cells can be used as the object to be inspected. During the battery production process, square aluminum-shelled battery cells are prone to defects in the front-end process due to contact between logistics line equipment and production line cleanliness. Therefore, it is necessary to perform defect detection on the blue-filmed surface of the battery cells after blue film coating, such as detecting various defects with morphological variations such as pits and foreign matter, to ensure the safety and consistency of the power battery and reduce safety hazards. In addition to defect detection, it is also necessary to quantitatively analyze the defects on the blue-filmed surface to meet the needs of quantitative sorting or screening. During the defect detection process of coated battery cells, due to the severe background interference on the blue-filmed surface, simple optical solutions have severe reflections when lighting, resulting in uneven illumination, making it difficult to detect slight morphological variations on the blue-filmed surface. At the same time, due to the variable location and shape of the defects, it is difficult to quantitatively measure the height and depth of the defects with morphological variations such as bumps and depressions on the blue-filmed surface.

[0252] The defect detection method in related technology usually detects defects on the surface of an object by deriving the step field and curvature of the surface of the object. The required calculation amount is large and the detection accuracy is not high. Therefore, it cannot meet the needs of quantitative calculation of the height distribution information of blue film surface defects and more accurate defect detection under the rapid production rhythm of battery cells.

[0253] On this basis, an embodiment of the present application provides a defect detection system for performing defect detection on the surface of a battery cell coated with a blue film. Figure 5A is a schematic diagram of the composition architecture of a defect detection system provided by an embodiment of the present application. As shown in Figure 5A, the defect detection system 500 includes a defect detection device 510, a first white light source 520, a second white light source 530, an image acquisition device 540 and a battery cell carrier (not shown in the figure). Data can be exchanged between the defect detection device 510, the first white light source 520, the second white light source 530, the infrared light source 540 and the image acquisition device 550 through a network, and the emission light of the first white light source 520 and the emission light of the second white light source 530 are symmetrical with respect to the optical axis of the image acquisition device 540; wherein:

[0254] The battery cell carrying device is used to carry the blue film-wrapped battery cell 560 to be tested. The battery cell carrying device has at least one testing station, for example, a first testing station and a second testing station.

[0255] Image capture device 550 is positioned at at least one inspection station on the battery cell support device and is configured to capture a first image, a second image, and a third image of battery cell 560. The first, second, and third images are captured by image capture device 550 from the surface of battery cell 550 coated with a blue film under illumination from a first white light source 520, a second white light source 530, and an infrared light source 540, respectively.

[0256] It should be understood that the configuration of the cell carrier, the parameters of the first white light source 520, the second white light source 530, and the infrared light source 540, the type and parameters of the image acquisition device 550, and the relative positions of these devices can all be adjusted based on the image acquisition requirements of the target cell to meet the image acquisition requirements of the target cell of this application. In practice, the infrared light source 540 can correspond to the third light source in the aforementioned embodiment.

[0257] The defect detection device 510 is configured to perform defect detection on the blue film-wrapped battery cell 560 to be inspected based on the first image, the second image, and the third image captured by the image acquisition device 540 to obtain a detection result.

[0258] The defect detection device 510 may have one or more processing units, including specialized processing units such as GPUs, FPGAs, and ASICs, as well as general-purpose processing units such as CPUs. Furthermore, one or more virtual machines may also be running on each defect detection device 510. In some embodiments, the defect detection device 510, image acquisition device 540, and battery cell carrier may be integrated or separately configured. In implementation, the defect detection device 510 may correspond to the computer device described in the aforementioned embodiments.

[0259] In some embodiments, the battery cell to be inspected can be identified by the battery cell QR code on the battery cell top patch, and the image data such as the first image, the second image and the third image of the battery cell to be inspected collected at each inspection station can be associated, and the defect detection result of the battery cell can be associated with its corresponding battery cell QR code, so as to record, query and trace the inspection results of the battery cell, and facilitate timely discovery and resolution of potential quality problems to improve product quality. The battery cell QR code is a special QR code used to track the manufacturing and transportation process of the battery and provide information about the battery for authentication. The battery cell QR code usually contains information such as the battery serial number, batch number, manufacturing date, manufacturer, battery specifications, etc. Figure 5B is a structural schematic diagram of a battery cell top patch provided in an embodiment of the present application. As shown in Figure 5B, the battery cell top patch 570 includes an explosion-proof valve 571, a battery cell QR code 572, a positive electrode column 573 and a negative electrode column 574.

[0260] Based on the above-described defect detection system, embodiments of the present application provide a defect detection method that obtains height distribution information of the blue film surface from a three-minute stroboscopic blue film surface image, and then performs a second defect detection on the target defect area on the surface of the blue film-coated battery cell, thereby improving the accuracy and efficiency of defect detection. This defect detection method can be performed by the defect detection device 510 in the above-described defect detection system.

[0261] FIG5C is a schematic diagram of an implementation flow of a defect detection method provided in an embodiment of the present application. As shown in FIG5C , the method includes the following steps S501 to S509:

[0262] Step S501 : obtaining a first image and a second image of the surface of the battery cell wrapped with a blue film, which are captured by an image capture device under the illumination of a first white light source and a second white light source respectively.

[0263] The light emitted by the first white light source and the light emitted by the second white light source are symmetrical with respect to the optical axis of the image acquisition device.

[0264] Step S502 : performing differential processing on the first image and the second image, and performing defect detection on the blue film surface based on the differentially processed image to obtain a first defect area on the blue film surface and an initial defect type of the first defect area.

[0265] The initial defect type includes a pit defect or a convex defect.

[0266] Step S503 : When the initial defect type is a pit defect, the first defect region is determined as a target defect region.

[0267] Step S504 , obtaining a third image captured by the image capture device of the surface of the battery cell under the blue film under the illumination of a third light source; wherein the third light source is an infrared light source.

[0268] Step S505 : classifying the first defect area based on the third image to obtain a target defect type of the first defect area.

[0269] Among them, the target defect types include bubble defects or foreign matter defects.

[0270] Step S506: When the initial defect type is a convex defect and the target defect type is a foreign body defect, the first defect area is determined as the target defect area.

[0271] Step S507 : determining the height of at least one sub-region block on the surface of the target defect region relative to the reference plane based on the first image and the second image.

[0272] The reference plane is determined based on the boundary of the target defect area.

[0273] In some implementations, step S507 may include the following steps S511 to S517:

[0274] Step S511 : determining a first image region corresponding to a target defect region in the first image as a first sub-image.

[0275] Step S512 : determining the second image region corresponding to the target defect region in the second image as a second sub-image.

[0276] Step S513 : determining the pixel value variation of each pixel point in the first sub-image in the first direction and the pixel value variation of each pixel point in the second sub-image in the first direction.

[0277] Step S514: For each sub-region block, determine the target pixel value change corresponding to the sub-region block based on the difference between the pixel value change in the first direction of the pixel point corresponding to the sub-region block in the first sub-image and the pixel value change in the first direction of the pixel point corresponding to the sub-region block in the second sub-image.

[0278] Step S515 , for each sub-region block, determine the normal vector angle of the sub-region block based on the target pixel value changes corresponding to the starting sub-region block and the sub-region block distributed along the first direction in the linear area where the sub-region block is located.

[0279] Step S516 : Using a first partial differential equation filter, filter the normal vector angle of each sub-region block according to the first constraint condition to obtain a filtered normal vector angle of each sub-region block.

[0280] The first constraint condition includes: the normal vector angle of each sub-region block located at the boundary of the target defect region is 0.

[0281] Step S517, determining a height change of each sub-region block in the first direction based on the normal vector angle of each sub-region block after filtering;

[0282] Step S518 : For each sub-region block, determine the height of the sub-region block based on the height changes corresponding to the starting sub-region block and the sub-region blocks distributed along the first direction in the linear region where the sub-region block is located.

[0283] Step S519 : Using a second partial differential equation filter, filter the height estimation of each sub-region block according to the second constraint condition to obtain the height of each sub-region block after filtering.

[0284] The second constraint condition includes: the height of each sub-region block located at the boundary of the target defect region is 0.

[0285] Step S508 : When the absolute value of the height of at least one sub-region block is greater than a preset height threshold, determining that the detection result is a first result indicating that a defect exists in the target defect region.

[0286] Step S509 : when the absolute value of the height of each sub-region block is not greater than the preset height threshold, determining that the detection result is a second result indicating that there is no defect in the target defect region.

[0287] In some embodiments, the surface of the target defect area may be restored in three dimensions based on the heights of the sub-region blocks in the target defect area to obtain and display a three-dimensional height map of the surface of the target defect area.

[0288] The defect detection method provided by the embodiments of the present application has at least the following advantages:

[0289] 1) The symmetrical arrangement of the first and second white light sources can reduce background interference on the blue film surface of the battery cell to be inspected at appropriate illumination intensity, and can also achieve good imaging effects for defects with slight morphological changes on the blue film surface. The symmetrical arrangement of the light sources can also greatly highlight the morphological changes of defects on the blue film surface, while also facilitating the calculation of normal vector angles in the subsequent height modeling process.

[0290] 2) The infrared light source (third light source) can image the surface of the battery cell under the blue film due to its different transmittance from white light, helping to distinguish between bubble defects and foreign matter defects;

[0291] 3) By estimating the height distribution information of the target defect area surface, the height of the target defect area can be re-judged without introducing a 3D camera, thereby achieving a re-judgment of the defects in the target defect area;

[0292] 4) The third image of infrared imaging and the first and second images of white light imaging are used to determine the target defect area, and the target defect area is mapped to the height map to perform local three-dimensional reconstruction of the target defect area to estimate the height distribution information of the surface of the target defect area, thereby eliminating defects that do not meet the height / depth requirements and realizing quantitative screening or sorting of battery cells.

[0293] In addition, for special areas such as the edges and rounded corners (R corners) of the battery cells, ordinary lighting methods may overexpose the edges and R corners due to the influence of incident lighting, thereby covering up defects. The defect detection method provided in the embodiment of the application combines the first image and the second image corresponding to the symmetrically arranged white light source to enhance the image of the R corners and edges, while retaining defect information while suppressing reflections to the greatest extent, reducing the occurrence of local overbrightness or overdarkness, thereby enhancing the performance of edge and R corner position detection.

[0294] Figure 5D is a line graph comparing the estimated surface height of the target defect area in a defect detection method provided in an embodiment of the present application with the surface height of the target defect area captured by a 3D camera. As shown in Figure 5B , the estimated surface height h1 of the target defect area in a defect detection method provided in an embodiment of the present application and the comparison line h2 of the surface height of the target defect area captured by a 3D camera show a high correlation, with the absolute difference being less than 50 microns.

[0295] Table 1 shows the comparison data of the height of the target defect area surface estimated in the defect detection method provided in the embodiment of the present application in the verification test and the height of the target defect area surface captured by the 3D camera.

[0296] Table 1 Comparison data between the estimated height in the embodiment of the present application and the height measured by 3D camera

[0297] As shown in Table 1 above, the correlation coefficient between the height broken line of the target defect area surface estimated in the defect detection method provided in the embodiment of the present application and the height of the target defect area surface captured by the three-dimensional camera is greater than 95%, and the absolute difference is less than 50 microns (μm).

[0298] It can be seen that the estimated height distribution information of the target defect area surface in the defect detection method provided in the embodiment of the present application can, to a certain extent, represent the actual height distribution information of the target defect area surface. Therefore, based on the defect detection method provided in the embodiment of the present application, the height of the target defect area can be re-evaluated more accurately without the introduction of a 3D camera, thereby achieving re-evaluation of defects in the target defect area.

[0299] Based on the foregoing embodiments, an embodiment of the present application provides a defect detection device, which includes the various units included and the various modules included in each unit, and can be implemented by a processor in a computer device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0300] FIG6 is a schematic diagram of the structure of a defect detection device provided in an embodiment of the present application. As shown in FIG6 , the defect detection device 600 includes: a first acquisition module 610, a first detection module 620, a first determination module 630, and a second detection module 640, wherein:

[0301] A first acquisition module 610 is configured to acquire a first image and a second image of the surface of the object to be inspected, captured by an image acquisition device under illumination of a first light source and a second light source, respectively; wherein the light emitted by the first light source and the light emitted by the second light source are symmetrical with respect to an optical axis of the image acquisition device;

[0302] A first detection module 620 is configured to perform defect detection on the surface of the object based on the first image and the second image to obtain a target defect area on the surface of the object;

[0303] A first determining module 630 is configured to determine height distribution information of a surface of the target defect area based on the first image and the second image;

[0304] The second detection module 640 is configured to perform defect detection on the target defect area based on the height distribution information to obtain a detection result.

[0305] In some embodiments, the first light source includes a first white light source, the second light source includes a second white light source, and the object includes a target object covered with a coating film; the first acquisition module is also used to: acquire a first image and a second image of the surface of the coating film captured by the image acquisition device under the illumination of the first white light source and the second white light source respectively.

[0306] In some embodiments, the first determination module is also used to: determine a target sub-image corresponding to the target defect area based on the first image and the second image; perform height modeling on the target defect area based on the pixel value change between each pixel point in the target sub-image to obtain height distribution information of the surface of the target defect area.

[0307] In some embodiments, the positive projection of the emitted light of the first light source on the surface of the coating film extends along a first direction, the target defect area includes a plurality of linear areas distributed side by side along a second direction, each of the linear areas includes a plurality of sub-region blocks distributed in sequence along the first direction, the second direction is perpendicular to the first direction, and the height distribution information of the surface of the target defect area includes the height of each sub-region block relative to a reference plane, and the reference plane is determined based on the boundary of the target defect area; the first determination module is also used to: estimate the normal vector angle of each sub-region block based on the pixel value change between each pixel point in the target sub-image, and obtain the estimated normal vector angle of each sub-region block; wherein the normal vector angle of each sub-region block is the angle between the normal vector of the sub-region block and the optical axis; based on the normal vector angle of each sub-region block, estimate the height of each sub-region block, and obtain the estimated height of each sub-region block.

[0308] In some embodiments, the target sub-image includes a first sub-image and a second sub-image; the first determination module is also used to: determine the first image area corresponding to the target defect area in the first image as the first sub-image; and determine the second image area corresponding to the target defect area in the second image as the second sub-image.

[0309] In some embodiments, the first determination module is also used to: determine the pixel value change of each pixel point in the first sub-image in the first direction; determine the pixel value change of each pixel point in the second sub-image in the first direction; for each of the sub-region blocks, determine the target pixel value change corresponding to the sub-region block based on the difference between the pixel value change of the pixel point corresponding to the sub-region block in the first sub-image and the pixel value change of the pixel point corresponding to the sub-region block in the second sub-image in the first direction; for each of the sub-region blocks, determine the normal vector angle of the sub-region block based on the target pixel value change corresponding to the starting sub-region block distributed along the first direction in the linear area where the sub-region block is located.

[0310] In some embodiments, the target sub-image includes a third sub-image; the first determination module is further used for one of the following: determining a first image area corresponding to the target defect area in the first image, and determining a second image area corresponding to the target defect area in the second image; determining the difference between the first image area and the second image area as the third sub-image; determining a first difference image between the first image and the second image, and determining the third image area corresponding to the target defect area in the first difference image as the third sub-image.

[0311] In some embodiments, the first determination module is also used to: determine the pixel value change of each pixel point in the third sub-image in the first direction; for each of the sub-region blocks, determine the normal vector angle of the sub-region block based on the pixel value change of the corresponding pixel points in the third sub-image from the starting sub-region block distributed along the first direction in the linear area where the sub-region block is located to the sub-region block.

[0312] In some embodiments, the device also includes: a first filtering module, used to filter the normal vector angle of each sub-region block along the first direction according to the first constraint condition, to obtain the normal vector angle of each sub-region block after filtering; wherein, the first constraint condition includes that the normal vector angle of each sub-region block located at the boundary of the target defect area is 0; the first determination module is also used to: based on the normal vector angle of each sub-region block after filtering, estimate the height of each sub-region block to obtain the estimated height of each sub-region block.

[0313] In some embodiments, the first determination module is also used to: determine the height change of each sub-region block in the first direction based on the normal vector angle of each sub-region block; for each sub-region block, determine the height of the sub-region block based on the height change corresponding to the starting sub-region block distributed along the first direction in the linear area where the sub-region block is located to the sub-region block.

[0314] In some embodiments, the device also includes: a second filtering module, used to filter the height estimation of each sub-region block along the first direction according to a second constraint condition, to obtain the height of each sub-region block after filtering; wherein, the second constraint condition includes that the height of each sub-region block located at the boundary of the target defect area is 0; the second detection module is also used to: perform defect detection on the target defect area based on the height of each sub-region block after filtering to obtain a detection result.

[0315] In some embodiments, the first detection module is further used to: perform differential processing on the first image and the second image; perform defect detection on the surface of the covering film based on the differentially processed image to obtain a target defect area on the surface of the covering film.

[0316] In some embodiments, the first detection module is further used to: determine a first difference image between the first image and the second image; determine the morphological distribution information of the surface of the coating film based on the first difference image; and enhance the first difference image based on the morphological distribution information.

[0317] In some embodiments, the first detection module is also used to: perform defect detection on the surface of the coating film based on the image after differential processing, obtain a first defect area on the surface of the coating film and an initial defect type of the first defect area; the initial defect type includes a pit defect or a convex defect; when the initial defect type is a pit defect, the first defect area is determined as a target defect area.

[0318] In some embodiments, the device also includes a third detection module, which is used to: obtain a third image of the surface of the target object located under the covering film captured by the image acquisition device under the illumination of a third light source; wherein the third light source is an infrared light source; based on the third image, classify the first defect area to obtain a target defect type of the first defect area; the first detection module is also used to: when the initial defect type is a raised defect, determine the target defect area based on the target defect type.

[0319] In some embodiments, the target defect type includes a bubble defect or a foreign matter defect; and the first detection module is further configured to: when the target defect type is a foreign matter defect, determine the first defect region as the target defect region.

[0320] In some embodiments, the second detection module is also used to: determine the height of each sub-region block in the target defect area based on the height distribution information; and when the absolute value of the height of at least one of the sub-region blocks is greater than a preset height threshold, determine that the detection result is a first result characterizing the presence of a defect in the target defect area.

[0321] In some embodiments, the device further includes: a restoration module for performing three-dimensional height restoration on the surface of the target defect area based on the height distribution information to obtain a three-dimensional height map of the surface of the target defect area; and a display module for displaying the three-dimensional height map.

[0322] The description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to perform the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0323] It should be noted that, in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software and firmware.

[0324] An embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0325] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above method. The computer-readable storage medium may be transient or non-transient.

[0326] An embodiment of the present application provides a computer program, including computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0327] An embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, implements some or all of the steps in the above method. The computer program product can be implemented specifically by hardware, software, or a combination thereof. In some embodiments, the computer program product is embodied as a computer storage medium. In other embodiments, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0328] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between the various embodiments, and their similarities or similarities can be referenced to each other. The descriptions of the above device, storage medium, computer program, and computer program product embodiments are similar to the descriptions of the above method embodiments and have similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the description of the method embodiments of this application for understanding.

[0329] It should be noted that FIG7 is a schematic diagram of a hardware entity of a computer device in an embodiment of the present application. As shown in FIG7 , the hardware entity of the computer device 700 includes: a processor 701, a communication interface 702, and a memory 703, wherein:

[0330] Processor 701 generally controls the overall operation of computer device 700 .

[0331] The communication interface 702 enables the computer device to communicate with other terminals or servers through a network.

[0332] The memory 703 is configured to store instructions and applications executable by the processor 701, and can also cache data to be processed or processed by the processor 701 and various modules in the computer device 700 (for example, image data, audio data, voice communication data, and video communication data). This can be implemented using flash memory (FLASH) or random access memory (RAM). Data can be transmitted between the processor 701, the communication interface 702, and the memory 703 via a bus 704.

[0333] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned steps / processes does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0334] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0335] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0336] The units described above as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, the functional units in the various embodiments of the present application may all be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0337] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0338] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0339] The above is only an implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any changes or replacements that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed in this application should be covered by the scope of protection of the present application.

Claims

1. A defect detection method, characterized in that: include: Acquire a first image and a second image of the surface of the object to be detected, respectively, captured by an image acquisition device under the illumination of a first light source and a second light source; wherein the light emitted by the first light source and the light emitted by the second light source are symmetrical with respect to the optical axis of the image acquisition device; Based on the first image and the second image, performing defect detection on the surface of the object to obtain a target defect area on the surface of the object; Determining height distribution information of the surface of the target defect area based on the first image and the second image; Based on the height distribution information, defect detection is performed on the target defect area to obtain a detection result.

2. The method according to claim 1, characterized in that The first light source includes a first white light source, the second light source includes a second white light source, and the object includes a target object covered with a coating film; The acquisition device acquires a first image and a second image of the surface of the object to be detected under the illumination of a first light source and a second light source, respectively, and includes: The first image and the second image of the surface of the covering film are respectively captured by an image acquisition device under the illumination of the first white light source and the second white light source.

3. The method according to claim 2, characterized in that The step of determining the height distribution information of the surface of the target defect area based on the first image and the second image includes: Determining a target sub-image corresponding to the target defect area based on the first image and the second image; Based on the pixel value variation between each pixel point in the target sub-image, the target defect area is height modeled to obtain height distribution information of the surface of the target defect area.

4. The method according to claim 3, characterized in that The orthographic projection of the light emitted by the first light source on the surface of the coating film extends along a first direction, the target defect area includes a plurality of linear areas distributed side by side along a second direction, each of the linear areas includes a plurality of sub-region blocks distributed in sequence along the first direction, the second direction is perpendicular to the first direction, and the height distribution information of the surface of the target defect area includes the height of each sub-region block relative to a reference plane, and the reference plane is determined based on the boundary of the target defect area; The step of performing height modeling on the target defect area based on the pixel value variation between each pixel point in the target sub-image to obtain height distribution information on the surface of the target defect area includes: Based on the pixel value variation between each pixel point in the target sub-image, the normal vector angle of each sub-region block is estimated to obtain the estimated normal vector angle of each sub-region block; wherein the normal vector angle of each sub-region block is the angle between the normal vector of the sub-region block and the optical axis; Based on the normal vector angle of each of the sub-region blocks, the height of each of the sub-region blocks is estimated to obtain the estimated height of each of the sub-region blocks.

5. The method according to claim 4, characterized in that The target sub-image includes a first sub-image and a second sub-image; The determining, based on the first image and the second image, a target sub-image corresponding to the target defect area comprises: Determine a first image region corresponding to the target defect region in the first image as the first sub-image; A second image region corresponding to the target defect region in the second image is determined as the second sub-image.

6. The method according to claim 5, characterized in that The estimating the normal vector angle of each of the sub-region blocks based on the pixel value variation between each pixel point in the target sub-image to obtain the estimated normal vector angle of each of the sub-region blocks includes: Determine a pixel value variation of each pixel point in the first sub-image in the first direction; Determine a pixel value variation of each pixel point in the second sub-image in the first direction; For each of the sub-region blocks, determining a target pixel value change amount corresponding to the sub-region block based on a difference between a pixel value change amount of a pixel point corresponding to the sub-region block in the first sub-image in the first direction and a pixel value change amount of a pixel point corresponding to the sub-region block in the second sub-image in the first direction; For each of the sub-region blocks, a normal vector angle of the sub-region block is determined based on target pixel value changes corresponding to a starting sub-region block distributed along the first direction in a linear region where the sub-region block is located and to the sub-region block.

7. The method according to claim 4, characterized in that The target sub-image includes a third sub-image; The determining, based on the first image and the second image, a target sub-image corresponding to the target defect area comprises one of the following: Determine a first image region corresponding to the target defect region in the first image, and determine a second image region corresponding to the target defect region in the second image; determining a difference between the first image area and the second image area as the third sub-image; A first difference image between the first image and the second image is determined, and a third image region corresponding to the target defect region in the first difference image is determined as the third sub-image.

8. The method according to claim 7, characterized in that The estimating the normal vector angle of each of the sub-region blocks based on the pixel value variation between each pixel point in the target sub-image to obtain the estimated normal vector angle of each of the sub-region blocks includes: Determine a pixel value variation of each pixel point in the third sub-image in the first direction; For each of the sub-region blocks, the normal vector angle of the sub-region block is determined based on the pixel value change from the starting sub-region block distributed along the first direction in the linear area where the sub-region block is located to the corresponding pixel points in the third sub-image of the sub-region block.

9. The method according to any one of claims 4 to 8, characterized in that Before estimating the height of each of the sub-region blocks based on the normal vector angle of each of the sub-region blocks to obtain the estimated height of each of the sub-region blocks, the method further includes: According to the first constraint condition, filtering the normal vector angle of each sub-region block along the first direction to obtain the normal vector angle of each sub-region block after filtering; wherein the first constraint condition includes that the normal vector angle of each sub-region block located at the boundary of the target defect area is 0; The estimating the height of each of the sub-region blocks based on the normal vector angle of each of the sub-region blocks to obtain the estimated height of each of the sub-region blocks includes: Based on the normal vector angle of each of the sub-region blocks after filtering, the height of each of the sub-region blocks is estimated to obtain the estimated height of each of the sub-region blocks.

10. The method according to any one of claims 4 to 9, characterized in that The estimating the height of each of the sub-region blocks based on the normal vector angle of each of the sub-region blocks to obtain the estimated height of each of the sub-region blocks includes: Determine the height of each sub-region block in the first direction based on the normal vector angle of each sub-region block. Degree change; For each of the sub-region blocks, the height of the sub-region block is determined based on height changes corresponding to a starting sub-region block distributed along the first direction in a linear region where the sub-region block is located and to the sub-region block.

11. The method according to any one of claims 4 to 10, characterized in that Before performing defect detection on the target defect area based on the height distribution information to obtain a detection result, the method further includes: According to the second constraint condition, filtering is performed on the height estimation of each sub-region block along the first direction to obtain the height of each sub-region block after filtering; wherein the second constraint condition includes that the height of each sub-region block located at the boundary of the target defect area is 0; The step of performing defect detection on the target defect area based on the height distribution information to obtain a detection result includes: Based on the height of each of the sub-region blocks after filtering, defect detection is performed on the target defect region to obtain a detection result.

12. The method according to any one of claims 2 to 11, characterized in that The step of performing defect detection on the surface of the object based on the first image and the second image to obtain a target defect area on the surface of the object includes: performing difference processing on the first image and the second image; Based on the image after the difference processing, defect detection is performed on the surface of the coating film to obtain a target defect area on the surface of the coating film.

13. The method according to claim 12, characterized in that The performing difference processing on the first image and the second image includes: determining a first difference image between the first image and the second image; Determining the morphological distribution information of the surface of the coating film based on the first difference image; Based on the morphological distribution information, the first difference image is enhanced.

14. The method according to claim 12 or 13, characterized in that The step of performing defect detection on the surface of the coating film based on the image after the difference processing to obtain a target defect area on the surface of the coating film includes: Based on the image after the difference processing, defect detection is performed on the surface of the coating film to obtain a first defect area on the surface of the coating film and an initial defect type of the first defect area; the initial defect type includes a pit defect or a convex defect; In a case where the initial defect type is a pit defect, the first defect region is determined as a target defect region.

15. The method according to claim 14, characterized in that The method further comprises: Acquire a third image captured by the image acquisition device of the surface of the target object located under the covering film under the illumination of a third light source; wherein the third light source is an infrared light source; Based on the third image, classify the first defect area to obtain a target defect type of the first defect area; The method of performing defect detection on the surface of the coating film based on the image after the difference processing to obtain a target defect area on the surface of the coating film further includes: In the case where the initial defect type is a convex defect, a target defect area is determined based on the target defect type.

16. The method according to claim 15, characterized in that The target defect type includes a bubble defect or a foreign body defect; The step of determining a target defect area based on the target defect type includes: When the target defect type is a foreign matter defect, the first defect area is determined as a target defect area.

17. The method according to any one of claims 1 to 11, characterized in that The step of performing defect detection on the target defect area based on the height distribution information to obtain a detection result includes: Based on the height distribution information, determining the height of each sub-region block in the target defect area; When the absolute value of the height of at least one of the sub-region blocks is greater than a preset height threshold, the detection result is determined to be a first result indicating that a defect exists in the target defect region.

18. The method according to any one of claims 1 to 11, characterized in that The method further comprises: Based on the height distribution information, performing three-dimensional height restoration on the surface of the target defect area to obtain a three-dimensional height map of the surface of the target defect area; The three-dimensional height map is displayed.

19. A defect detection device, characterized in that: The device comprises: A first acquisition module, used to acquire a first image and a second image of the surface of the object to be detected, which are acquired by an image acquisition device under the illumination of a first light source and a second light source, respectively; wherein the emitted light of the first light source and the emitted light of the second light source are symmetrical with respect to the optical axis of the image acquisition device; A first detection module, configured to perform defect detection on the surface of the object based on the first image and the second image to obtain a target defect area on the surface of the object; A first determination module, configured to determine height distribution information of a surface of the target defect area based on the first image and the second image; The second detection module is used to perform defect detection on the target defect area based on the height distribution information to obtain a detection result.

20. A computer device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, wherein: When the processor executes the program, the steps in the method according to any one of claims 1 to 18 are implemented.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 18 are implemented.

22. A computer program product, comprising a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is read and executed by a computer, the steps of the method according to any one of claims 1 to 18 are implemented.

Citation Information

Patent Citations

  • A surface defect detection method that fuses grayscale and depth information

    CN102288613A

  • Hole plugging defect detection method and hole plugging defect detection equipment based on image detection

    CN112444490A

  • Object detection method, storage medium, processor and system

    CN114359158A

  • Defect detection method and system based on photometric stereo

    CN114998308A

  • Surface defect detection system and surface detection production line

    CN115436383A

Cited By

  • Film surface defect detection method and system

    CN119224001A

  • Battery inclination detection method and device and computer readable storage medium

    CN119850630A

  • Food package visual inspection method and system based on machine vision

    CN120525858A

  • Light guide plate conveying line foreign matter visual detection method and system and storage medium

    CN120563505A

  • Silica gel product surface defect detection method based on AI vision

    CN122243985A