Defect detection methods, devices, computer equipment, and storage media

Abstract

This application relates to a defect detection method, apparatus, computer device, and storage medium. The method includes: acquiring a color image of a target object captured under a single light source; acquiring a multi-light source image corresponding to the target object, the multi-light source image comprising multiple light source images captured under calibrated light sources in different directions; generating a material image and a surface geometry image based on the multi-light source images; the material image representing the material information of the target object, and the surface geometry image representing the depth information of the target object's surface; and performing defect detection on the target object based on the color image, the material image, and the surface geometry image to obtain a defect detection result for the target object. This method can improve the accuracy and comprehensiveness of defect detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a defect detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of computer technology, surface defect detection technology based on machine vision has emerged. This technology enables the detection of defects on the surfaces of various manufactured products, identifying surface flaws and defects, and effectively improving product quality.

[0003] Traditional defect detection techniques typically involve capturing RGB images of manufactured products and then analyzing and processing these images to detect defects. However, products exhibit a wide variety of defect types, and relying solely on RGB images for defect detection is insufficient to address diverse defect categories, easily leading to missed detections. Summary of the Invention

[0004] Therefore, it is necessary to provide a defect detection method, apparatus, computer equipment, and storage medium that can improve the accuracy and comprehensiveness of the above-mentioned technical problems.

[0005] A defect detection method, the method comprising:

[0006] To acquire a color image of a target object taken under a single light source;

[0007] Acquire multi-light source images corresponding to the target object, wherein the multi-light source images include multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions;

[0008] A material image and a surface geometry image are generated based on the multi-light source image; the material image represents the material information of the target object, and the surface geometry image represents the depth information of the surface of the target object;

[0009] Based on the color image, the material image, and the surface geometry image, defect detection is performed on the target object to obtain the defect detection result of the target object.

[0010] A defect detection device, the device comprising:

[0011] The first acquisition module is used to acquire a color image of the target object obtained by taking a picture under a single light source;

[0012] The second acquisition module is used to acquire a multi-light source image corresponding to the target object. The multi-light source image includes multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions.

[0013] A generation module is used to generate a material image and a surface geometry image based on the multi-light source image; the material image represents the material information of the target object, and the surface geometry image represents the depth information of the surface of the target object;

[0014] The detection module is used to perform defect detection on the target object based on the color image, the material image, and the surface geometry image, and obtain the defect detection result of the target object.

[0015] In one embodiment, the device further includes a calibration module; the calibration module is configured to acquire calibration images containing bright areas obtained by photographing the sphere under illumination from different calibration light sources; determine the bright position of the bright area and the center position of the sphere in each calibration image, and determine the normal vector of the sphere surface in each calibration image based on a preset reflection direction, the center position of the sphere, and the bright position; and determine the direction corresponding to each calibration light source based on each normal vector and the preset reflection direction.

[0016] In one embodiment, the calibration module is further configured to determine the position of key points in the highlighted region of each calibration image; and to determine the highlight position corresponding to each highlighted region based on the position of the key points in each highlighted region.

[0017] In one embodiment, the generation module is further configured to perform pixel matching processing on the multiple light source images to obtain matching target pixels in the multiple light source images; determine the reflection information corresponding to the target pixel based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image; generate a material image based on the reflection information corresponding to each target pixel; and generate a surface geometry image based on the direction vector of each calibrated light source, the target pixel, and the reflection information.

[0018] In one embodiment, the material image includes a red channel material image, a green channel material image, and a blue channel material image. The generation module is further configured to determine the red channel value, green channel value, and blue channel value of the target pixel in the corresponding light source image; determine the reflection information of the target pixel corresponding to the red channel, green channel, and blue channel based on the direction vector of each calibrated light source and the red channel value, green channel value, and blue channel value corresponding to the target pixel; and generate a red channel material image, a green channel material image, and a blue channel material image based on the reflection information of the red channel, green channel, and blue channel corresponding to each target pixel.

[0019] In one embodiment, the generation module is further configured to determine the normal vector corresponding to the target pixel based on the reflection information, the direction vector of each of the calibrated light sources, and the pixel value of the target pixel in the corresponding light source image; and generate a surface geometric image based on the normal vector corresponding to each target pixel.

[0020] In one embodiment, the detection module is further configured to extract features from the color image, the material image, and the surface geometry image respectively to obtain color features, material features, and geometric features; fuse the color features, the material features, and the geometric features to obtain fused features; and perform defect detection on the target object based on the fused features to obtain the defect detection result of the target object.

[0021] In one embodiment, the detection module is further configured to: extract local features to be detected from the fused features; perform defect detection on the extracted local features to obtain local detection results corresponding to the local features; return to the step of extracting local features to be detected from the fused features and continue execution until there are no local features in the fused features that have not been defect detected, and stop to obtain local detection results corresponding to each local feature; and generate defect detection results for the target object based on each local detection result; the defect detection results include at least one of defect category, defect location, and defect area in the target object.

[0022] In one embodiment, the detection module is further configured to perform defect category detection, defect location detection, and defect area detection based on the extracted local features, respectively, to obtain local detection results corresponding to the local features; the local detection results include the defect category, defect mask, and defect bounding box corresponding to the local features.

[0023] In one embodiment, the detection module is further configured to generate an object mask containing the target object based on the defect mask corresponding to each of the local features; and to obtain the defect category corresponding to the defect existing in the target object, the defect location corresponding to each defect, and the defect area of ​​each defect according to the object mask, the defect category corresponding to each of the local features, and the defect bounding box.

[0024] In one embodiment, the target object is a metal injection molded device; the multi-light source image includes at least three light source images obtained by taking pictures of the metal injection molded device under at least three calibrated light sources in at least three different directions; the defect category in the defect detection result includes at least one of cracks, dents, material sticking, material shortage, bright marks, and dirt.

[0025] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0026] To acquire a color image of a target object taken under a single light source;

[0027] Acquire multi-light source images corresponding to the target object, wherein the multi-light source images include multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions;

[0028] A material image and a surface geometry image are generated based on the multi-light source image; the material image represents the material information of the target object, and the surface geometry image represents the depth information of the surface of the target object;

[0029] Based on the color image, the material image, and the surface geometry image, defect detection is performed on the target object to obtain the defect detection result of the target object.

[0030] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0031] To acquire a color image of a target object taken under a single light source;

[0032] Acquire multi-light source images corresponding to the target object, wherein the multi-light source images include multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions;

[0033] A material image and a surface geometry image are generated based on the multi-light source image; the material image represents the material information of the target object, and the surface geometry image represents the depth information of the surface of the target object;

[0034] Based on the color image, the material image, and the surface geometry image, defect detection is performed on the target object to obtain the defect detection result of the target object.

[0035] A computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0036] To acquire a color image of a target object taken under a single light source;

[0037] Acquire multi-light source images corresponding to the target object, wherein the multi-light source images include multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions;

[0038] A material image and a surface geometry image are generated based on the multi-light source image; the material image represents the material information of the target object, and the surface geometry image represents the depth information of the surface of the target object;

[0039] Based on the color image, the material image, and the surface geometry image, defect detection is performed on the target object to obtain the defect detection result of the target object.

[0040] The aforementioned defect detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire a color image of a target object under a single light source, and then acquire a multi-light source image corresponding to the target object. This multi-light source image includes multiple light source images of the target object captured under calibrated light sources from different directions. Based on the multi-light source image, a material image characterizing the material information of the target object and a surface geometric image characterizing the depth information of the target object's surface can be accurately generated. The color image and material image can better depict defects lacking depth information, while the surface geometric image can better depict the depth information of defects. Defect detection of the target object based on the color image, material image, and surface geometric image achieves complementarity in defect detection with and without depth information, thereby more accurately and comprehensively detecting defects present in the target object, reducing the false negative rate, and effectively improving the accuracy and comprehensiveness of defect detection. Attached Figure Description

[0041] Figure 1 This is a diagram illustrating the application environment of a defect detection method in one embodiment;

[0042] Figure 2 This is a flowchart illustrating a defect detection method in one embodiment;

[0043] Figure 3 This is a schematic diagram illustrating the generation of material images and surface geometry images based on multi-light source images in one embodiment;

[0044] Figure 4 This is a schematic diagram of the surface of a sphere under different calibration light sources in one embodiment;

[0045] Figure 5 This is a schematic diagram illustrating the relationship between the incident direction, normal vector, and reflection direction of a light source in one embodiment;

[0046] Figure 6 This is a schematic diagram of the process for generating material images and surface geometry images based on multi-light source images in one embodiment;

[0047] Figure 7This is a schematic diagram illustrating how the reflection information corresponding to a target pixel is determined based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image in one embodiment.

[0048] Figure 8 This is a schematic diagram comparing a color image and a surface geometry image in one embodiment;

[0049] Figure 9 This is a schematic diagram illustrating the defect detection results of a target object based on fusion features in one embodiment.

[0050] Figure 10 This is an architecture diagram of a defect detection model in one embodiment;

[0051] Figure 11 This is a structural block diagram of a defect detection device in one embodiment;

[0052] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] This application relates to the field of Artificial Intelligence (AI) technology. AI is the theory, methods, techniques, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine capable of reacting in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. The solutions provided in this application relate to methods for detecting defects in artificial intelligence, which are specifically illustrated through the following embodiments.

[0055] The defect detection method provided in this application can be applied to, for example... Figure 1 The defect detection system shown. (As shown) Figure 1As shown, the defect detection system includes a terminal 110 and a server 120. In one embodiment, both the terminal 110 and the server 120 can individually execute the defect detection method provided in this embodiment. The terminal 110 and server 120 can also work together to execute the defect detection method provided in this embodiment. When the terminal 110 and server 120 work together to execute the defect detection method provided in this embodiment, the terminal 110 can capture a color image of the target object under a single light source, and capture multiple light source images of the target object under calibrated light sources in different directions. The terminal 110 sends the color image and the multi-light source images to the server 120, which generates a material image and a surface geometry image based on the multi-light source images. The material image represents the material information of the target object, and the surface geometry image represents the depth information of the target object's surface. The server 120 performs defect detection on the target object based on the color image, material image, and surface geometry image, obtaining the defect detection result of the target object. The server 120 returns the defect detection result of the target object to the terminal 110.

[0056] The server 120 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. The terminal 110 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, vehicle terminal, smart TV, etc., but is not limited to these. The terminal 110 and the server 120 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0057] In one embodiment, multiple servers can form a blockchain, with the servers acting as nodes on the blockchain.

[0058] In one embodiment, data related to the defect detection method may be stored on a blockchain, for example, in one embodiment, such as Figure 2 As shown, a defect detection method is provided, which can be applied to... Figure 1 The computer equipment in the middle (the computer equipment can specifically be) Figure 1 Taking a terminal or server as an example, the following steps are included:

[0059] Step S202: Obtain a color image of the target object taken under a single light source.

[0060] The target object refers to the object that needs to be inspected for defects, which can be metal parts, glass, ceramics, plastics, etc., but is not limited to these. A color image, or RGB image, refers to an image in which each pixel is composed of R (Red), G (Green), and B (Blue) components, where R, G, and B are described by different gray levels.

[0061] Specifically, computer equipment acquires a color image of a target object obtained by photographing it under a single light source.

[0062] In one embodiment, the computer device can be a terminal that can capture a color image of the target object requiring defect detection using a camera under a single light source. Alternatively, the terminal can capture a color image of the target object using other imaging devices under a single light source.

[0063] In one embodiment, the computer device can be a server, which can then capture a color image of the target object under a single light source using a terminal or other imaging device, and then upload the color image to the server.

[0064] Step S204: Obtain a multi-light source image corresponding to the target object. The multi-light source image includes multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions.

[0065] Specifically, the computer equipment can acquire multiple light source images of the target object under calibrated light sources in different directions, and use these multiple light source images as a multi-light source image.

[0066] In one embodiment, the computer device can be a terminal, which can capture images of the target object under calibrated light sources in different directions using a camera, obtaining light source images corresponding to each direction. Alternatively, the terminal can capture images of the target object under calibrated light sources in different directions using other imaging devices, obtaining light source images corresponding to each direction.

[0067] In one embodiment, the computer device can be a server, which can take pictures of the target object under calibrated light sources in different directions through a terminal or other shooting device, obtain the light source image corresponding to each direction, and then upload the multiple light source images as a multi-light source image to the server.

[0068] Step S206: Generate a material image and a surface geometry image based on the multi-source image; the material image represents the material information of the target object, and the surface geometry image represents the depth information of the surface of the target object.

[0069] Among them, the material image, also known as the reflectivity image, is used to characterize the material information of the target object's surface, that is, to characterize the reflection information of the target object's surface to a calibrated light source. The surface geometry image refers to the normal vector map of the target object's surface, which can characterize the depth information of the target object's surface.

[0070] Specifically, the computer device calculates the reflection information of the target object's surface to a calibrated light source based on multi-light source images, and generates a material image based on the reflection information. The computer device then uses the material image to generate a surface geometry image that characterizes the depth information of the target object's surface.

[0071] In one embodiment, a computer device can determine the normal vector of the target object's surface from a material image and generate a surface geometry image based on the normal vector.

[0072] like Figure 3 The diagram illustrates a method for generating material and surface geometry images based on multi-source images in one embodiment. A sector light source emits calibration light sources in different directions, which illuminate the surface of the target object. A camera then performs multi-source imaging of the target object's surface, obtaining light source images corresponding to each direction. Photometric data is synthesized based on these multi-source images to obtain a material image characterizing the diffuse reflectance of the target object and a surface geometry image characterizing the depth information of the target object's surface.

[0073] Step S208: Based on the color image, material image, and surface geometry image, perform defect detection on the target object to obtain the defect detection result of the target object.

[0074] The defect detection results include at least one of the following: defect category, defect location, and defect area, corresponding to the defects present in the target object.

[0075] Specifically, the computer device can extract features from color images, material images, and surface geometry images, and perform defect detection on the target object based on the extracted features to determine the defects present in the target object, as well as at least one of the defect category, defect location, and defect area corresponding to each defect.

[0076] In one embodiment, when the target object is an industrial device, the defect category includes at least one of the following: cracks, dents, material adhesion, material shortage, bright marks, and dirt. A crack refers to a fissure formed in the material under at least one stress or environmental influence. A dent refers to a depression on the surface of an industrial device caused by external force during processing. Material adhesion refers to excess material adhering to the surface of the industrial device, resulting in a raised surface. Material shortage refers to incomplete or missing parts in the shape of the industrial device due to incomplete filling of the cavity during processing. Bright marks refer to scratches on the surface of the industrial device caused by friction. Dirt refers to other stains on the surface of the industrial device, such as contamination with other materials or colors.

[0077] In the aforementioned defect detection method, a color image of the target object is acquired under a single light source, and a multi-light source image corresponding to the target object is acquired. This multi-light source image includes multiple images of the target object captured under calibrated light sources from different directions. Based on the multi-light source image, a material image representing the material information of the target object and a surface geometry image representing the depth information of the target object's surface can be accurately generated. The color image and material image can better characterize defects lacking depth information, while the surface geometry image can better characterize the depth information of the defects. Defect detection of the target object based on the color image, material image, and surface geometry image achieves complementarity in defect detection with and without depth information, thereby more accurately and comprehensively detecting defects present in the target object, reducing the false negative rate, and effectively improving the accuracy and comprehensiveness of defect detection.

[0078] In one embodiment, calibration light sources in different directions are obtained through a calibration step; this calibration step includes:

[0079] Acquire calibration images containing highlighted areas of a sphere obtained by photographing the sphere under different calibration light sources; determine the highlight position of the highlighted area and the center position of the sphere in each calibration image, and determine the normal vector of the sphere surface in each calibration image based on the preset reflection direction, the center position, and the highlight position; determine the direction corresponding to each calibration light source based on each normal vector and the preset reflection direction.

[0080] Specifically, the computer equipment needs to calibrate the direction of the calibration light sources to determine the corresponding direction of each calibration light source. The computer equipment can place a smooth sphere under different calibration light sources to create an image, causing a bright area to appear on the surface of the sphere. The direction of the calibration light source can be reflected by the bright area.

[0081] The sphere was illuminated by different calibration light sources, and photographed under each type of illumination, resulting in calibration images containing highlighted areas. For example... Figure 4 As shown, the highlighted areas on the surface of the sphere under four different calibration light sources are presented. Figure 4 The rightmost image represents the position mask of the sphere in the corresponding calibration image.

[0082] For each calibration image, the computer device can determine the center position of the sphere in the calibration image and the highlight position of the highlighted area in the calibration image. The computer device can obtain a preset reflection direction, which can be represented by a preset reflection vector, which is the distance vector from the observer to the target object. For example, the preset reflection vector can be [0, 0, 1]. Based on the preset reflection direction, the center position of the sphere, and the highlight position, the computer device can calculate the normal vector of the sphere's surface in the calibration image. Based on the normal vector and the preset reflection vector, the incident direction vector of the calibration light source corresponding to the calibration image is calculated. The incident direction vector of the calibration light source is the direction of the calibration light source.

[0083] In one embodiment, the center position of the sphere can be the coordinates of the center of the sphere in the corresponding calibration image, and the highlighted position can be the coordinates in the corresponding calibration image.

[0084] By processing each calibration image in the same way, the incident direction vector of the calibration light source corresponding to each calibration image can be calculated, and the direction of each calibration light source can be obtained.

[0085] In one embodiment, such as Figure 5 As shown, the direction of the calibrated light source is L, i.e., the incident direction L; the normal vector of the sphere surface is M; and the preset reflection direction is R. According to the Phong Reflection lighting model, the relationship between the direction L of the calibrated light source, the normal vector M, and the preset reflection direction R is as follows:

[0086] R = 2(M·L)ML (1)

[0087] Where R takes the value [0, 0, 1], it represents the distance vector from the observer to the target object. (P) x ,P y ) are the coordinates of the highlighted area, (C x C y If ) represents the coordinates of the center of the sphere on the calibrated image, then we have the expression for M:

[0088]

[0089] The coordinates of the surface normal vector M of the sphere can be calculated using formula (2), and the direction vector corresponding to the direction L of the calibration light source can be calculated using formula (1).

[0090] In this embodiment, calibration images containing bright areas are obtained by taking pictures of a sphere under different calibration light sources. The bright position of the bright area and the center position of the sphere in each calibration image are determined. Based on the preset reflection direction, the center position, and the bright position, the normal vector of the sphere surface in each calibration image is determined. Based on each normal vector and the preset reflection direction, the direction corresponding to each calibration light source is determined. Thus, the direction of the light source can be accurately calibrated by the bright area reflected from the sphere surface, effectively realizing the calibration of the light source direction.

[0091] In one embodiment, determining the highlight position of a highlighted region in each calibration image includes: determining the position of a key point in the highlighted region of each calibration image; and determining the highlight position corresponding to each highlighted region based on the position of the key point in each highlighted region.

[0092] The key point can be at least one of the edge point and the center point of the highlighted area.

[0093] Specifically, the computer device can identify highlighted areas in the calibration image and extract key points from these areas. The computer device determines the position corresponding to each key point, calculates an average position based on the position of each key point, and uses this average position as the highlighted position for that highlighted area. Following the same processing method, the highlighted positions corresponding to highlighted areas in each calibration image can be calculated.

[0094] In one embodiment, the location of a key point can be the coordinates of the key point in the corresponding calibration image, and the highlight location corresponding to the highlight area can be the highlight coordinates corresponding to the highlight area.

[0095] In one embodiment, the computer device can extract edge points from the highlighted area and determine the coordinates of each edge point. An average coordinate is calculated based on the coordinates of each edge point, and this average coordinate is used as the highlight coordinates corresponding to the highlighted area.

[0096] In one embodiment, the computer device can determine the coordinates of the center point of the highlighted area and use the coordinates of the center point as the highlighted coordinates corresponding to the highlighted area.

[0097] In this embodiment, the location of key points in the highlighted area of ​​each calibration image is determined, which enables the accurate determination of the highlighted position corresponding to each highlighted area based on the location of key points in each highlighted area.

[0098] In one embodiment, such as Figure 6 As shown, the generation of material images and surface geometry images based on multi-light source images includes:

[0099] Step S602: Perform pixel matching processing on multiple light source images to obtain the matching target pixels in the multiple light source images.

[0100] Specifically, the computer device performs pixel matching processing on multiple light source images to obtain matching target pixels among the multiple light source images. For example, pixel matching processing is performed on three light source images to obtain matching target pixels among these three light source images.

[0101] In one embodiment, a computer device may extract pixels from one of the light source images, and then match the extracted pixels with the remaining light source images to obtain matching target pixels.

[0102] Step S604: Determine the reflection information corresponding to the target pixel based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image.

[0103] The reflection information refers to the relevant information about the surface of the target object reflecting the incident calibration light source. Furthermore, the reflection information may include relevant information about the surface of the target object reflecting red, green, and blue light from the incident calibration light source.

[0104] Specifically, the computer device determines the pixel value of each target pixel in the corresponding light source image and obtains the direction vector corresponding to each calibrated light source. Based on the direction vector of each calibrated light source and the pixel value of each target pixel in the corresponding light source image, the reflection information corresponding to the target pixel is calculated.

[0105] In one embodiment, the reflection information includes reflectivity, and the computer device can calculate the reflectivity corresponding to each target pixel based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image.

[0106] Step S606: Generate a material image based on the reflection information corresponding to each target pixel.

[0107] Specifically, the computer device generates a material image based on the reflection information corresponding to each target pixel.

[0108] In one embodiment, the reflection information includes reflectivity. After the computer device calculates the reflectivity corresponding to each target pixel, it generates a material image based on the reflectivity of each target pixel.

[0109] It is understandable that matching target pixels have the same reflectance; for example, three matching target pixels correspond to the same reflectance.

[0110] In one embodiment, if there is a first correlation between the pixel values ​​of the calibration image, the reflectivity of the pixels, and the direction vector of the calibration light source, then the computer device can construct a matrix from the direction vectors of each calibration light source, construct a vector from the pixel values ​​of the matching target pixels in the corresponding calibration image, and substitute the constructed matrix and vector into the first correlation to calculate the reflectivity corresponding to the matching target pixel. Thus, the reflectivity corresponding to each matching target pixel can be calculated.

[0111] Step S608: Generate a surface geometric image based on the direction vector of each calibrated light source, the target pixel point, and the reflection information.

[0112] Specifically, the computer device generates a surface geometry image based on the pixel value of each target pixel in the corresponding calibration image, the direction vector of each calibration light source, and the reflection information corresponding to each target pixel.

[0113] In this embodiment, pixel matching processing is performed on multiple light source images to obtain matching target pixels in the multiple light source images. Based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image, the reflectivity of the calibrated light source for the target pixel pair can be accurately calculated. Based on the reflectivity corresponding to each target pixel, a material image is generated, enabling accurate characterization of the surface properties of the target object. Based on the direction vector of each calibrated light source, the target pixel, and reflection information, a surface geometry image is accurately generated, allowing the depth information of the target object's surface to be represented through the surface geometry image.

[0114] In one embodiment, such as Figure 7 As shown, the material image includes a red channel material image, a green channel material image, and a blue channel material image. Based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image, the reflection information corresponding to the target pixel is determined, including steps S702 to S704:

[0115] Step S702: Determine the red channel value, green channel value, and blue channel value of the target pixel in the corresponding light source image.

[0116] Specifically, the computer device performs pixel matching processing on multiple light source images to obtain the target pixels that match in the multiple light source images.

[0117] The pixel value of a target pixel is composed of the red channel value in the red channel, the green channel value in the green channel, and the blue channel value in the blue channel. The computer device determines the red channel value, green channel value, and blue channel value of each target pixel in the corresponding calibration image.

[0118] Step S704: Based on the direction vector of each calibrated light source and the red channel value, green channel value and blue channel value corresponding to the target pixel, determine the reflection information of the red channel, green channel and blue channel of the target pixel.

[0119] Specifically, the reflection information may include information related to the reflection of red, green, and blue light from an incident calibrated light source by the surface of the target object.

[0120] The computer device acquires the direction vectors corresponding to each calibrated light source, and calculates the reflection information of the target pixel corresponding to the red channel based on the direction vectors of each calibrated light source and the red channel value of each target pixel in the corresponding light source image.

[0121] The computer equipment calculates the reflection information of the target pixel corresponding to the green channel based on the direction vector of each calibrated light source and the green channel value of each target pixel in the corresponding light source image.

[0122] The computer device calculates the reflection information of the target pixel corresponding to the blue channel based on the direction vector of each calibrated light source and the blue channel value of each target pixel in the corresponding light source image.

[0123] In one embodiment, the reflection information includes reflectance. The computer device can calculate the reflectance of the target pixel corresponding to the red channel based on the direction vector of each calibrated light source and the red channel value of the target pixel in the corresponding light source image. The reflectance of the target pixel corresponding to the red channel characterizes the reflectance of the target pixel to red light in the calibrated light source.

[0124] The computer device can calculate the reflectance of the target pixel's green channel based on the direction vector of each calibrated light source and the green channel value of the target pixel in the corresponding light source image. The reflectance of the target pixel's green channel characterizes the reflectance of that target pixel to green light from the calibrated light source.

[0125] The computer device can calculate the reflectance of the target pixel in the blue channel based on the direction vector of each calibrated light source and the blue channel value of the target pixel in the corresponding light source image. The reflectance of the target pixel in the blue channel represents the reflectance of the target pixel to blue light from the calibrated light source.

[0126] Based on the reflection information corresponding to each target pixel, a material image is generated, including step S706:

[0127] Step S706: Based on the reflection information of the red channel, the green channel, and the blue channel corresponding to each target pixel, generate a red channel material image, a green channel material image, and a blue channel material image, respectively.

[0128] Specifically, the computer device generates a red channel material image based on the reflection information of the red channel corresponding to each target pixel. The computer device generates a green channel material image based on the reflection information of the green channel corresponding to each target pixel. The computer device generates a blue channel material image based on the reflection information of the blue channel corresponding to each target pixel.

[0129] In one embodiment, the computer device generates a red channel material image based on the reflectance of the red channel corresponding to each target pixel. The computer device generates a green channel material image based on the reflectance of the green channel corresponding to each target pixel. The computer device generates a blue channel material image based on the reflectance of the blue channel corresponding to each target pixel.

[0130] In one embodiment, there is a mapping relationship between the pixel values ​​and reflectance of the calibration image and the direction vector of the calibration light source. The computer device can then construct a matrix using the direction vectors of each calibration light source, construct a channel vector using the red channel values ​​of the matching target pixel in the corresponding calibration image, and substitute the constructed matrix and channel vector into the mapping relationship to calculate the reflectance of the matching target pixel corresponding to the red channel. Using the same processing method, the reflectance of the target pixel corresponding to the green channel and the reflectance corresponding to the blue channel can be calculated.

[0131] In this embodiment, the red, green, and blue channel values ​​of the target pixel in the corresponding light source image are determined. Based on the direction vector of each calibrated light source and the red, green, and blue channel values ​​corresponding to the target pixel, the reflection information of the target pixel in the red, green, and blue channels is determined. This allows for the generation of corresponding red channel material images, green channel material images, and blue channel material images based on the reflection information of each target pixel in the red, green, and blue channels. By performing independent reflectivity calculations for each color channel, the reflection information of the pixel to the calibrated light source in different color channels can be calculated separately. This allows for a comprehensive representation of the surface characteristics of the target object, improving the completeness of subsequent defect detection.

[0132] In one embodiment, generating a surface geometric image based on the direction vector of each calibrated light source, the target pixel, and reflection information includes: determining the normal vector corresponding to the target pixel based on the reflection information, the direction vector of each calibrated light source, and the pixel value of the target pixel in the corresponding light source image; and generating a surface geometric image based on the normal vector corresponding to each target pixel.

[0133] Specifically, the computer device calculates the normal vector corresponding to each target pixel based on the pixel value of each target pixel in the corresponding calibration image, the direction vectors of each calibration light source, and the reflection information corresponding to each target pixel. Based on the normal vector corresponding to each target pixel, a surface geometric image representing the depth information of the target object is generated.

[0134] In one embodiment, a second correlation exists between the pixel values ​​of the calibration image, the reflectivity of the pixels, the direction vector of the calibration light source, and the normal vector of the target object surface. The computer device can then construct a matrix using the direction vectors of each calibration light source, construct a vector using the pixel values ​​of the matching target pixels in the corresponding calibration image, and substitute the constructed matrix and vector into the first correlation to calculate the reflectivity corresponding to the matching target pixel. After calculating the reflectivity, the constructed matrix, vector, and reflectivity can be substituted into the second correlation to calculate the normal vector corresponding to the matching target pixel. Thus, the normal vector corresponding to each matching target pixel can be calculated.

[0135] In this embodiment, the normal vector corresponding to the target pixel is accurately determined based on the reflection information, the direction vector of each calibrated light source, and the pixel value of the target pixel in the corresponding light source image. Based on the normal vector corresponding to each target pixel, a surface geometric image can be generated to characterize the depth information of the target object, thereby providing corresponding depth information in subsequent defect detection and improving the accuracy of defect detection.

[0136] In one embodiment, the target object surface has Lambertian reflection characteristics, meaning that the target object surface diffusely reflects incident light, and the intensity of the reflected light is consistent in every direction. Then, the pixel values ​​of the calibration image, the reflectivity of the pixels, the direction vector of the calibration light source, and the normal vector of the target object surface are related by the following formula:

[0137] t ij =ρ i (N i ·L j )l j (3)

[0138] Among them, t ij t represents the pixel value formed by the i-th point on the surface of the target object illuminated by the calibration light source j. ij This represents the pixel value of pixel i in the calibration image corresponding to the calibration light source j. ρ i Let N be the reflectance of pixel i. i ρ is the unit normal vector of the target object's surface, i.e., the normal vector of pixel i. i and N iThis represents the characteristics of the target object's surface, namely ρ. i It reflects the material information of the target object, while N i It reflects the surface geometry information of the target object. L j To calibrate the direction vector of light source j, this direction vector can be a unit vector. j To calibrate the light source intensity of light source j.

[0139] Assume N i and L j Both are 3×1 unit vectors in three-dimensional space, and the intensity of the light source can be represented by a constant 1. Pixel value t ij This can be obtained directly from the calibration image, and the direction vector of the calibration light source can be determined through the calibration process. Then, without moving the target object or the camera, at least three non-coplanar light rays can be used to illuminate the i-th point on the surface of the target object, thereby illuminating N. i and ρ i Solve it.

[0140] For example, using three calibration light sources in different directions, the matrix formed by the direction vectors of the three calibration light sources is denoted as L = [L1, L2, L3]. T One calibration image is captured under each calibration light source, resulting in three calibration images. The pixel values ​​of pixel i in the three calibration images form a vector t. i Assume L -1 If it exists, then the above formula (3) can be converted to:

[0141]

[0142] To avoid shadows cast by the calibration light source on the target object's surface, which could prevent the three light sources from simultaneously illuminating areas and thus create unsolvable problems, more calibration light sources can be used to illuminate the target object from different directions and form an image. For example, in one embodiment, assuming there are n ≥ 3 light sources, then:

[0143] L = [L1, L2, L3, ..., L n ] T ,t i =[t i1 ,t i2 ,t i3 ,…t in ] T (5)

[0144] The intensity of the light source can be represented by a constant 1, so formula (3) can be transformed into:

[0145] t ij =ρ i N i L (6)

[0146] Therefore:

[0147]

[0148] The above formula can be used to calculate ρ for each pixel i. i and N i Through the ρ corresponding to each pixel i It can generate material images, through N corresponding to each pixel. i It can generate surface geometry images. The generated surface geometry images are as follows: Figure 8 As shown, Figure 8 The image provides both a color image and a surface geometry image of the target object. As can be clearly seen from the image, the surface geometry image can clearly depict the depth information of the target object's surface compared to the color image.

[0149] In one embodiment, defect detection is performed on a target object based on a color image, a material image, and a surface geometry image to obtain defect detection results for the target object, including:

[0150] Feature extraction is performed on the color image, material image, and surface geometry image to obtain color features, material features, and geometric features, respectively. The color features, material features, and geometric features are then fused to obtain fused features. Defect detection is performed on the target object based on the fused features to obtain the defect detection results of the target object.

[0151] Specifically, the computer equipment extracts features from the color image, the material image, and the surface geometry image respectively, to obtain the color features corresponding to the color image, the material features corresponding to the material image, and the geometric features corresponding to the surface geometry image.

[0152] In one embodiment, the computer device can obtain the weights corresponding to color features, material features, and geometric features, and generate a fused feature based on the color features, material features, geometric features, and their respective weights. Further, the computer device can multiply the color features, material features, geometric features, and their respective weights, and then add the products to obtain the fused feature.

[0153] Computer equipment can perform defect detection on a target object based on fused features, so as to detect the defects existing in the target object and generate corresponding defect detection results.

[0154] In one embodiment, a computer device can input a color image, a material image, and a surface geometry image into a defect detection model. The defect detection model extracts depth features from the color image, material image, and surface geometry image respectively, obtaining color features corresponding to the color image, material features corresponding to the material image, and geometric features corresponding to the surface geometry image. The defect detection model multiplies the color features, material features, and geometric features by their respective weights, and then sums the products to obtain a fused feature. The defect model performs detection based on this fused feature and outputs the defect detection result for the target object.

[0155] In one embodiment, defect category detection, defect location detection, and defect area detection can be performed by fusing features to obtain the defect category, defect location, and defect area of ​​each defect in the target object. Based on the defect category, defect location, and defect area of ​​the target object, defect detection results for the target object are generated.

[0156] In this embodiment, feature extraction is performed on the color image, material image, and surface geometry image respectively to obtain color features, material features, and geometric features, thereby extracting depth features from the image. The color features, material features, and geometric features are then fused to obtain fused features, which integrate features from the three types of images. This fused feature includes multiple aspects of the target object, such as color information, material information, and depth information. Defect detection of the target object is performed based on this multi-faceted fused feature, enabling complementary detection of defects with and without depth information. This effectively avoids missed or false detections of defects and improves the accuracy of defect detection.

[0157] In one embodiment, such as Figure 9 As shown, defect detection of the target object is performed based on fusion features, and the defect detection results of the target object are obtained, including:

[0158] Step S902: Extract the local features to be detected from the fused features, perform defect detection on the extracted local features, and obtain the local detection results corresponding to the local features.

[0159] Specifically, the computer device can extract the local features to be detected from the fused features, and perform defect detection on the extracted local features to obtain the local detection result corresponding to the local feature.

[0160] In one embodiment, a computer device can extract local features to be detected from fused features through a defect detection model. The defect detection model performs defect category detection, defect location detection, and defect area detection on the local features to obtain the local detection result corresponding to the local feature.

[0161] Step S904: Determine whether there are any local features in the fused features that have not been defect detected. If yes, return to step S902 and continue execution; otherwise, execute step S906.

[0162] Specifically, after obtaining the local detection results corresponding to the extracted local features, the computer device continues to extract the next local feature to be detected from the fusion extraction, and performs defect detection on the extracted local features to obtain the local detection results corresponding to the local features, until there are no local features in the fusion features that have not been defect detected, and then stops, obtaining the local detection results corresponding to each local feature.

[0163] Step S906: Based on the local detection results, generate the defect detection results of the target object; the defect detection results include at least one of the following: defect category, defect location, and defect area.

[0164] Specifically, the computer device determines the defects existing in the target object based on the local detection results corresponding to each local feature region, as well as at least one of the defect category, defect location, and defect area. Based on the defects existing in the target object and at least one of the defect category, defect location, and defect area, a defect detection result for the target object is generated.

[0165] In this embodiment, local features to be detected are extracted from the fused features, and defect detection is performed on the extracted local features to obtain local detection results corresponding to the local features. This enables local defect detection of the target object based on local features, avoiding missed detections. The step of extracting local features to be detected from the fused features is returned and continues until there are no undetected local features in the fused features. By performing local defect detection on the target object, local detection results corresponding to each local feature can be obtained, allowing a complete defect detection result of the target object to be generated based on each local detection result, thus improving the comprehensiveness of defect detection.

[0166] In one embodiment, defect detection is performed on the extracted local features to obtain local detection results corresponding to the local features, including:

[0167] Based on the extracted local features, defect category detection, defect location detection, and defect area detection are performed respectively to obtain the local detection results corresponding to the local features. The local detection results include the defect category, defect mask, and defect bounding box corresponding to the local features.

[0168] Specifically, the computer device extracts local features from the fused features, performs defect category detection on the extracted local features to obtain the defect category corresponding to the defects present in the local features. The computer device then performs defect location detection on the extracted local features to obtain a defect mask map corresponding to the local feature. This defect mask map reflects the defect location within the local feature. Next, the computer device performs defect area detection on the extracted local features, generating a defect bounding box within the local feature region. The area enclosed by this defect bounding box represents the area occupied by the defect within the local feature, i.e., the defect area. Based on the defect category, defect mask map, and defect bounding box corresponding to the local feature, the computer device generates the local detection result for that local feature.

[0169] In this embodiment, defect category detection, defect location detection, and defect area detection are performed based on the extracted local features. This allows for multi-branch detection tasks based on the same local features, improving detection efficiency and quickly obtaining the defect category, defect mask, and defect bounding box corresponding to the defects in the local features, thereby accurately obtaining the local detection results corresponding to the local features.

[0170] In one embodiment, based on the results of each local detection, defect detection results for the target object are generated, including:

[0171] Based on the defect mask map corresponding to each local feature, an object mask map containing the target object is generated; based on the object mask map, the defect category corresponding to each local feature and the defect bounding box, the defect category corresponding to the defect existing in the target object, the defect location corresponding to each defect, and the defect area of ​​each defect are obtained.

[0172] Specifically, the local detection results corresponding to each local feature include a defect category, a defect mask, and a defect bounding box. The computer device can acquire the defect mask corresponding to each local feature and, based on the defect masks corresponding to each local feature, generate an object mask corresponding to the target object. This object mask image can represent the defect location corresponding to the defects existing in the target object. The computer device maps the defect category and defect bounding box corresponding to each local feature to the object mask image, thereby obtaining the defects existing in the target object, as well as the defect category and the defect bounding box corresponding to each defect. The area of ​​the defect bounding box is calculated using its coordinates; this area is the defect area. Based on the determined defect categories, defect locations, and defect areas of the target object, the defect detection results for the target object are generated.

[0173] In one embodiment, the object mask image is the same size as the color image, material image, and surface geometry image.

[0174] In this embodiment, an object mask containing the target object is generated based on the defect mask corresponding to each local feature. This allows for the accurate generation of a mask with the original image size based on the local mask. According to the object mask, the defect category corresponding to each local feature, and the defect bounding box, the defect category corresponding to the defects existing in the target object, the defect location corresponding to each defect, and the defect area of ​​each defect are accurately detected.

[0175] In one embodiment, the target object is a metal injection molded device; the multi-light source image includes at least three light source images obtained by taking pictures of the metal injection molded device under at least three calibrated light sources in at least three different directions; the defect category in the defect detection result includes at least one of cracks, dents, material sticking, material shortage, bright marks, and dirt.

[0176] Specifically, in industry, it is necessary to perform defect detection on the manufactured metal injection molded (MIM) parts. However, the defects of MIM parts usually contain depth information, and it is difficult to capture depth features with RGB data from a single light source. Therefore, most traditional defect inspection methods based on RGB data are prone to missing defects.

[0177] In this embodiment, the direction vector corresponding to each calibration light source is determined through a calibration step. When performing defect detection on a MIM component, at least three calibration light sources emitting from a sector light source can illuminate the surface of the MIM component. A camera is used to capture images of the MIM component surface under the calibration light sources from different directions, obtaining three corresponding light source images. Alternatively, a color image can be obtained by capturing an image of the MIM component surface under a single light source.

[0178] Computer equipment can perform pixel matching processing on three light source images to obtain matching target pixels in the three light source images, and then construct a vector from the pixel values ​​of the three matching target pixels in their respective light source images.

[0179] The direction vectors corresponding to the three calibration light sources are used to construct a matrix. Substituting the constructed vectors and matrix into the above formula (7), i.e., ρ i =|(L T L) -1 L T t i |, The reflectance and normal vector corresponding to each target pixel can be calculated separately. It can be understood that three matching target pixels correspond to the same reflectance and the same normal vector.

[0180] The computer device generates a material image for the MIM component based on the reflectivity of each target pixel. This material image represents the reflection information of the MIM component surface. The computer device also generates a surface geometry image based on the normal vector corresponding to each target pixel. This surface geometry image represents the depth information of the MIM component surface.

[0181] The computer equipment extracts features from the color image, material image, and surface geometry image respectively, obtaining corresponding color features, material features, and geometric features. The color features, material features, and geometric features are then weighted and fused with their respective weights to obtain fused features. Local features to be detected are extracted from the fused features. Based on the extracted local features, defect category detection, defect location detection, and defect area detection are performed respectively, obtaining local detection results corresponding to the local features. The local detection results include the defect category, defect mask, and defect bounding box corresponding to the local feature. The defect categories of this MIM part include at least one of cracks, indentations, material adhesion, missing material, bright marks, and dirt. The process returns to the step of extracting the local features to be detected from the fused features and continues until there are no undetected local features in the fused features, at which point the process stops, obtaining the local detection results corresponding to each local feature.

[0182] The computer device generates an object mask containing the MIM component based on the defect mask corresponding to each local feature. According to the object mask, the defect categories and defect bounding boxes corresponding to each local feature, the computer device obtains the defect category, the defect location, and the defect area of ​​each defect for each defect present in the MIM component. The computer device integrates the defects present in the MIM component, their corresponding defect categories, their defect locations, and their defect areas into a defect detection result for the MIM component.

[0183] In this embodiment, the defect detection method is applied to industrial metal injection molded devices. Multi-source image data allows for better characterization of defect depth information. Combining RGB imaging from a single light source with photometric stereoscopic data from multiple light sources provides complementarity in characterizing defect features in metal injection molded devices, thereby enabling higher-quality defect inspection of these devices.

[0184] like Figure 10 The diagram shown is an architecture diagram of a defect detection model in one embodiment, which includes the following multiple processing stages:

[0185] (1) Multiple source input

[0186] The defect detection model takes as input a single-light source RGB image I, and a material image A and a surface geometry image N obtained based on a photometric stereo algorithm. The RGB image I and material image A can effectively characterize defects lacking depth information, such as cracks, bright spots, and dirt. The surface geometry image N can effectively characterize defect depth information, such as indentations and missing material. Therefore, the RGB image I, material image A, and surface geometry image N are complementary in characterizing defect features, which can increase the defect detection rate of MIM parts.

[0187] (2) Feature extraction

[0188] The defects present in MIM components are generally small. Therefore, using HRNet, a high-resolution image preservation network, as the base network to extract image features can better preserve key image information and improve training efficiency. In this HRNet, the RGB image I, the material image A, and the surface geometry image N can simultaneously share the network, and after passing through the base network, depth feature maps F are obtained respectively. I F N F A .

[0189] (3) Feature fusion

[0190] During training, to ensure that the three different data sources—RGB image I, material image A, and surface geometry image N—complement each other at the feature level, the feature map F is trained to... I F N F A They all learn a weight vector along a channel dimension, denoted as w. I w N w A This is to achieve better defect detection and segmentation. Weight parameter w I w N w A Learned from training data.

[0191] Through F=w I F I +w N F N +w A F A Then the fusion feature F can be obtained.

[0192] (4) Bullish prediction

[0193] MIM component defect inspection not only needs to predict the defect type, but also the defect location and area size. By decoupling the prediction header, the three tasks are decoupled into three independent prediction branches, which respectively complete defect type detection, defect location detection, and defect area detection, thus avoiding conflicts between the three tasks.

[0194] Local features are extracted using ROIAlign on the fused feature F, and then fed into three sub-prediction branches: a classification branch, a defect mask branch, and a bounding box prediction branch. The classification branch mainly consists of two fully connected layers. The bounding box prediction branch mainly consists of several concatenated convolutional layers, and finally uses global average pooling to regress the position coordinates of the corresponding candidate regions. The defect mask branch also consists of convolutional layers, with two blinenar upsampling operations used to upsample the ROIAlign features from 7×7 to 14×14, and then from 14×14 to 28×28. The mask predicted by the defect mask branch is post-processed, combining the position coordinates of the candidate regions and the image size to restore the size and fill the background, resulting in a mask of the original image size.

[0195] During the training of the defect detection model, the aspect ratio and area information of the bounding boxes in the training set can be statistically analyzed; these bounding boxes are the defect bounding boxes. KMeans clustering is performed on the aspect ratios of the bounding boxes, with the number of cluster centers set to 5, 9, and 15. Considering the results of at least three clustering operations, the anchor ratios are adjusted to: 0.1, 0.2, 0.5, 1.0, 2.0, 3.5, 10, and 25. KMeans clustering is then performed on the areas of the bounding boxes, taking into account both the anchor size and the clustering results; ultimately, the anchor scale is set to 2, 4, and 8.

[0196] In one embodiment, a comparison of partial defect quality inspection results is based on the fusion of RGB images and multi-source data, where multi-source data refers to RGB images, material images, and surface geometry images. The comparison results are shown in the table below:

[0197] Method Comparison mAP@50 mAP@10 crack pressure injury Material shortage Dipping sauce RGB image 37.8 39.9 89.2 47.8 57.5 5.2 Multi-source data fusion 40.1 42.6 90.08 50.9 60.8 10.6

[0198] The average precision (AP) for each defect category is measured at an Intersection over Union (IOU) ratio of 0.1. mAP@50 and mAP@10 represent the average AP for all categories when the IOU is 0.5 and 0.1, respectively. For example, 37.8 refers to the average AP for cracks, indentations, missing material, and adhesion when the IOU is 0.5. mAP (mean average precision) is the average AP value.

[0199] As can be seen from the table, the network based on RGB data and multi-source data fusion significantly improves all indicators of defect inspection, especially for defects with depth information such as dents, missing materials, and material adhesion. However, RGB data struggles to accurately capture this information, leading to missed detections. Using a network that fuses RGB data and multi-source data effectively leverages the data's ability to capture defect features, achieving excellent defect inspection results.

[0200] In one embodiment, a defect detection method is provided, applied to a computer device, comprising:

[0201] Acquire calibration images containing highlighted areas of a sphere obtained by photographing the sphere under different calibration light sources; determine the positions of key points in the highlighted areas of each calibration image; and determine the highlighted positions corresponding to each highlighted area based on the positions of the key points in each highlighted area.

[0202] Determine the center position of the sphere in each calibration image, and determine the normal vector of the sphere surface in each calibration image based on the preset reflection direction, center position, and highlight position; determine the direction vector corresponding to each calibration light source based on the normal vector of the sphere surface in each calibration image and the preset reflection direction vector.

[0203] Acquire a color image of the target object taken under a single light source, and acquire at least three light source images of the target object taken under three calibrated light sources from three different directions. One light source image is obtained from a calibrated light source taken from one direction.

[0204] At least three light source images are subjected to pixel matching processing to obtain the target pixels that match in the at least three light source images.

[0205] Determine the red, green, and blue channel values ​​of the target pixel in the corresponding light source image; based on the direction vector of each calibrated light source and the red, green, and blue channel values ​​of the target pixel, determine the reflectivity of the red, green, and blue channels of the target pixel.

[0206] A red channel material image is generated based on the reflectance of the red channel corresponding to each target pixel; a green channel material image is generated based on the reflectance of the green channel corresponding to each target pixel; and a blue channel material image is generated based on the reflectance of the blue channel corresponding to each target pixel.

[0207] The normal vector corresponding to the target pixel is determined based on the reflectance of the red channel, green channel, and blue channel of the target pixel, the direction vector of each calibrated light source, and the pixel value of the target pixel in the corresponding light source image.

[0208] A surface geometry image is generated based on the normal vector corresponding to each target pixel; the surface geometry image represents the depth information of the target object's surface.

[0209] Color images, material images, and surface geometry images are input into the defect detection model, which includes a high resolution preservation network (HRNet), a feature fusion network, and a multi-head prediction network.

[0210] HRNet is used to extract deep features from color images, material images, and surface geometry images respectively, resulting in color features corresponding to color images, material features corresponding to material images, and geometric features corresponding to surface geometry images.

[0211] The feature fusion network includes weights corresponding to color features, material features, and geometric features. The fused features are obtained by multiplying the color features, material features, geometric features, and their corresponding weights.

[0212] The fused features are input into a multi-head prediction network, which includes three prediction sub-branches: a classification branch, a defect mask branch, and a bounding box prediction branch. Local features to be detected are extracted from the fused features and input into each of the three prediction sub-branches. The classification branch predicts the defects present in the local features and their respective defect categories based on the local features. The defect mask branch predicts the defect mask map corresponding to the defects present in the local features. The bounding box prediction branch generates defect bounding boxes within the local features, thus determining the defect area by enclosing the defects within the local features.

[0213] The defect detection model generates an object mask containing the target object based on the defect mask corresponding to each local feature. According to the object mask, the defect category and defect bounding box corresponding to each local feature, the defect category corresponding to the defect existing in the target object, the defect location corresponding to each defect, and the defect area of ​​each defect are obtained.

[0214] In this embodiment, the direction vectors corresponding to each calibrated light source are determined in advance by calibrating the light source direction. When defect detection is required, multiple light source images of the target object are acquired under calibrated light sources with different directions, as well as a color image of the target object under a single light source. The red, green, and blue channel values ​​of the target pixel in the corresponding light source image are determined. Based on the direction vectors of each calibrated light source and the red, green, and blue channel values ​​of the target pixel, the reflection information of the target pixel in the red, green, and blue channels is determined. Thus, based on the reflection information of each target pixel in the red, green, and blue channels, corresponding red channel material images, green channel material images, and blue channel material images can be generated respectively. By performing independent reflectivity calculations on each color channel, the reflection information of the pixel on the calibrated light source in different color channels can be calculated separately.

[0215] Based on reflection information, the direction vectors of each calibrated light source, and the pixel value of the target pixel in the corresponding light source image, the normal vector corresponding to the target pixel is accurately determined. Based on the normal vector corresponding to each target pixel, a surface geometry image representing the depth information of the target object can be generated, providing corresponding depth information in subsequent defect detection and improving defect detection accuracy.

[0216] Feature extraction is performed on color images, material images, and surface geometry images separately to obtain color features, material features, and geometric features, thereby extracting depth features from the images. These color features, material features, and geometric features are then fused to obtain fused features, which integrate features from the three types of images. These fused features contain multiple aspects of the target object, including color, material, and depth information. Defect detection of the target object is performed based on these multi-faceted fused features, enabling complementary detection of defects with and without depth information. This effectively avoids missed or false detections of defects, improving the accuracy and comprehensiveness of defect detection.

[0217] It should be understood that, although Figures 2-10 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2-10At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0218] In one embodiment, such as Figure 11 As shown, a defect detection device 1100 is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: a first acquisition module 1102, a second acquisition module 1104, a generation module 1106, and a detection module 1108, wherein:

[0219] The first acquisition module 1102 is used to acquire a color image of a target object obtained by taking a picture under a single light source.

[0220] The second acquisition module 1104 is used to acquire multi-light source images corresponding to the target object. The multi-light source images include multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions.

[0221] The generation module 1106 is used to generate material images and surface geometry images based on multi-source images; the material image represents the material information of the target object, and the surface geometry image represents the depth information of the surface of the target object.

[0222] The detection module 1108 is used to perform defect detection on the target object based on the color image, material image and surface geometry image, and obtain the defect detection result of the target object.

[0223] In this embodiment, a color image of the target object taken under a single light source is acquired, and a multi-light source image corresponding to the target object is obtained. This multi-light source image includes multiple light source images taken under calibrated light sources from different directions. Based on the multi-light source image, a material image representing the material information of the target object and a surface geometry image representing the depth information of the target object's surface can be accurately generated. The color image and material image can better characterize defects lacking depth information, while the surface geometry image can better characterize the depth information of defects. Defect detection of the target object based on the color image, material image, and surface geometry image achieves complementarity in defect detection with and without depth information, thereby more accurately and comprehensively detecting defects in the target object, reducing the false negative rate, and effectively improving the accuracy and comprehensiveness of defect detection.

[0224] In one embodiment, the device further includes a calibration module; the calibration module is used to acquire calibration images containing bright areas obtained by taking pictures of the sphere under illumination from different calibration light sources; determine the bright position of the bright area and the center position of the sphere in each calibration image, and determine the normal vector of the sphere surface in each calibration image according to the preset reflection direction, the center position, and the bright position; and determine the direction corresponding to each calibration light source based on each normal vector and the preset reflection direction.

[0225] In this embodiment, calibration images containing bright areas are obtained by taking pictures of a sphere under different calibration light sources. The bright position of the bright area and the center position of the sphere in each calibration image are determined. Based on the preset reflection direction, the center position, and the bright position, the normal vector of the sphere surface in each calibration image is determined. Based on each normal vector and the preset reflection direction, the direction corresponding to each calibration light source is determined. Thus, the direction of the light source can be accurately calibrated by the bright area reflected from the sphere surface, effectively realizing the calibration of the light source direction.

[0226] In one embodiment, the calibration module is further configured to determine the position of key points in the highlighted areas of each calibration image; and to determine the highlighted position corresponding to each highlighted area based on the position of the key points in each highlighted area.

[0227] In this embodiment, the location of key points in the highlighted area of ​​each calibration image is determined, which enables the accurate determination of the highlighted position corresponding to each highlighted area based on the location of key points in each highlighted area.

[0228] In one embodiment, the generation module 1106 is further configured to perform pixel matching processing on multiple light source images to obtain matching target pixels in the multiple light source images; determine the reflection information corresponding to the target pixel based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image; generate a material image based on the reflection information corresponding to each target pixel; and generate a surface geometry image based on the direction vector of each calibrated light source, the target pixel and the reflection information.

[0229] In this embodiment, pixel matching processing is performed on multiple light source images to obtain matching target pixels in the multiple light source images. Based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image, the reflectivity of the calibrated light source for the target pixel pair can be accurately calculated. Based on the reflectivity corresponding to each target pixel, a material image is generated, enabling accurate characterization of the surface properties of the target object. Based on the direction vector of each calibrated light source, the target pixel, and reflection information, a surface geometry image is accurately generated, allowing the depth information of the target object's surface to be represented through the surface geometry image.

[0230] In one embodiment, the material image includes a red channel material image, a green channel material image, and a blue channel material image. The generation module 1106 is further configured to determine the red channel value, green channel value, and blue channel value of the target pixel in the corresponding light source image; determine the reflection information of the target pixel in the red channel, green channel, and blue channel based on the direction vector of each calibrated light source and the red channel value, green channel value, and blue channel value corresponding to the target pixel; and generate the red channel material image, green channel material image, and blue channel material image respectively based on the reflection information of the red channel, green channel, and blue channel corresponding to each target pixel.

[0231] In this embodiment, the red, green, and blue channel values ​​of the target pixel in the corresponding light source image are determined. Based on the direction vector of each calibrated light source and the red, green, and blue channel values ​​corresponding to the target pixel, the reflection information of the target pixel in the red, green, and blue channels is determined. This allows for the generation of corresponding red channel material images, green channel material images, and blue channel material images based on the reflection information of each target pixel in the red, green, and blue channels. By performing independent reflectivity calculations for each color channel, the reflection information of the pixel to the calibrated light source in different color channels can be calculated separately. This allows for a comprehensive representation of the surface characteristics of the target object, improving the completeness of subsequent defect detection.

[0232] In one embodiment, the generation module 1106 is further configured to determine the normal vector corresponding to the target pixel based on the reflection information, the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image; and generate a surface geometric image based on the normal vector corresponding to each target pixel.

[0233] In this embodiment, the normal vector corresponding to the target pixel is accurately determined based on the reflection information, the direction vector of each calibrated light source, and the pixel value of the target pixel in the corresponding light source image. Based on the normal vector corresponding to each target pixel, a surface geometric image can be generated to characterize the depth information of the target object, thereby providing corresponding depth information in subsequent defect detection and improving the accuracy of defect detection.

[0234] In one embodiment, the detection module 1108 is further configured to extract features from the color image, the material image, and the surface geometry image respectively to obtain color features, material features, and geometric features; fuse the color features, material features, and geometric features to obtain fused features; and perform defect detection on the target object based on the fused features to obtain the defect detection result of the target object.

[0235] In this embodiment, feature extraction is performed on the color image, material image, and surface geometry image respectively to obtain color features, material features, and geometric features, thereby extracting depth features from the image. The color features, material features, and geometric features are then fused to obtain fused features, which integrate features from the three types of images. This fused feature includes multiple aspects of the target object, such as color information, material information, and depth information. Defect detection of the target object is performed based on this multi-faceted fused feature, enabling complementary detection of defects with and without depth information. This effectively avoids missed or false detections of defects and improves the accuracy of defect detection.

[0236] In one embodiment, the detection module 1108 is further configured to extract local features to be detected from the fused features, perform defect detection on the extracted local features, and obtain local detection results corresponding to the local features; return to the step of extracting local features to be detected from the fused features and continue execution until there are no local features in the fused features that have not been defect detected, and stop, thereby obtaining local detection results corresponding to each local feature; and generate defect detection results for the target object based on each local detection result; the defect detection results include at least one of the following: defect category, defect location, and defect area in the target object.

[0237] In this embodiment, local features to be detected are extracted from the fused features, and defect detection is performed on the extracted local features to obtain local detection results corresponding to the local features. This enables local defect detection of the target object based on local features, avoiding missed detections. The step of extracting local features to be detected from the fused features is returned and continues until there are no undetected local features in the fused features. By performing local defect detection on the target object, local detection results corresponding to each local feature can be obtained, allowing a complete defect detection result of the target object to be generated based on each local detection result, thus improving the comprehensiveness of defect detection.

[0238] In one embodiment, the detection module 1108 is further configured to perform defect category detection, defect location detection, and defect area detection based on the extracted local features, and obtain local detection results corresponding to the local features; the local detection results include the defect category, defect mask map, and defect bounding box corresponding to the local features.

[0239] In this embodiment, defect category detection, defect location detection, and defect area detection are performed based on the extracted local features. This allows for multi-branch detection tasks based on the same local features, improving detection efficiency and quickly obtaining the defect category, defect mask, and defect bounding box corresponding to the defects in the local features, thereby accurately obtaining the local detection results corresponding to the local features.

[0240] In one embodiment, the detection module 1108 is further configured to generate an object mask containing the target object based on the defect mask corresponding to each local feature; and to obtain the defect category corresponding to the defect existing in the target object, the defect location corresponding to each defect, and the defect area of ​​each defect based on the object mask, the defect category corresponding to each local feature, and the defect bounding box.

[0241] In this embodiment, an object mask containing the target object is generated based on the defect mask corresponding to each local feature. This allows for the accurate generation of a mask with the original image size based on the local mask. According to the object mask, the defect category corresponding to each local feature, and the defect bounding box, the defect category corresponding to the defects existing in the target object, the defect location corresponding to each defect, and the defect area of ​​each defect are accurately detected.

[0242] In one embodiment, the target object is a metal injection molded device; the multi-light source image includes at least three light source images obtained by taking pictures of the metal injection molded device under at least three calibrated light sources in at least three different directions; the defect category in the defect detection result includes at least one of cracks, dents, material sticking, material shortage, bright marks, and dirt.

[0243] In this embodiment, the defect detection method is applied to industrial metal injection molded devices. Multi-source image data allows for better characterization of defect depth information. Combining RGB imaging from a single light source with photometric stereoscopic data from multiple light sources provides complementarity in characterizing defect features in metal injection molded devices, thereby enabling higher-quality defect inspection of these devices.

[0244] Specific limitations regarding the defect detection device can be found in the limitations of the defect detection method described above, and will not be repeated here. Each module in the aforementioned defect detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0245] In one embodiment, a computer device is provided, which may be a terminal or a server. This embodiment takes a terminal as an example, and its internal structure diagram can be as follows. Figure 12As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a defect detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0246] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0247] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0248] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0249] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0250] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0251] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0252] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A defect detection method, characterized in that, The method includes: To acquire a color image of a target object taken under a single light source; Acquire multi-light source images corresponding to the target object, wherein the multi-light source images include multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions; The multiple light source images are subjected to pixel matching processing to obtain the matching target pixels in the multiple light source images; Based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image, the reflection information corresponding to the target pixel is determined; A material image is generated based on the reflection information corresponding to each target pixel, and the material image represents the material information of the target object. A surface geometric image is generated based on the multi-source image, and the surface geometric image represents the depth information of the surface of the target object; Based on the color image, the material image, and the surface geometry image, defect detection is performed on the target object to obtain the defect detection result of the target object.

2. The method according to claim 1, characterized in that, The calibration light sources in different directions are obtained through a calibration process; the calibration process includes: Acquire calibration images containing highlighted areas of a sphere obtained by photographing it under different calibration light sources; Determine the highlight position of the highlighted area and the center position of the sphere in each calibration image, and determine the normal vector of the sphere surface in each calibration image based on the preset reflection direction, the center position of the sphere, and the highlight position; Based on the normal vectors and the preset reflection direction, the directions corresponding to each calibration light source are determined.

3. The method according to claim 2, characterized in that, Determining the highlight position of the highlighted region in each of the calibrated images includes: Determine the location of key points in the highlighted region of each of the calibrated images; Based on the location of the key points in each of the highlighted regions, the corresponding highlight position for each highlighted region is determined.

4. The method according to claim 1, characterized in that, The generation of the surface geometric image based on the multi-source image includes: A surface geometric image is generated based on the direction vectors of each of the calibrated light sources, the target pixel, and the reflection information.

5. The method according to claim 4, characterized in that, The material image includes a red channel material image, a green channel material image, and a blue channel material image. Determining the reflection information corresponding to the target pixel based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image includes: Determine the red channel value, green channel value, and blue channel value of the target pixel in the corresponding light source image; Based on the direction vectors of each calibrated light source and the red, green, and blue channel values ​​corresponding to the target pixel, determine the reflection information of the red channel, green channel, and blue channel of the target pixel; The step of generating a material image based on the reflection information corresponding to each target pixel includes: Based on the reflection information of the red channel, green channel, and blue channel corresponding to each target pixel, a red channel material image, a green channel material image, and a blue channel material image are generated respectively.

6. The method according to claim 4, characterized in that, The step of generating a surface geometric image based on the direction vectors of each of the calibrated light sources, the target pixel, and the reflection information includes: Based on the reflection information, the direction vectors of each of the calibrated light sources, and the pixel value of the target pixel in the corresponding light source image, the normal vector corresponding to the target pixel is determined. A surface geometric image is generated based on the normal vector corresponding to each target pixel.

7. The method according to claim 1, characterized in that, The step of performing defect detection on the target object based on the color image, the material image, and the surface geometry image to obtain the defect detection result of the target object includes: Feature extraction is performed on the color image, the material image, and the surface geometry image to obtain color features, material features, and geometric features, respectively. The color feature, the material feature, and the geometric feature are fused together to obtain the fused feature; Based on the fusion features, defect detection is performed on the target object to obtain the defect detection result of the target object.

8. The method according to claim 7, characterized in that, The step of performing defect detection on the target object based on the fused features to obtain the defect detection result of the target object includes: The local features to be detected are extracted from the fused features, and the extracted local features are subjected to defect detection to obtain the local detection results corresponding to the local features. Return to the step of extracting the local features to be detected from the fused features and continue execution until there are no local features in the fused features that have not been defect detected, and stop to obtain the local detection results corresponding to each local feature; Based on the local detection results, a defect detection result for the target object is generated; the defect detection result includes at least one of the following: defect category, defect location, and defect area.

9. The method according to claim 8, characterized in that, The step of performing defect detection on the extracted local features to obtain the local detection results corresponding to the local features includes: Based on the extracted local features, defect category detection, defect location detection, and defect area detection are performed respectively to obtain the local detection results corresponding to the local features; the local detection results include the defect category, defect mask, and defect bounding box corresponding to the local features.

10. The method according to claim 9, characterized in that, The step of generating defect detection results for the target object based on the local detection results includes: Based on the defect mask map corresponding to each of the local features, an object mask map containing the target object is generated; Based on the object mask, the defect category and defect bounding box corresponding to each local feature, the defect category corresponding to the defect existing in the target object, the defect location corresponding to each defect, and the defect area of ​​each defect are obtained.

11. The method according to any one of claims 1 to 10, characterized in that, The target object is a metal injection molded device; the multi-light source image includes at least three light source images obtained by taking pictures of the metal injection molded device under at least three calibrated light sources in at least three different directions; the defect category in the defect detection result includes at least one of cracks, dents, material sticking, material shortage, bright marks, and dirt.

12. A defect detection device, characterized in that, The device includes: The first acquisition module is used to acquire a color image of the target object obtained by taking a picture under a single light source; The second acquisition module is used to acquire a multi-light source image corresponding to the target object. The multi-light source image includes multiple light source images obtained by taking pictures of the target object under calibrated light sources in different directions. The generation module is used to perform pixel matching processing on the multiple light source images to obtain matching target pixels in the multiple light source images; determine the reflection information corresponding to the target pixel based on the direction vector of each calibrated light source and the pixel value of the target pixel in the corresponding light source image; generate a material image based on the reflection information corresponding to each target pixel, the material image representing the material information of the target object; and generate a surface geometry image based on the multiple light source images, the surface geometry image representing the depth information of the surface of the target object. The detection module is used to perform defect detection on the target object based on the color image, the material image, and the surface geometry image, and obtain the defect detection result of the target object.

13. The apparatus according to claim 12, characterized in that, The device further includes: The calibration module is used to acquire calibration images containing bright areas obtained by taking pictures of a sphere under illumination from different calibration light sources; determine the highlight position of the bright area and the center position of the sphere in each calibration image; and determine the normal vector of the sphere surface in each calibration image based on the preset reflection direction, the center position, and the highlight position; and determine the direction corresponding to each calibration light source based on each normal vector and the preset reflection direction.

14. The apparatus according to claim 13, characterized in that, The calibration module is further configured to determine the position of key points in the highlighted region of each calibration image; and to determine the highlight position corresponding to each highlighted region based on the position of key points in each highlighted region.

15. The apparatus according to claim 12, characterized in that, The material image includes a red channel material image, a green channel material image, and a blue channel material image; The generation module is further configured to determine the red channel value, green channel value, and blue channel value of the target pixel in the corresponding light source image; determine the reflection information of the target pixel in the red channel, green channel, and blue channel according to the direction vector of each calibrated light source and the red channel value, green channel value, and blue channel value corresponding to the target pixel; and generate a red channel material image, a green channel material image, and a blue channel material image respectively based on the reflection information of the red channel, green channel, and blue channel corresponding to each target pixel.

16. The apparatus according to claim 12, characterized in that, The generation module is further configured to determine the normal vector corresponding to the target pixel based on the reflection information, the direction vector of each of the calibration light sources and the pixel value of the target pixel in the corresponding light source image; and generate a surface geometric image based on the normal vector corresponding to each target pixel.

17. The apparatus according to claim 12, characterized in that, The detection module is further configured to extract features from the color image, the material image, and the surface geometry image respectively to obtain color features, material features, and geometric features; and to fuse the color features, the material features, and the geometric features to obtain fused features. Based on the fusion features, defect detection is performed on the target object to obtain the defect detection result of the target object.

18. The apparatus according to claim 17, characterized in that, The detection module is further configured to extract local features to be detected from the fused features, perform defect detection on the extracted local features, and obtain local detection results corresponding to the local features; return to the step of extracting local features to be detected from the fused features and continue execution until there are no local features in the fused features that have not been defect detected, and stop, to obtain local detection results corresponding to each local feature; and generate defect detection results for the target object based on each local detection result; the defect detection results include at least one of defect category, defect location, and defect area in the target object.

19. The apparatus according to claim 18, characterized in that, The detection module is further configured to perform defect category detection, defect location detection, and defect area detection based on the extracted local features, and obtain local detection results corresponding to the local features; the local detection results include the defect category, defect mask, and defect bounding box corresponding to the local features.

20. The apparatus according to claim 19, characterized in that, The detection module is further configured to generate an object mask containing the target object based on the defect mask corresponding to each of the local features; and to obtain the defect category corresponding to the defect existing in the target object, the defect location corresponding to each defect, and the defect area of ​​each defect according to the object mask, the defect category corresponding to each of the local features, and the defect bounding box.

21. The apparatus according to any one of claims 12 to 20, characterized in that, The target object is a metal injection molded device; the multi-light source image includes at least three light source images obtained by taking pictures of the metal injection molded device under at least three calibrated light sources in at least three different directions; the defect category in the defect detection result includes at least one of cracks, dents, material sticking, material shortage, bright marks, and dirt.

22. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

23. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

24. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images