Product detection method and device, computer equipment and computer readable storage medium

Abstract

The invention relates to a product detection method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring at least two target images respectively acquired under at least two illumination conditions for a to-be-detected area of a target product; performing luminosity synthesis based on the at least two target images to obtain a synthesized image; fusing the composite image and the at least two target images to obtain a fused image; detecting the fused image based on a pre-trained appearance defect detection model to obtain appearance defect detection information corresponding to the to-be-detected area; and obtaining a product detection result for the target product according to the appearance defect detection information corresponding to each to-be-detected area of the target product. According to the invention, the accuracy of product detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a product detection method, device, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In the field of product manufacturing, product appearance inspection is a key link to ensure product quality and consumer satisfaction. Traditional product appearance inspection methods mainly rely on manual visual inspection, but manual inspection is easily affected by the subjective factors of the inspectors, resulting in inconsistency and instability in the inspection results. Moreover, there are many types of product appearance defects, including obvious defects such as surface blemishes, scratches and damage, as well as subtle defects that are difficult to detect such as tiny bubbles, color differences, and dimensional deviations. For tiny or complex defects, manual inspection is often difficult to accurately identify and judge, resulting in low accuracy of current product inspections. Summary of the invention

[0003] Based on this, it is necessary to provide a product detection method, device, computer equipment, computer readable storage medium and computer program product to improve the accuracy of product detection in response to the above technical problems.

[0004] In a first aspect, the present application provides a product detection method, comprising:

[0005] Acquire at least two target images respectively acquired from the area to be inspected of the target product under at least two lighting conditions;

[0006] Perform photometric synthesis based on at least two target images to obtain a synthesized image;

[0007] fusing the synthesized image and at least two target images to obtain a fused image;

[0008] Based on the pre-trained appearance defect detection model, the fused image is tested to obtain the appearance defect detection information corresponding to the area to be tested;

[0009] According to the appearance defect detection information corresponding to each to-be-detected area of ​​the target product, the product detection result for the target product is obtained.

[0010] In a second aspect, the present application provides a product detection device, comprising:

[0011] A target image acquisition module, used to acquire at least two target images respectively acquired under at least two lighting conditions for the area to be inspected of the target product;

[0012] A photometric synthesis module, used for performing photometric synthesis based on at least two target images to obtain a synthesized image;

[0013] An image fusion module, used for fusing the synthesized image with at least two target images to obtain a fused image;

[0014] A model detection module is used to detect the fused image based on a pre-trained appearance defect detection model to obtain appearance defect detection information corresponding to the area to be detected;

[0015] The detection result acquisition module is used to obtain the product detection result for the target product according to the appearance defect detection information corresponding to each to-be-detected area of ​​the target product.

[0016] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0018] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0019] The above-mentioned product inspection method, device, computer equipment, computer-readable storage medium and computer program product, for multiple target images collected respectively under different lighting conditions, perform photometric synthesis on the multiple target images and then fuse them, and use a pre-trained appearance defect detection model to detect the obtained fused image, and obtain the product inspection result for the target product after traversing each area to be inspected. Multiple target images are collected under different lighting conditions for photometric synthesis, and inspection is performed based on the appearance defect detection model. It can adapt to various subtle appearance defects of the product, thereby improving the accuracy of product inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 An application environment diagram of a product detection method provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a process for a product detection method provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a process flow of regional detection processing in a product detection method provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of a visual imaging device in a product detection method provided in an embodiment of the present application;

[0024] Figure 5 A structural block diagram of a product detection device provided in an embodiment of the present application;

[0025] Figure 6 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0026] Figure 7 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0027] Figure 8 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] The product detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can send at least two target images captured by the camera to the server 104 through the communication network. The target image can be captured by the camera under at least two lighting conditions for the area to be inspected of the target product. After the server 104 obtains the target image through the communication network, the server 104 can perform photometric synthesis and then fuse the multiple target images, and perform detection on the obtained fused image through the pre-trained appearance defect detection model, and obtain the product detection result for the target product after traversing each area to be inspected. The server 104 can also feedback the product detection result to the terminal 102 so that the product detection result can be displayed in the terminal 102. In some embodiments, the product detection method can also be implemented independently by the terminal 102 or the server 104. For example, after the terminal 102 obtains the target image captured by the camera, the terminal 102 directly performs product detection. For another example, the target image captured by the camera can be stored in a data storage system, and the server 104 can directly obtain the target image from the data storage system for product detection processing.

[0030] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0031] like Figure 2 As shown, the embodiment of the present application provides a product detection method, which is applied to Figure 1 The terminal 102 or the server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of the terminal and the server. The method comprises the following steps:

[0032] S202: Acquire at least two target images of a target product's area to be inspected, respectively captured under at least two lighting conditions.

[0033] Among them, the target product is a product that needs to be inspected for appearance, and specifically, the product can be inspected for surface flaws, scratches and damage, or various appearance defects such as bubbles, color difference, and size deviation. The area to be inspected may include the area on the surface of the target product that needs to be inspected for appearance defects, and specifically may include the different outer surfaces of the target product. The target image includes at least two, which are respectively obtained by capturing the area to be inspected under different lighting conditions. The lighting condition can be achieved by adjusting the position or direction of the light source, that is, by adjusting different light source positions or light source directions to construct different lighting conditions, and capturing images of the area to be inspected under the constructed lighting conditions to obtain the corresponding target image.

[0034] Optionally, the computer device can obtain a target image captured for the area to be detected of the target product, and the number of target images includes at least two, which are respectively captured under at least two lighting conditions. In a specific application, the camera can be used to capture images of the area to be detected of the target product. Specifically, different lighting conditions can be constructed to capture images respectively, such as lighting light sources at different positions can be sequentially lit to capture images of the area to be detected of the target product, thereby obtaining different target images. In some embodiments, the target product may include different areas to be detected, and the area to be detected may include the outer surface of the target product.

[0035] Under the same lighting conditions, at least one target image can be captured for the area to be inspected of the target product, and at least two target images can be obtained by adjusting different lighting conditions to respectively capture images. In some embodiments, when multiple images are captured for the same area to be inspected under the same lighting conditions, they can be screened to obtain corresponding target images, so that the same area to be inspected corresponds to one target image under the same lighting conditions. In a specific application, multiple images are captured for the same area to be inspected under the same lighting conditions, and the computer device can perform image quality analysis on the multiple images to obtain image quality scores for each image, and screen out a target image from each image based on the image quality score.

[0036] S204: Perform photometric synthesis based on at least two target images to obtain a synthesized image.

[0037] Among them, photometric synthesis refers to the processing of image synthesis based on the photometric stereo synthesis algorithm for the target image. The photometric stereo synthesis algorithm is also called the photometric stereo fusion algorithm (Photometric Stereo Fusion Algorithm), which is a high dynamic range imaging (HDR) technology. Photometric stereo synthesis is based on the principle that light has different effects on the surface of an object at different angles. By using multiple light sources to illuminate the object at different angles and capturing the corresponding images, and then observing the brightness changes of each pixel under different lighting conditions, the normal direction of the pixel (the direction perpendicular to the surface of the object) can be calculated. Finally, using the normal information on the entire image, the three-dimensional shape of the object surface can be reconstructed or a three-dimensional image can be generated. The basic principle of photometric stereo synthesis is to use multiple light sources to illuminate the object from different angles, and then process the images under these different lighting conditions through a computer, so as to reconstruct the three-dimensional shape of the object or generate a stereo image with a wide dynamic range.

[0038] Exemplarily, with respect to the obtained multiple target images, the computer device may synthesize the respective target images based on a photometric stereo synthesis algorithm to obtain a synthesized image with a sense of depth and stereo.

[0039] S206: Fuse the synthesized image with at least two target images to obtain a fused image.

[0040] The fused image is obtained by fusing the synthetic image and the target image, and can be specifically obtained by splicing the target image and the synthetic image. For example, after the target image is photometrically synthesized to obtain the synthetic image, the computer device can fuse the synthetic image with each target image, and specifically splice the synthetic image with each target image to obtain the fused image. The fused image carries the information of the original collected target image, which can ensure the accuracy of appearance defect detection.

[0041] S208 . Perform detection on the fused image based on the pre-trained appearance defect detection model to obtain appearance defect detection information corresponding to the area to be detected.

[0042] Among them, the appearance defect detection model can be pre-trained to perform appearance detection on the input image and output the appearance defect detection result in the input image, such as appearance defect detection information describing the appearance defect. The appearance defect detection model can be trained based on the historical appearance defect detection data of the product category to which the target product belongs. The appearance defect detection model can be constructed based on various neural network algorithms, such as convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory network (LSTM), generative adversarial network (GAN), attention mechanism (Attention Mechanism), residual network (ResNet), deep reinforcement learning (DRL), and transformer network. At least one algorithm is constructed.

[0043] Exemplarily, the computer device can obtain a pre-trained appearance defect detection model, and the appearance defect detection model can be pre-trained according to actual needs. In a specific application, for the production line of the target product, the computer device can train a corresponding appearance defect detection model to perform appearance defect detection on the products produced by the same production line through the appearance defect detection model. In some embodiments, the appearance defect detection model can also be pre-trained based on the product category, that is, the corresponding appearance defect detection model can be pre-trained for each product category of products, so that the appearance defect detection model obtained by training can be used to perform appearance defect detection on the products of the same product category. The computer device can input the fused image into the appearance defect detection model, so that the appearance defect detection model can detect the input fused image, and output the appearance defect detection information corresponding to the area to be detected. The appearance defect detection information can specifically include but is not limited to information such as the material positioning results and defect detection results of each point to be detected in the area to be detected.

[0044] S210 , obtaining a product inspection result for the target product according to the appearance defect inspection information corresponding to each to-be-inspected area of ​​the target product.

[0045] Among them, the target product may include at least one area to be inspected, and for each area to be inspected, the computer device may perform appearance defect detection for each area to be inspected to obtain corresponding appearance defect detection information. After traversing each area to be inspected of the target product, the computer device may obtain the appearance defect detection information corresponding to each area to be inspected, and obtain the product inspection result for the target product based on each appearance defect detection information. In some embodiments, the computer device may perform statistics based on the appearance defect detection information corresponding to each area to be inspected, obtain the inspection information statistics result, and obtain the product inspection result for the target product based on the inspection information statistics result, and the product inspection result can be used to indicate the product quality of the target product.

[0046] It can be seen that in the embodiment of the present application, for multiple target images collected separately under different lighting conditions, the multiple target images are photometrically synthesized and then fused, and the obtained fused image is inspected by the pre-trained appearance defect detection model. After traversing each area to be inspected, the product inspection result for the target product is obtained. By collecting multiple target images under different lighting conditions for photometric synthesis and performing inspection based on the appearance defect detection model, it can adapt to various subtle appearance defects of the product, thereby improving the accuracy of product inspection.

[0047] In some embodiments, detection is performed on the fused image based on a pre-trained appearance defect detection model to obtain appearance defect detection information corresponding to the area to be detected, including: obtaining a pre-trained appearance defect detection model; inputting the fused image into the appearance defect detection model for detection, and obtaining appearance defect detection information corresponding to the area to be detected output by the appearance defect detection model.

[0048] Among them, the appearance defect detection model can be trained based on historical appearance detection data according to actual needs, and can be specifically trained based on different production lines or different product categories. Exemplarily, the computer device can obtain a pre-trained appearance defect detection model, such as querying the corresponding appearance defect detection model based on the product category to which the target product belongs, or querying the corresponding appearance defect detection model based on the production line corresponding to the target product. The computer device can input the fused image into the appearance defect detection model, so that the appearance defect detection model can detect the input fused image and output the appearance defect detection information corresponding to the area to be detected.

[0049] It can be seen that in this embodiment, the computer device can input the fused image into the pre-trained appearance defect detection model for detection, so as to perform appearance defect detection based on the appearance defect detection model, and can detect various subtle appearance defects that may exist in the target product, thereby improving the accuracy of product detection.

[0050] In some embodiments, based on the appearance defect detection information corresponding to each area to be detected of the target product, a product inspection result for the target product is obtained, including: performing appearance defect analysis on each appearance defect detection information corresponding to each area to be detected of the target product to obtain defect attribute information of the area to be detected corresponding to each appearance defect detection information; determining the area detection result of the area to be detected corresponding to each appearance defect detection information based on the defect attribute information and defect judgment conditions of the area to be detected corresponding to each appearance defect detection information; and obtaining the product inspection result for the target product based on the area detection results corresponding to each area to be detected of the target product.

[0051] Among them, the appearance defect detection information may include the positioning results of each point to be detected in the area to be detected, and the point to be detected is the characteristic position point that needs to be detected in the area to be detected. The defect attribute information is the attribute information obtained after the appearance defect analysis is performed based on the appearance defect detection information, and may specifically include but is not limited to at least one of the attribute information such as the category, size or regional affiliation of each defect. The defect judgment condition may include rules for appearance defect judgment to determine whether there is an appearance defect in the area to be detected. The defect judgment condition can be set according to actual needs, such as including a size threshold. Based on the size threshold, the appearance defects in the area to be detected can be classified to obtain the corresponding regional detection result. The regional detection result is the appearance defect detection result corresponding to the area to be detected. Each area to be detected can obtain its own corresponding appearance defect detection result respectively. By combining the regional detection results corresponding to each area to be detected, the product detection result for the target product as a whole can be obtained.

[0052] Optionally, the computer device can perform appearance defect analysis based on the appearance defect detection information corresponding to the area to be detected, such as performing size analysis based on the positioning results of each point to be detected in the appearance defect detection information to obtain the defect attribute information of the corresponding area to be detected. The computer device can traverse each appearance defect detection information, so as to obtain the defect attribute information of each area to be detected. In the process of traversing each appearance defect detection information, for the current appearance defect detection information, the computer device can obtain the pre-set defect judgment condition, and compare the defect attribute information with the defect judgment condition to determine the area detection result of the corresponding area to be detected. After the computer device traverses each appearance defect detection information, it can obtain the area detection result corresponding to each area to be detected, and the computer device can obtain the product detection result of the target product based on the detection results of each area. In a specific application, the computer device can perform statistics based on the detection results of each area to obtain a statistical result, and based on the statistical result, the product detection result for the target product as a whole can be obtained.

[0053] It can be seen that in this embodiment, the computer device traverses each appearance defect detection information to perform appearance defect analysis, and obtains the overall product detection result of the target product based on the area detection results corresponding to each area to be detected, so that a comprehensive detection can be performed on the target product to ensure the accuracy of product detection.

[0054] In some embodiments, Figure 3 As shown, the processing process of area detection, that is, determining the area detection result of the area to be detected corresponding to each appearance defect detection information according to the defect attribute information and defect judgment conditions of the area to be detected, includes the following steps:

[0055] S302: Obtain defect determination conditions associated with the points to be inspected in the area to be inspected and the target product.

[0056] The points to be detected include characteristic position points in the area to be detected that need to be detected for appearance defects, and the area to be detected may include at least one point to be detected. The defect judgment condition is used to perform appearance defect detection on the area to be detected, and the defect judgment condition can be associated with the points to be detected and the target product, that is, the defect judgment condition is pre-configured for the target product and the points to be detected in the area to be detected, and different types of target products or different points to be detected can be configured with different defect judgment conditions.

[0057] Exemplarily, the computer device may obtain pre-configured defect determination conditions, and specifically may obtain associated defect determination conditions based on the points to be detected in the area to be detected and the product type to which the target product belongs. In some embodiments, different defect determination conditions may be configured in advance for different product types and different points to be detected, and a defect determination condition table may be constructed. The computer device may query the defect determination condition table based on the product type to which the target product belongs and the points to be detected in the area to be detected to obtain associated defect determination conditions.

[0058] S304: Compare the defect attribute information of the to-be-detected area corresponding to each appearance defect detection information with the defect determination condition to obtain a comparison result.

[0059] The comparison result is the result of comparing the defect attribute information with the defect judgment condition, and based on the comparison result, the regional detection result of the area to be detected corresponding to the appearance defect detection information can be obtained. Exemplarily, the computer device can make judgments on the defect attribute information of the area to be detected corresponding to each appearance defect detection information based on the defect judgment condition, and specifically compare each defect attribute information with the defect judgment condition to obtain the comparison result of each appearance defect detection information.

[0060] S306: Based on the comparison result, determine the regional detection result of the to-be-detected area corresponding to each appearance defect detection information.

[0061] Optionally, the computer device may obtain a regional detection result based on the comparison result, and specifically, the comparison result may be used as a regional detection result of the area to be detected corresponding to the appearance defect detection information.

[0062] It can be seen that in this embodiment, the computer device can improve the specificity of defect judgment and thus improve the accuracy of product detection by comparing and judging the defect attribute information with the defect judgment conditions associated with the points to be detected in the area to be detected and the target product.

[0063] In some embodiments, obtaining at least two target images of the target product's area to be inspected under at least two lighting conditions, respectively, includes: configuring the product posture of the target product through a motion platform so that the target product's area to be inspected supports image acquisition; when the target product is in the target product posture, respectively acquiring images of the area to be inspected under at least two lighting conditions, to obtain at least two target images.

[0064] The motion platform is used to adjust the product posture of the target product. By adjusting the product posture of the target product, the area to be detected of the target product can be supported by the camera for image acquisition, that is, the camera can acquire images for different areas to be detected of the target product respectively. The computer device can control the motion platform to update the product posture of the target product, and the camera can acquire target images corresponding to different areas to be detected for target products with different product postures. The target product posture is the product posture of the target product at present, and the camera can acquire at least two target images for the target product under the target product posture.

[0065] Exemplarily, the computer device can configure the product posture of the target product through the motion platform, such as configuring the target product to be in the target product posture, so that the area to be detected of the target product supports the camera to perform image acquisition under the target product posture. When the target product is in the target product posture, the area to be detected of the target product can face the camera, so that the camera can perform image acquisition on the area to be detected of the target product. When the target product is in the target product posture, the computer device can adjust different lighting conditions to respectively perform image acquisition on the area to be detected of the target product under at least two lighting conditions, thereby obtaining at least two target images.

[0066] Furthermore, before obtaining the product inspection result for the target product based on the appearance defect detection information corresponding to each area to be inspected of the target product, the product inspection method also includes: adjusting the product posture of the target product through a motion platform to adjust the area to be inspected of the target product to support image acquisition; performing image acquisition based on the adjusted product posture to obtain at least two target images of each area to be inspected of the target product respectively captured under at least two lighting conditions.

[0067] Optionally, after the image acquisition for a certain area to be detected is completed, the computer device can control the motion platform to adjust the product posture of the target product, thereby adjusting the area to be detected that the target product supports for image acquisition, so as to acquire the target images corresponding to different areas to be detected. Specifically, the computer device can adjust the product posture of the target product through the motion platform, such as translating and rotating the target product to adjust the area to be detected that the target product supports for image acquisition. After the product posture of the target product is adjusted, the area to be detected facing the camera in the target product is also adjusted accordingly. The computer device can control the camera to perform image acquisition based on the adjusted product posture, and specifically can perform image acquisition for the adjusted area to be detected under at least two lighting conditions, so as to obtain the target image corresponding to the adjusted area to be detected. After traversing each product posture to realize the image acquisition of each area to be detected, the computer device can obtain at least two target images obtained by collecting each area to be detected of the target product under at least two lighting conditions.

[0068] It can be seen that in this embodiment, the computer device can adjust the product posture of the target product through the motion platform to realize image acquisition of each area to be inspected of the target product, thereby ensuring comprehensive inspection of each area to be inspected of the target product, which is conducive to improving the accuracy of product inspection.

[0069] In some embodiments, performing photometric synthesis based on at least two target images to obtain a synthesized image includes: performing photometric synthesis on at least two target images according to a photometric stereo algorithm to obtain a synthesized image.

[0070] The photometric stereo algorithm is based on the physical properties of light and geometric principles. It determines the brightness and shadow of the object surface by tracing the path of light and calculating the interaction between light and the surface of the object, thereby enhancing the rendering effect or achieving three-dimensional reconstruction. Optionally, the computer device can perform photometric synthesis on each target image based on the photometric stereo algorithm to obtain a synthesized image.

[0071] It can be seen that in this embodiment, the target image is photometrically synthesized based on the photometric stereo algorithm, and the obtained synthesized image can carry depth information, thereby enabling accurate detection of the target product and improving the accuracy of product detection.

[0072] In some embodiments, before obtaining the product inspection result for the target product based on the appearance defect detection information corresponding to each area to be inspected of the target product, the product inspection method also includes: obtaining historical product inspection data for the product type to which the target product belongs; based on the historical product inspection data, statistics are collected on the historical appearance defect information of each area of ​​the product in the product type, and an area of ​​interest of the target product is determined based on the obtained statistical results; and each area to be inspected of the target product is determined based on the area of ​​interest.

[0073] Among them, historical product inspection data refers to the data that has been inspected in the past, which can be divided according to different product types, that is, different types of products can be summarized with different historical product inspection data. The region of interest is the area in each area of ​​the target product that needs to be inspected for appearance defects, such as the area that is prone to appearance defects.

[0074] Exemplarily, the computer device can determine the product type to which the target product belongs, and query the corresponding historical product detection data based on the product type. The historical product detection data may include the historical product detection results of each product in the product type. The computer device can count the historical appearance defect information of each area of ​​each product of the same product type as the target product, and specifically count the number and type of defects in each area of ​​each product. The computer device can determine the area of ​​interest of the target product based on the statistical results, such as the computer device can determine the area where the number of defects exceeds the number threshold as the area of ​​interest, and for example, the area where the target type defects appear can be determined as the area of ​​interest. The computer device can obtain the various areas to be detected of the target product based on the area of ​​interest, and specifically can directly use the area of ​​interest as the various areas to be detected of the target product. In some embodiments, the computer device can also perform preprocessing on the area of ​​interest, such as after preprocessing such as screening and fusion, to obtain the various areas to be detected of the target product.

[0075] It can be seen that in this embodiment, the computer device determines the area of ​​interest of the target product based on the historical product inspection data of the product type to which the target product belongs, and determines the various areas to be inspected of the target product based on the area of ​​interest, so that the various areas to be inspected that need to be inspected can be determined based on the historical product inspection data, which can improve the targetedness of appearance inspection, thereby improving the accuracy of product inspection.

[0076] The present application also provides an application scenario, which applies the above-mentioned product detection method. Specifically, the application of the product detection method in this application scenario is as follows:

[0077] The product inspection method provided in this embodiment can be applied to the appearance inspection of electroplated injection molded parts. At present, factories usually use manual inspection, which has low inspection efficiency and accuracy. The product inspection method provided in this embodiment uses an optical imaging module and a multi-dimensional five-axis motion platform to simulate the manual inspection process and achieve 360° comprehensive inspection of the product.

[0078] Specifically, the current inspection of electroplated injection molded parts is mainly based on manual inspection, which has low inspection efficiency and unstable inspection accuracy, and there is no mature automated inspection equipment for implementation. Based on this, the product inspection method provided in this embodiment is based on photometric stereo technology combined with a five-axis motion platform to simulate the manual inspection process and achieve 360° appearance inspection.

[0079] Specifically, Figure 4 As shown, the visual imaging device of the product inspection method provided in this embodiment is composed of an industrial camera, a lens, and a light source. The technical parameters of the industrial camera can be confirmed by factors such as the inspection accuracy and inspection efficiency of the inspected product; the technical parameters of the lens are confirmed by the space requirements and imaging accuracy requirements of the equipment; the light source is determined by the principle of the imaging solution and the technical solution. Different product postures can be adjusted for the inspected product through the motion platform, so that different areas to be inspected can be obtained through camera shooting for appearance defect detection. For the imaging system, the camera can select a 1200W (million) pixel high-speed camera and a USB (Universal Serial Bus) 3.0 interface to meet the needs of efficient inspection; the lens selects an industrial lens with a focal length of 50mm (millimeter); the imaging accuracy is 15μm (micrometer); the light source uses a coaxial light source and a combined strip light source.

[0080] Furthermore, 2.5D (Diamends) photometric stereo is a technology that combines photometric stereo and two-dimensional image processing, aiming to recover the depth information of objects from images taken under multiple lighting conditions. Compared with traditional 3D reconstruction methods, 2.5D photometric stereo generates two-dimensional images with depth information, namely "2.5D" images. It focuses on simplifying the complexity of three-dimensional reconstruction while retaining the surface shape details of the object.

[0081] In this embodiment, the combined bar light source is composed of four independent bar light sources. During the imaging process of the product to be inspected, the four bar light sources are lit in sequence to obtain four pictures with four light sources lit separately. The image effect is then synthesized and processed through a photometric stereo algorithm before defect detection is performed.

[0082] Furthermore, the motion platform can include a five-axis motion device, based on which the camera & light source can have motion control in the Z-axis and X-direction; based on which the product & fixture can have motion control in the Y-axis movement, theta-axis rotation (rotation around the Z-axis), and A-axis rotation (rotation around the X-axis). Through the above five-dimensional motion of the five-axis motion device, the posture of the product can be automatically adjusted, and the process of manual inspection can be simulated, thereby realizing comprehensive appearance defect detection for the product.

[0083] In the specific inspection process, the inspection area of ​​the inspected product is divided into different inspection points, and the imaging module at each point collects the corresponding image; through the five-axis motion platform, different inspection areas are photographed to achieve defect detection of all appearance surfaces of the inspected product.

[0084] Specifically, the deep learning image segmentation algorithm framework can be used to detect surface defects of products. Specifically, the original target image and the synthetic image are spliced ​​to obtain a fused image; then the fused image is input into the tuned algorithm model to obtain the inference result. Furthermore, the inference result can include the current point material positioning result and the defect detection result. According to the output result, the dimensions such as defect size are analyzed, and the relevant attributes such as category / size / region affiliation of each defect are recorded; then for convex / concave defects, the gradient change is calculated in the photometric synthesis image as the attribute of the defect severity; finally, according to the various attributes of the defect, the acceptance (Accept) inspection is carried out under the corresponding rules to obtain the OK (qualified) or NG (unqualified) judgment result.

[0085] The detection accuracy and stability of the product detection method provided in this embodiment are better than manual detection capabilities, and have certain applicability and can be applied to other similar industrial detection scenarios. In some embodiments, the five-axis platform can be replaced by a manipulator that can rotate and move the detected product in multiple dimensions.

[0086] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0087] Based on the same inventive concept, the embodiment of the present application also provides a product detection device. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more product detection device embodiments provided below can refer to the limitations on the product detection method above, and will not be repeated here.

[0088] like Figure 5 As shown, the embodiment of the present application provides a product detection device 500, including:

[0089] A target image acquisition module 502 is used to acquire at least two target images respectively acquired from the area to be inspected of the target product under at least two lighting conditions;

[0090] A photometric synthesis module 504, configured to perform photometric synthesis based on at least two target images to obtain a synthesized image;

[0091] An image fusion module 506, used for fusing the synthesized image with at least two target images to obtain a fused image;

[0092] The model detection module 508 is used to detect the fused image based on the pre-trained appearance defect detection model to obtain appearance defect detection information corresponding to the area to be detected;

[0093] The detection result obtaining module 510 is used to obtain the product detection result for the target product according to the appearance defect detection information corresponding to each to-be-detected area of ​​the target product.

[0094] In some embodiments, in terms of performing detection on the fused image based on a pre-trained appearance defect detection model to obtain appearance defect detection information corresponding to the area to be detected, the model detection module 508 is specifically used to: obtain a pre-trained appearance defect detection model; input the fused image into the appearance defect detection model for detection, and obtain the appearance defect detection information corresponding to the area to be detected output by the appearance defect detection model.

[0095] In some embodiments, in terms of obtaining product inspection results for a target product based on appearance defect inspection information corresponding to each area to be inspected of the target product, the inspection result acquisition module 510 is specifically used to: perform appearance defect analysis on each appearance defect inspection information corresponding to each area to be inspected of the target product, and obtain defect attribute information of the area to be inspected corresponding to each appearance defect inspection information; determine area inspection results of the area to be inspected corresponding to each appearance defect inspection information based on the defect attribute information and defect judgment conditions of the area to be inspected corresponding to each appearance defect inspection information; and obtain product inspection results for the target product based on the area inspection results corresponding to each area to be inspected of the target product.

[0096] In some embodiments, in terms of determining the regional detection result of the area to be detected corresponding to each appearance defect detection information based on the defect attribute information and the defect judgment condition of the area to be detected corresponding to each appearance defect detection information, the detection result acquisition module 510 is specifically used to: obtain the defect judgment condition associated with the point to be detected in the area to be detected and the target product; compare the defect attribute information of the area to be detected corresponding to each appearance defect detection information with the defect judgment condition to obtain a comparison result; based on the comparison result, determine the regional detection result of the area to be detected corresponding to each appearance defect detection information.

[0097] In some embodiments, in terms of acquiring at least two target images respectively captured under at least two lighting conditions for the area to be inspected of the target product, the target image acquisition module 502 is specifically used to: configure the product posture of the target product through a motion platform so that the area to be inspected of the target product supports image acquisition; when the target product is in the target product posture, perform image acquisition on the area to be inspected under at least two lighting conditions to obtain at least two target images; adjust the product posture of the target product through the motion platform to adjust the area to be inspected of the target product that supports image acquisition; perform image acquisition based on the adjusted product posture to obtain at least two target images respectively captured under at least two lighting conditions for each area to be inspected of the target product.

[0098] In some embodiments, in terms of performing photometric synthesis based on at least two target images to obtain a synthesized image, the photometric synthesis module 504 is specifically used to perform photometric synthesis on at least two target images according to a photometric stereo algorithm to obtain a synthesized image.

[0099] In some embodiments, the product inspection device 500 also includes a module for determining an area to be inspected, which is specifically used to: obtain historical product inspection data for the product type to which the target product belongs; based on the historical product inspection data, perform statistics on historical appearance defect information of each area of ​​the product in the product type, and determine the area of ​​interest of the target product based on the obtained statistical results; determine each area to be inspected of the target product based on the area of ​​interest.

[0100] Each module in the above-mentioned product detection device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0101] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store various data involved in the product detection method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned product detection method are implemented.

[0102] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned product detection method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0103] Those skilled in the art will understand that Figure 6 or Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0104] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0105] In some embodiments, Figure 8 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0106] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0107] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0108] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0109] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A product detection method, characterized in that: include: Acquire at least two target images respectively acquired from the area to be inspected of the target product under at least two lighting conditions; Perform photometric synthesis based on the at least two target images to obtain a synthesized image; fusing the synthesized image and the at least two target images to obtain a fused image; Performing detection on the fused image based on a pre-trained appearance defect detection model to obtain appearance defect detection information corresponding to the area to be detected; A product inspection result for the target product is obtained based on the appearance defect inspection information corresponding to each to-be-inspected area of ​​the target product.

2. The method according to claim 1, characterized in that: The pre-trained appearance defect detection model performs detection on the fused image to obtain appearance defect detection information corresponding to the area to be detected, including: Obtain a pre-trained appearance defect detection model; The fused image is input into the appearance defect detection model for detection, and appearance defect detection information corresponding to the area to be detected output by the appearance defect detection model is obtained.

3. The method according to claim 1, characterized in that The obtaining of a product inspection result for the target product according to the appearance defect inspection information corresponding to each to-be-inspected area of ​​the target product includes: Performing appearance defect analysis on each piece of appearance defect detection information corresponding to each to-be-detected area of ​​the target product to obtain defect attribute information of the to-be-detected area corresponding to each piece of appearance defect detection information; Determine the regional detection result of the area to be detected corresponding to each appearance defect detection information according to the defect attribute information and defect judgment conditions of the area to be detected corresponding to each appearance defect detection information; A product detection result for the target product is obtained according to the area detection results corresponding to each to-be-detected area of ​​the target product.

4. The method according to claim 3, characterized in that The step of determining the regional detection result of the area to be detected corresponding to each appearance defect detection information according to the defect attribute information and the defect determination condition of the area to be detected corresponding to each appearance defect detection information comprises: Acquiring defect determination conditions associated with the points to be inspected in the area to be inspected and the target product; Compare the defect attribute information of the to-be-detected area corresponding to each appearance defect detection information with the defect determination condition to obtain a comparison result; Based on the comparison result, a regional detection result of the area to be detected corresponding to each appearance defect detection information is determined.

5. The method according to claim 1, characterized in that The step of acquiring at least two target images respectively acquired from the area to be inspected of the target product under at least two lighting conditions includes: Configuring the product posture of the target product through the motion platform so that the area to be inspected of the target product supports image acquisition; When the target product is in a target product posture, performing image acquisition on the area to be detected under at least two lighting conditions respectively to obtain at least two target images; Before obtaining a product inspection result for the target product according to the appearance defect inspection information corresponding to each to-be-inspected area of ​​the target product, the method further includes: Adjusting the product posture of the target product by the motion platform to adjust the area to be detected of the target product supporting image acquisition; Based on the adjusted product posture, image acquisition is performed to obtain at least two target images of each to-be-detected area of ​​the target product respectively acquired under at least two lighting conditions.

6. The method according to claim 1, characterized in that The step of performing photometric synthesis based on the at least two target images to obtain a synthesized image includes: The at least two target images are photometrically synthesized according to a photometric stereo algorithm to obtain a synthesized image.

7. The method according to any one of claims 1 to 6, characterized in that: Before obtaining the product inspection result for the target product according to the appearance defect inspection information corresponding to each to-be-inspected area of ​​the target product, the method further includes: Obtain historical product testing data for the product type to which the target product belongs; Based on the historical product inspection data, statistics are collected on the historical appearance defect information of each area of ​​the product of the product type, and the area of ​​interest of the target product is determined according to the obtained statistical results; The various regions to be inspected of the target product are determined based on the region of interest.

8. A product detection device, characterized in that: include: A target image acquisition module, used to acquire at least two target images respectively acquired under at least two lighting conditions for the area to be inspected of the target product; A photometric synthesis module, used for performing photometric synthesis based on the at least two target images to obtain a synthesized image; An image fusion module, used for fusing the composite image with the at least two target images to obtain a fused image; A model detection module, used to detect the fused image based on a pre-trained appearance defect detection model to obtain appearance defect detection information corresponding to the area to be detected; The detection result obtaining module is used to obtain the product detection result for the target product according to the appearance defect detection information corresponding to each to-be-detected area of ​​the target product.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images