A product detection method, device, medium and equipment based on machine vision

Abstract

Embodiments of the present application disclose a product detection method, device, medium, and equipment based on machine vision, which relate to the field of image processing technology. The present application captures images from directly above the PCB panel for recognition to ensure that the camera's field of view can evenly and completely cover the PCB panel. Since the standard sample images are obtained under different lighting conditions, the model can be adapted to recognition under different lighting conditions. Moreover, each standard sample image is obtained by fusing images taken from multiple different directions with the design drawing of the standard PCB panel, thus transferring the normal feature representations of the parts on the standard PCB panel that are affected by environmental light to the standard design drawing. Furthermore, using the fused images as training samples enables the model to not only recognize defect features but also recognize features affected by specular interference under different conditions, thereby avoiding misjudgment and missed judgment of defect features and effectively improving the quality of defect detection for the PCB panel.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to a product detection method, device, medium and equipment based on machine vision. Background Art

[0002] Machine vision is a rapidly developing branch of artificial intelligence. Briefly speaking, machine vision is to use a machine to replace the human eye for measurement and judgment. A machine vision system converts the captured target into an image signal through a machine vision product (i.e., an image acquisition device, divided into CMOS and CCD types), transmits it to a dedicated image processing system, obtains the morphological information of the captured target, and converts it into a digital signal according to information such as pixel distribution, brightness, and color. The image system performs various operations on these signals to extract the features of the target, and then controls the actions of on-site equipment according to the discrimination results.

[0003] Product detection based on machine vision is easily affected by environmental light. For example, in the visual inspection of a PCB panel, reflective light will be generated on the exposed metal surface, smooth coating or solder joints on the PCB surface, and some areas treated with special processes such as gold plating and polishing. If the electronic components on the PCB are relatively large, a shadow may also be formed on one side, which will all lead to a decline in the quality of defect detection of the PCB panel. Summary of the Invention

[0004] The main purpose of the present application is to provide a product detection method, device, medium and equipment based on machine vision, aiming to solve the problem of low quality of defect detection of PCB panels based on machine vision in the prior art.

[0005] To achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a product detection method based on machine vision, including the following steps:

[0007] Obtain the original image of the PCB panel; wherein, the original image is taken from directly above the PCB panel;

[0008] Input the original image into a defect detection model to obtain a detection image; wherein, the defect detection model is trained based on sample images, and the sample images include a number of defect sample images and standard sample images. The number of standard sample images are obtained under different illumination conditions respectively, and each standard sample image is obtained by fusing the image of the standard PCB panel taken from multiple different directions under the same illumination condition with the design drawing of the standard PCB panel.

[0009] In a possible implementation manner of the first aspect, before inputting the original image into the defect detection model to obtain a detection image, the method further includes:

[0010] Under different lighting conditions, the standard PCB panel is photographed from multiple different directions respectively to obtain a number of photographed images;

[0011] The photographed images under the same lighting condition are fused with the design drawing of the standard PCB panel to obtain a number of standard sample images;

[0012] Based on a number of defective sample images and standard sample images for training, a defect detection model is obtained.

[0013] In a possible implementation manner of the first aspect, fusing the photographed images under the same lighting condition with the design drawing of the standard PCB panel to obtain a number of standard sample images includes:

[0014] Stretch the photographed images under the same lighting condition according to the shooting direction and make the size of the photographed images match that of the standard PCB panel to obtain the first photographed image;

[0015] Perform primary fusion on the first photographed images under the same lighting condition to obtain a fused image;

[0016] Perform secondary fusion on the fused image and the design drawing of the standard PCB panel to obtain a number of standard sample images.

[0017] In a possible implementation manner of the first aspect, stretching the photographed images under the same lighting condition according to the shooting direction includes:

[0018] Determine the stretching direction according to the shooting direction;

[0019] Determine the stretching gradient and stretching interval according to the positional relationship between the shooting camera and the standard PCB panel;

[0020] Stretch the photographed images under the same lighting condition along the stretching direction according to the stretching gradient and stretching interval.

[0021] In a possible implementation manner of the first aspect, after photographing the standard PCB panel from multiple different directions respectively under different lighting conditions to obtain a number of photographed images, the method further includes:

[0022] Draw the region of interest according to the design drawing of the standard PCB panel;

[0023] Perform threshold segmentation on the photographed images to mark the reflective regions in the region of interest to obtain a marked image;

[0024] Restore the reflective regions based on a number of marked images under the same lighting condition to obtain the first photographed image;

[0025] Fuse the captured images under the same lighting condition with the design drawing of the standard PCB panel to obtain a number of standard sample images, including:

[0026] Fuse the first captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a number of standard sample images.

[0027] In a possible implementation manner of the first aspect, based on a number of marked images under the same lighting condition, restore the reflective area to obtain the first captured image, including:

[0028] Segment based on a number of marked images under the same lighting condition to eliminate the reflective area and obtain the first marked image;

[0029] Respectively take one first marked image as the target image and the remaining first marked images as non-target images, and according to the regions of interest on the non-target images, restore the reflective area on the target image to obtain the first captured image.

[0030] In a possible implementation manner of the first aspect, after respectively taking one first marked image as the target image and the remaining first marked images as non-target images, and according to the regions of interest on the non-target images, restoring the reflective area on the target image to obtain the first captured image, the method further includes:

[0031] According to the pixel distribution of the regions of interest on the first captured image, adjust the pixel values of the regions of interest to obtain the second captured image; wherein, the adjustment of the pixel values includes: increasing or decreasing the pixel values and complementing the pixel values;

[0032] Fuse the first captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a number of standard sample images, including:

[0033] Fuse the second captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a number of standard sample images.

[0034] In a second aspect, an embodiment of the present application provides a product detection device based on machine vision, including:

[0035] An acquisition module, configured to acquire the original image of the PCB panel; wherein, the original image is acquired from directly above the PCB panel;

[0036] A detection module for inputting an original image into a defect detection model to obtain a detected image. The defect detection model is trained based on sample images, which include a number of defective sample images and standard sample images. The standard sample images are obtained under different illumination conditions, and each standard sample image is obtained by fusing an image of a standard PCB panel taken from multiple different directions under the same illumination condition with the design drawing of the standard PCB panel.

[0037] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which when loaded and executed by a processor, implements the machine vision-based product detection method provided in any one of the first aspects above.

[0038] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory.

[0039] The memory is used to store a computer program.

[0040] The processor is used to load and execute the computer program so that the electronic device executes the machine vision-based product detection method provided in any one of the first aspects above.

[0041] Compared with the prior art, the beneficial effects of the present application are:

[0042] A product detection method, device, medium, and equipment based on machine vision proposed in an embodiment of the present application. The method includes: obtaining an original image of a PCB panel, where the original image is taken from directly above the PCB panel; inputting the original image into a defect detection model to obtain a detection image, where the defect detection model is trained based on sample images, and the sample images include a number of defect sample images and standard sample images. The number of standard sample images are obtained under different lighting conditions respectively, and each standard sample image is obtained by fusing the images of the standard PCB panel taken from multiple different directions under the same lighting condition with the design drawing of the standard PCB panel. The present application takes images from directly above the PCB panel for recognition to ensure that the camera's field of view can evenly and completely cover the PCB panel. By inputting the taken original image into the defect detection model for recognition, a detection image is obtained to complete the detection of defects. Since the sample images for training the defect detection model include defect sample images and standard sample images, the model can learn the differences between the defect sample images and the standard sample images, thereby realizing the recognition of defects. And the standard sample images are obtained under different lighting conditions, enabling the model to adapt to recognition under different lighting conditions. Moreover, each standard sample image is obtained by fusing the images of the standard PCB panel taken from multiple different directions with the design drawing of the standard PCB panel, which transfers the normal feature representations of the parts of the standard PCB panel that are affected by environmental lighting to the standard design drawing. Then, using the fused images as training samples, the model can not only recognize defect features but also recognize features affected by specular interference under different conditions, thus avoiding misjudgment and missed judgment of defect features and effectively improving the quality of defect detection for the PCB panel. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic structural diagram of an electronic device for the hardware operating environment involved in an embodiment of the present application;

[0044] Figure 2 It is a schematic flowchart of a product detection method based on machine vision provided by an embodiment of the present application;

[0045] Figure 3 It is a schematic module diagram of a product detection device based on machine vision provided by an embodiment of the present application;

[0046] Reference numerals in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] Refer to the attached Figure 1 attachmentFigure 1 The following is a schematic diagram of the electronic device structure of the hardware operating environment involved in the solution of the embodiment of this application. The electronic device may include: a processor 101, such as a Central Processing Unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 104 may further include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed Random Access Memory (RAM) or a stable Non-Volatile Memory (NVM), such as at least one disk memory; the processor 101 may be a general-purpose processor, including a central processor, a network processor, etc., or may also be a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0049] Those skilled in the art can understand that the structure shown in the appendix Figure 1 does not constitute a limitation on the electronic device, and it may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0050] As shown in the appendix Figure 1 the memory 105, as a storage medium, may include an operating system, a network communication module, a user interface module, and a product detection device based on machine vision.

[0051] In the electronic device shown in the appendix Figure 1 the network interface 103 is mainly used for data communication with a network server; the user interface 104 is mainly used for data interaction with users; in this application, the processor 101 and the memory 105 may be set in the electronic device. The electronic device calls the product detection device based on machine vision stored in the memory 105 through the processor 101 and executes the product detection method based on machine vision provided by the embodiment of this application.

[0052] Referring to the appendix Figure 2 based on the hardware device of the foregoing embodiment, the embodiment of this application provides a product detection method based on machine vision, including the following steps:

[0053] S10: Obtain the original image of the PCB panel; wherein, the original image is obtained by taking a photo directly above the PCB panel.

[0054] In the specific implementation process, the product detection of this application is aimed at the PCB panel. The Chinese name of PCB is printed circuit board, also known as printed wiring board. It is an important electronic component, a support for electronic components, and a carrier for the electrical interconnection of electronic components. Since the PCB panel is thin and sheet-shaped, taking a photo directly above it to obtain the original image can ensure as much as possible the uniform coverage of the PCB panel by the camera's field of view.

[0055] S20: Input the original image into the defect detection model to obtain a detection image; wherein, the defect detection model is obtained by training based on sample images. The sample images include a number of defect sample images and standard sample images. The number of standard sample images are obtained under different lighting conditions respectively. Each standard sample image is obtained by fusing the images of the standard PCB panel taken from multiple different directions under the same lighting condition with the design drawing of the standard PCB panel.

[0056] In the specific implementation process, the trained defect detection model is used to quickly and accurately detect the original image. The trained sample images include defect sample images and standard sample images. Through training, the model learns the differences between the defect sample images and the standard sample images, enabling it to identify the location of the defect on the image, complete the defect detection, and output the detection image. To adapt to different lighting conditions, the standard sample images prepared in the training stage are obtained under different lighting conditions. In the real scene, the light source itself may come from different directions, such as natural light, artificial light, etc. Therefore, the lighting condition here refers to the intensity of the light. After adapting to different lighting intensities, in order to let the model learn the characteristic manifestations brought by the lighting conditions, the images taken from multiple different directions are fused with the design drawing of the standard PCB panel. The basic principle of defect detection is to compare the image with a standard defect-free image, so that the defect can be separated from the background. After fusing the images, the normal features such as reflection and shadow shown on the taken image due to the influence of light are transferred to the standard image, and when these features appear again on the original image, they will not be recognized as defect features by the model.

[0057] In this embodiment, an image is captured from directly above the PCB panel for recognition to ensure that the camera's field of view can evenly and completely cover the PCB panel. The captured original image is input into a defect detection model for recognition to obtain a detection image to complete the defect detection. Since the sample images for training the defect detection model include defect sample images and standard sample images, the model can learn the differences between the defect sample images and the standard sample images, thereby realizing defect recognition. Moreover, the standard sample images are obtained under different lighting conditions, enabling the model to adapt to recognition under different lighting conditions. Each standard sample image is obtained by fusing images of the standard PCB panel taken from multiple different directions with the design drawing of the standard PCB panel, thus transferring the normal feature representations of the parts of the standard PCB panel that are affected by environmental light to the standard design drawing. Furthermore, using the fused images as training samples allows the model to not only recognize defect features but also recognize features affected by reflection interference under different conditions, thereby avoiding misjudgment and missed judgment of defect features and effectively improving the quality of defect detection for the PCB panel.

[0058] In one embodiment, before inputting the original image into the defect detection model to obtain a detection image, the method further includes:

[0059] Under different lighting conditions, the standard PCB panel is photographed from multiple different directions to obtain a number of captured images;

[0060] Fuse the captured images under the same lighting condition with the design drawing of the standard PCB panel to obtain a number of standard sample images;

[0061] Train based on a number of defect sample images and standard sample images to obtain a defect detection model.

[0062] In the specific implementation process, during the training stage of the model, first, the standard PCB panel (i.e., the PCB panel without defects) is photographed under different lighting conditions. The shooting directions are preferably 2 - 8 directions, and multiple directions are distributed as symmetrically as possible around the center or axis, such as a triangular distribution at the three vertices of an equilateral triangle, or a cross-shaped or star-shaped distribution, so that the influence of the direction on the captured image can evenly cover the entire product. Considering that there are deviations between the images captured from different directions and the overall board image to be compared, it is necessary to fuse the captured images. The design drawing of the standard PCB panel is introduced as a basis to fuse it with the captured images into an image with the same perspective from directly above as the original image.

[0063] In one embodiment, fusing the captured images under the same lighting condition with the design drawing of the standard PCB panel to obtain a number of standard sample images includes:

[0064] Stretch the captured images under the same lighting condition according to the shooting direction, and make the size of the captured images match that of the standard PCB panel to obtain the first captured image;

[0065] Perform primary fusion on the first captured images under the same lighting condition to obtain a fused image;

[0066] Perform secondary fusion on the fused image and the design drawing of the standard PCB panel to obtain a number of standard sample images.

[0067] In the specific implementation process, since the standard PCB panel is photographed from multiple different directions, the captured images are stretched and tilted relative to the overall board image in the front view direction. Therefore, in the stage of fusing the images, stretching reduction should be carried out first to make the size of the captured images match that of the standard PCB panel. The image after size matching is recorded as the first captured image. In the fusion stage, first fuse the first captured images under the same lighting condition to fuse the characteristic manifestations of the different effects brought by the lighting in different shooting directions, and then fuse the fused image with the design drawing of the standard PCB panel again to transfer these characteristic manifestations to the standard design drawing, so that these characteristic manifestations are present on the obtained standard sample images. These characteristic manifestations are brought by the lighting but do not belong to the defect characteristics. Cooperating with the learning of the defect samples enables the model to effectively distinguish the defect characteristics and the influence characteristics brought by the lighting.

[0068] In one embodiment, stretching the captured images under the same lighting condition according to the shooting direction includes:

[0069] Determine the stretching direction according to the shooting direction;

[0070] Determine the stretching gradient and stretching interval according to the position relationship between the shooting camera and the standard PCB panel;

[0071] Stretch the captured images under the same lighting condition along the stretching direction according to the stretching gradient and stretching interval.

[0072] In the specific implementation process, since the camera shooting has the characteristic that distant objects appear smaller and near objects appear larger, a single linear stretching method cannot be used during stretching and restoration. The embodiment of the present application provides a gradient stretching method, that is, different stretching ratios are used for different regions. First, the stretching direction needs to be determined. The stretching inclination of the image is caused by different shooting directions. Therefore, the stretching direction is determined based on the shooting direction. The stretching ratio is adjusted according to the camera shooting characteristics. First, the positional relationship between the shooting camera and the PCB panel to be shot is determined, and then the stretching gradient and stretching interval are determined according to the positional relationship, that is, the captured image is divided into multiple segments along the shooting direction, that is, different stretching intervals. According to the characteristic that distant objects appear smaller and near objects appear larger in the camera-captured image, the stretching interval closer to the camera requires a larger stretching gradient, and vice versa. In this way, each part of the image can be stretched to match its imaging characteristics, and the captured image can be restored more accurately, thereby improving the quality of the fused image.

[0073] In one embodiment, after capturing a standard PCB panel from multiple different directions under different lighting conditions to obtain a number of captured images, the method further includes:

[0074] Drawing a region of interest according to the design drawing of the standard PCB panel;

[0075] Performing threshold segmentation on the captured image to mark the reflective region in the region of interest and obtain a marked image;

[0076] Restoring the reflective region based on a number of marked images under the same lighting condition to obtain a first captured image.

[0077] In the specific implementation process, after migrating the characteristics affected by lighting to the standard image, even if there is the influence of shadows, during the comparison process, if there are defects in the shadow part, it can also be determined whether there are defects according to the gray value expression and pixel value distribution of the shadow part. For the highlighted reflective part, although the possibility of overlapping with the defective area is reduced, once this situation occurs, it will inevitably lead to missed detection of defects. Therefore, the reflective region is removed and restored by the above means. Specifically, first, a region of interest is drawn on the captured image according to the design drawing of the standard PCB panel, that is, according to the design drawing of the standard PCB panel, it is determined which parts will have reflections, such as exposed metal surfaces, smooth coatings or solder joints, and some areas treated with special processes such as gold plating and polishing. These areas are marked as regions of interest by the method of active drawing.

[0078] The gray value of the reflective highlight part after grayscale conversion is significantly different from that of the surrounding area. Therefore, the reflective area is marked in the region of interest through threshold segmentation. It should be noted that the appearance of reflection is always based on the component and will not affect the area outside the component, that is, the marking of the reflective area is always carried out within the region of interest. And the reflection is affected by the direction. The reflective areas generated by light in different directions on the component are different. The area that has reflection in a certain image may appear normal in the image taken from another shooting direction. Therefore, the marked images under the same lighting condition can be comprehensively used to restore the reflective area, and the restored image is the first captured image.

[0079] Based on the foregoing steps, the captured images under the same lighting condition are fused with the design drawing of the standard PCB panel to obtain a number of standard sample images, including:

[0080] The first captured image under the same lighting condition is fused with the design drawing of the standard PCB panel to obtain a number of standard sample images.

[0081] In one embodiment, based on a number of marked images under the same lighting condition, the reflective area is restored to obtain the first captured image, including:

[0082] Segment the number of marked images under the same lighting condition to remove the reflective area and obtain the first marked image;

[0083] Taking one first marked image as the target image and the remaining first marked images as non-target images respectively, according to the region of interest on the non-target images, restore the reflective area on the target image to obtain the first captured image.

[0084] In the specific implementation process, during the specific restoration process, first, according to the markings, the reflective areas on their respective marked images are removed, and the image after removal is denoted as the first marked image. Then, in order to improve the quality of the restored image, instead of taking a certain image as a fixed standard, a multi-image restoration method is adopted. Taking each first marked image as the target image respectively, and the remaining first marked images as non-target images, which are used to restore the missing part of the reflective area on the target image. In this way, the first captured images restored under multiple standards can more accurately restore the original information of the reflective area on the image after fusion. If there are defects in the reflective area at the same time, then the defects will be retained on some images that have not been removed, and thus the defect information can also be restored after fusion, thereby avoiding the missed detection of defects in the reflective part by the model.

[0085] In one embodiment, after taking one first marked image as the target image and the remaining first marked images as non-target images respectively, according to the region of interest on the non-target images, restoring the reflective area on the target image to obtain the first captured image, the method further includes:

[0086] According to the pixel distribution of the region of interest on the first captured image, adjust the pixel values of the region of interest to obtain a second captured image; wherein, the adjustment of the pixel values includes: increasing or decreasing the pixel values and complementing the pixel values.

[0087] In the specific implementation process, during the process of restoring the reflective area, due to different shooting directions, different pixel value representations may occur at the same position on the image, and there is even a small probability that a certain area will be excluded from all images, and no basis for restoration can be found during the restoration process. Therefore, the embodiment of the present application can further adjust the first captured image after restoration. On the one hand, by the pixel value distribution of the region of interest, increase or decrease the pixel values to avoid large unevenness and actual deviation of pixel values caused during the restoration process. On the other hand, complement the pixel values of the missing part through the pixel values of the region of interest. Specifically, according to its distribution, comprehensively consider the size of the pixel values and the change of the pixel values to complement the pixel values of the missing part, so that the restored reflective area can be closer to the real situation and improve the quality of defect detection.

[0088] Based on the foregoing steps, fuse the first captured image under the same illumination condition with the design drawing of the standard PCB panel to obtain a number of standard sample images, including:

[0089] Fuse the second captured image under the same illumination condition with the design drawing of the standard PCB panel to obtain a number of standard sample images.

[0090] Refer to Appendix Figure 3 , based on the same inventive concept as in the foregoing embodiments, the embodiment of the present application further provides a product detection device based on machine vision, including:

[0091] An acquisition module, configured to acquire the original image of the PCB panel; wherein, the original image is captured from directly above the PCB panel;

[0092] A detection module, configured to input the original image into a defect detection model to obtain a detection image; wherein, the defect detection model is trained based on sample images, and the sample images include a number of defect sample images and standard sample images. The number of standard sample images are obtained under different illumination conditions respectively, and each standard sample image is obtained by fusing the images of the standard PCB panel captured from multiple different directions under the same illumination condition with the design drawing of the standard PCB panel.

[0093] Those skilled in the art should understand that the division of each module in the embodiments is only a division of logical functions. In actual applications, they can be fully or partially integrated into one or more actual carriers, and these modules can all be implemented in the form of software called by a processing unit, or all be implemented in the form of hardware, or be implemented in the form of a combination of software and hardware. It should be noted that each module in the product detection device based on machine vision in this embodiment corresponds one by one to each step in the product detection method based on machine vision in the foregoing embodiment. Therefore, the specific implementation manners of this embodiment can refer to the implementation manners of the foregoing product detection method based on machine vision, and will not be elaborated here.

[0094] Based on the same inventive concept as in the foregoing embodiment, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when loaded and executed by a processor, implements the product detection method based on machine vision provided in the embodiment of the present application.

[0095] Based on the same inventive concept as in the foregoing embodiment, an embodiment of the present application further provides an electronic device, including a processor and a memory, wherein,

[0096] The memory is used to store a computer program;

[0097] The processor is used to load and execute the computer program so that the electronic device executes the product detection method based on machine vision provided in the embodiment of the present application.

[0098] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or may be various devices including one or any combination of the foregoing memories. The computer may be various computing devices including intelligent terminals and servers.

[0099] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, and may be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0100] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0101] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0102] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0103] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0105] In summary, a product detection method, device, medium, and equipment based on machine vision provided by an embodiment of the present application include: obtaining an original image of a PCB panel; wherein, the original image is obtained by taking a picture directly above the PCB panel; inputting the original image into a defect detection model to obtain a detection image; wherein, the defect detection model is trained based on sample images, and the sample images include a number of defect sample images and standard sample images. The number of standard sample images are obtained under different lighting conditions respectively, and each standard sample image is obtained by fusing the images of the standard PCB panel taken from multiple different directions under the same lighting condition with the design drawing of the standard PCB panel. The present application takes pictures directly above the PCB panel for recognition to ensure that the camera's field of view can evenly and completely cover the PCB panel. By inputting the taken original image into the defect detection model for recognition, a detection image is obtained to complete the defect detection. Since the sample images for training the defect detection model include defect sample images and standard sample images, the model can learn the differences between the defect sample images and the standard sample images, thereby realizing the recognition of defects. Moreover, the standard sample images are obtained under different lighting conditions, enabling the model to adapt to recognition under different lighting conditions. And each standard sample image is obtained by fusing the images of the standard PCB panel taken from multiple different directions with the design drawing of the standard PCB panel, which transfers the normal feature representations of the parts of the standard PCB panel that are affected by environmental lighting to the standard design drawing. Furthermore, using the fused image as a training sample allows the model to not only recognize defect features but also recognize features affected by specular interference under different conditions, thus avoiding misjudgment and missed judgment of defect features and effectively improving the quality of defect detection for the PCB panel.

[0106] The foregoing are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A product detection method based on machine vision, characterized in that: The following steps are involved: Acquire an original image of the PCB panel; wherein the original image is obtained by photographing from directly above the PCB panel; Input the original image into a defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample image includes a plurality of defect sample images and a standard sample image, the plurality of the standard sample images are respectively obtained under different lighting conditions, and each of the standard sample images is obtained by fusing an image of a standard PCB panel taken from a plurality of different directions under the same lighting condition with a design drawing of the standard PCB panel; before inputting the original image into the defect detection model to obtain the detection image, the method further includes: Under different lighting conditions, photographing the standard PCB panel from multiple different directions to obtain a number of photographed images; The captured image under the same lighting condition is merged with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images; after the standard PCB panel is captured from a plurality of different directions under different lighting conditions to obtain a plurality of captured images, the method further comprises: According to the design drawing of the standard PCB panel, drawing the region of interest; Performing threshold segmentation on the captured image to mark a reflective area in the region of interest to obtain a marked image; Based on the plurality of the marked images under the same lighting condition, the reflective area is restored to obtain a first captured image; based on the plurality of the marked images under the same lighting condition, the reflective area is restored to obtain a first captured image, comprising: Segmenting the plurality of marked images under the same lighting condition to remove the reflective area to obtain a first marked image; Taking one of the first marked images as a target image and the other first marked images as non-target images, and restoring the reflective area on the target image according to the area of ​​interest on the non-target image to obtain a first captured image; The step of fusing the captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images comprises: Merging the first captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images; The defect detection model is obtained by training based on a number of the defect sample images and the standard sample images.

2. The product detection method based on machine vision according to claim 1, characterized in that: The step of fusing the captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images includes: Stretching the captured image under the same lighting condition according to the shooting direction, and matching the captured image with the size of the standard PCB panel to obtain the matched captured image; Performing an initial fusion of the matched captured images under the same lighting condition to obtain a fused image; The fused image is fused again with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images.

3. The product detection method based on machine vision according to claim 2, characterized in that: The step of stretching the captured images under the same lighting conditions according to the shooting direction includes: Determine the stretching direction according to the shooting direction; Determine the stretching gradient and the stretching interval according to the positional relationship between the photographing camera and the standard PCB panel; The captured image under the same illumination condition is stretched along the stretching direction according to the stretching gradient and the stretching interval.

4. The product detection method based on machine vision according to claim 1, characterized in that: After taking one of the first marked images as a target image and the other first marked images as non-target images, and restoring the reflective area on the target image according to the area of ​​interest on the non-target image to obtain the first captured image, the method further includes: According to the pixel distribution of the region of interest on the first captured image, the pixel value of the region of interest is adjusted to obtain a second captured image; wherein the adjustment of the pixel value includes: increasing or decreasing the pixel value and completing the pixel value; The step of fusing the first captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images includes: The second captured image under the same lighting condition is merged with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images.

5. A product inspection device based on machine vision, characterized in that: include: An acquisition module, used to acquire an original image of the PCB panel; wherein the original image is obtained by photographing from directly above the PCB panel; A detection module is used to input the original image into a defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample image includes a plurality of defect sample images and a standard sample image, the plurality of standard sample images are respectively obtained under different lighting conditions, and each of the standard sample images is obtained by fusing an image of a standard PCB panel taken from a plurality of different directions under the same lighting condition with a design drawing of the standard PCB panel; before inputting the original image into the defect detection model to obtain the detection image, it also includes: Under different lighting conditions, photographing the standard PCB panel from multiple different directions to obtain a number of photographed images; The captured image under the same lighting condition is merged with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images; after the standard PCB panel is captured from a plurality of different directions under different lighting conditions to obtain a plurality of captured images, the method further includes: According to the design drawing of the standard PCB panel, drawing the region of interest; Performing threshold segmentation on the captured image to mark a reflective area in the region of interest to obtain a marked image; Based on the plurality of the marked images under the same lighting condition, the reflective area is restored to obtain a first captured image; based on the plurality of the marked images under the same lighting condition, the reflective area is restored to obtain a first captured image, comprising: Segmenting the plurality of marked images under the same lighting condition to remove the reflective area to obtain a first marked image; Taking one of the first marked images as a target image and the other first marked images as non-target images, and restoring the reflective area on the target image according to the area of ​​interest on the non-target image to obtain a first captured image; The step of fusing the captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images includes: Merging the first captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images; The defect detection model is obtained by training based on a number of the defect sample images and the standard sample images.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is loaded and executed by the processor, the product detection method based on machine vision as described in any one of claims 1 to 4 is implemented.

7. An electronic device, characterized in that: comprising a processor and a memory, wherein: The memory is used to store computer programs; The processor is used to load and execute the computer program so that the electronic device executes the product detection method based on machine vision as described in any one of claims 1 to 4.

Images