Product detection method and device based on machine vision, medium and equipment

Abstract

The embodiment of the invention discloses a product detection method and device based on machine vision, a medium and equipment, and relates to the technical field of image processing, an image is shot right above a PCB panel for recognition, it is ensured that the visual field of a camera can uniformly and completely cover the PCB panel, and due to the fact that a standard sample image is obtained under different illumination conditions, the detection accuracy is improved. The model can adapt to identification under different illumination conditions, and each standard sample image is obtained by fusing images shot in multiple different directions with a design drawing of a standard PCB panel, so that partial normal feature representation, which can be illuminated by the environment, on the standard PCB panel is transferred to the standard design drawing; and the fused image is used as a training sample, so that the model not only can identify defect features, but also can identify features interfered by reflection under different conditions, thereby avoiding misjudgment and missed judgment of the defect features, and effectively improving the quality of defect detection on the PCB panel.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, medium and equipment for product detection based on machine vision. Background Art

[0002] Machine vision is a rapidly developing branch of artificial intelligence. Simply put, machine vision is to use machines to replace human eyes for measurement and judgment. The machine vision system uses machine vision products (i.e. image acquisition devices, divided into CMOS and CCD) to convert the captured target into image signals, transmit them to a dedicated image processing system, obtain the morphological information of the captured target, and convert them into digital signals based on pixel distribution, brightness, color and other information. The image system performs various operations on these signals to extract the characteristics of the target, and then controls the action of the equipment on site based on the judgment results.

[0003] Product inspection based on machine vision is easily affected by ambient lighting. For example, in the visual inspection of PCB panels, the exposed metal surface of the PCB, smooth coatings or solder joints, and some areas that have been treated with special processes such as gold plating and polishing will produce reflections. If the electronic components on the PCB are large, shadows may also be formed on one side, which will lead to a decrease in the quality of defect detection of PCB panels. Summary of the invention

[0004] The main purpose of this application is to provide a product inspection method, device, medium and equipment based on machine vision, aiming to solve the problem of low quality of defect inspection of PCB panels based on machine vision in the prior art.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows: In a first aspect, an embodiment of the present application provides a product detection method based on machine vision, comprising the following steps: Acquire an original image of the PCB panel; wherein the original image is obtained by photographing from directly above the PCB panel; The original image is input into the defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample images include several defect sample images and standard sample images, the several standard sample images are obtained under different lighting conditions, and each standard sample image is obtained by fusing an image of a standard PCB panel taken from multiple different directions under the same lighting conditions with a design drawing of the standard PCB panel.

[0006] In a possible implementation manner of the first aspect, before inputting the original image into the defect detection model to obtain the detection image, the method further includes: Under different lighting conditions, a standard PCB panel is photographed from multiple different directions to obtain several photographed images; The images captured under the same lighting conditions are merged with the design drawings of the standard PCB panel to obtain several standard sample images; The defect detection model is obtained by training based on several defect sample images and standard sample images.

[0007] In a possible implementation of the first aspect, the captured image under the same lighting conditions is fused with the design drawing of the standard PCB panel to obtain several standard sample images, including: Stretching the captured image under the same lighting condition according to the shooting direction, and matching the captured image with the size of the standard PCB panel to obtain a first captured image; Performing initial fusion on the first captured images under the same illumination condition to obtain a fused image; The fused image is fused again with the design drawing of the standard PCB panel to obtain several standard sample images.

[0008] In a possible implementation manner of the first aspect, stretching images captured under the same lighting conditions according to a shooting direction includes: Determine the stretching direction according to the shooting direction; Determine the stretching gradient and stretching range according to the positional relationship between the camera and the standard PCB panel; According to the stretching gradient and the stretching interval, the images captured under the same lighting conditions are stretched along the stretching direction.

[0009] In a possible implementation manner of the first aspect, after photographing a standard PCB panel from multiple directions under different lighting conditions to obtain a plurality of photographed images, the method further includes: Draw the area of ​​interest based on the design drawing of the standard PCB panel; Performing threshold segmentation on the captured image to mark the reflective area in the region of interest to obtain a marked image; Based on a plurality of marked images under the same lighting condition, the reflective area is restored to obtain a first captured image; The images captured under the same lighting conditions are merged with the design drawings of the standard PCB panel to obtain several standard sample images, including: The first captured image under the same lighting condition is fused with the design drawing of the standard PCB panel to obtain a number of standard sample images.

[0010] In a possible implementation manner of the first aspect, restoring the reflective area based on a plurality of marked images under the same lighting condition to obtain a first captured image includes: Segmenting the plurality of marked images under the same illumination condition to remove the reflective area to obtain a first marked image; One of the first marked images is used as the target image, and the other first marked images are used as non-target images. The reflective area on the target image is restored according to the area of ​​interest on the non-target image to obtain the first captured image.

[0011] In a possible implementation manner of the first aspect, one first marked image is used as a target image, and the other first marked images are used as non-target images. After restoring the reflective area on the target image according to the area of ​​interest on the non-target image and obtaining the first captured image, the method further includes: According to the pixel distribution of the region of interest on the first captured image, the pixel value of the region of interest is adjusted to obtain a second captured image; wherein the adjustment of the pixel value includes: increasing or decreasing the pixel value and completing the pixel value; The first captured image under the same lighting conditions is fused with the design drawing of the standard PCB panel to obtain several standard sample images, including: The second captured image under the same lighting condition is fused with the design drawing of the standard PCB panel to obtain several standard sample images.

[0012] In a second aspect, an embodiment of the present application provides a product inspection device based on machine vision, comprising: An acquisition module is used to acquire an original image of the PCB panel; wherein the original image is obtained by photographing from directly above the PCB panel; The detection module is used to input the original image into the defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample images include a number of defect sample images and standard sample images, the several standard sample images are obtained under different lighting conditions, and each standard sample image is obtained by fusing the image of a standard PCB panel taken from multiple different directions under the same lighting conditions with the design drawing of the standard PCB panel.

[0013] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, a product inspection method based on machine vision as provided in any one of the first aspects above is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein: The memory is used to store computer programs; The processor is used to load and execute a computer program so that the electronic device executes a product detection method based on machine vision as provided in any one of the first aspects above.

[0015] Compared with the prior art, the beneficial effects of this application are: The embodiments of the present application propose a product inspection method, device, medium and equipment based on machine vision, the method comprising: obtaining an original image of a PCB panel; wherein the original image is obtained by shooting from directly above the PCB panel; inputting the original image into a defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample image comprises a number of defect sample images and a standard sample image, the number of standard sample images are respectively obtained under different lighting conditions, and each standard sample image is obtained by fusing an image of a standard PCB panel shot from multiple different directions under the same lighting conditions with a design drawing of the standard PCB panel. The present application takes images from directly above the PCB panel for identification, ensuring that the camera field of view can evenly and completely cover the PCB panel. The original image taken is input into the defect detection model for identification to obtain the detection image to complete the defect detection. Since the sample images for training the defect detection model include defect sample images and standard sample images, the model can learn the difference between the defect sample images and the standard sample images, thereby realizing the identification of defects. The standard sample images are obtained under different lighting conditions, so that the model can adapt to the identification under different lighting conditions. Each standard sample image is obtained by fusing images of standard PCB panels taken from multiple different directions with the design drawing of the standard PCB panel, which means that some normal features on the standard PCB panel that are affected by ambient light are transferred to the standard design drawing. The fused image is used as a training sample, so that the model can not only identify defect features, but also identify features that are interfered by reflections under different conditions, thereby avoiding misjudgment and omission of defect features, and effectively improving the quality of defect detection on PCB panels. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the structure of an electronic device of a hardware operating environment involved in an embodiment of the present application; Figure 2 A flowchart of a product detection method based on machine vision provided in an embodiment of the present application; Figure 3 A schematic diagram of a module of a product inspection device based on machine vision provided in an embodiment of the present application; Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory. DETAILED DESCRIPTION

[0017] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0018] See attached Figure 1 , attached Figure 1 This is a schematic diagram of the structure of an electronic device of the hardware operating environment involved in the embodiment of the present application. The electronic device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display (Display), an input unit such as a keyboard (Keyboard), and the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) memory, or it may be a stable non-volatile memory (NVM), such as at least one disk storage. The processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or it may be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0019] Those skilled in the art will appreciate that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0020] As attached Figure 1 As shown, the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and a product detection device based on machine vision.

[0021] In the attached Figure 1 In the electronic device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in the present application can be set in the electronic device, and the electronic device calls the machine vision-based product detection device stored in the memory 105 through the processor 101, and executes the machine vision-based product detection method provided in the embodiment of the present application.

[0022] See attached Figure 2 Based on the hardware device of the aforementioned embodiment, the embodiment of the present application provides a product detection method based on machine vision, comprising the following steps: S10: Acquire an original image of the PCB panel; wherein the original image is captured from directly above the PCB panel.

[0023] In the specific implementation process, the product inspection of this application is aimed at PCB panels. The Chinese name of PCB is printed circuit board, also known as printed circuit board, which is an important electronic component, a support for electronic components, and a carrier for the electrical connection of electronic components. Since the PCB panel is a thin plate, the original image is obtained by shooting from directly above it, which can ensure that the camera field of view covers the PCB panel evenly as much as possible.

[0024] S20: Inputting the original image into a defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample images include a number of defect sample images and standard sample images, the number of standard sample images are obtained under different lighting conditions, and each standard sample image is obtained by fusing an image of a standard PCB panel taken from multiple different directions under the same lighting conditions with a design drawing of the standard PCB panel.

[0025] In the specific implementation process, the trained defect detection model is used to achieve fast and accurate detection of the original image. The training sample images include defect sample images and standard sample images. Through training, the model learns the difference between the defect sample images and the standard sample images, so that it can identify the defects on the image, complete defect detection and output the detection image. In order to adapt to different lighting conditions, the standard sample images prepared in the training stage are obtained under different lighting conditions. The light source in the real scene may come from different directions, such as natural light, lighting, etc., so the lighting condition here refers to the intensity of the light. After adapting to different light intensities, in order to allow the model to learn the characteristic performance brought by the lighting conditions, multiple images taken from different directions are fused with the design drawing of the standard PCB panel. The basic principle of defect detection is to compare the image with a standard defect-free image, so that the defect can be separated from the background. After the image is fused, the normal features such as reflection and shadow on the captured image due to the influence of light are transferred to the standard image, and these features will not be recognized as defect features by the model when they appear on the original image.

[0026] In this embodiment, an image is captured from directly above the PCB panel for identification to ensure that the camera field of view can evenly and completely cover the PCB panel. The captured original image is input into the defect detection model for identification to obtain a detection image to complete defect detection. Since the sample images for training the defect detection model include defect sample images and standard sample images, the model can learn the difference between the defect sample images and the standard sample images, thereby realizing defect identification. The standard sample images are obtained under different lighting conditions, so that the model can adapt to identification under different lighting conditions. Each standard sample image is obtained by fusing images of the standard PCB panel captured from multiple different directions with the design drawing of the standard PCB panel, so that some normal features on the standard PCB panel that are affected by ambient light are transferred to the standard design drawing. The fused image is used as a training sample, so that the model can not only identify defect features, but also identify features that are interfered by reflections under different conditions, thereby avoiding misjudgment and omission of defect features, and effectively improving the quality of defect detection on the PCB panel.

[0027] In one embodiment, before inputting the original image into the defect detection model to obtain the detection image, the method further includes: Under different lighting conditions, a standard PCB panel is photographed from multiple different directions to obtain several photographed images; The images captured under the same lighting conditions are merged with the design drawings of the standard PCB panel to obtain several standard sample images; The defect detection model is obtained by training based on several defect sample images and standard sample images.

[0028] In the specific implementation process, in the training stage of the model, firstly, the standard PCB panel is photographed under different lighting conditions. The standard PCB panel is also a defect-free PCB panel. The shooting direction is preferably in 2-8 directions, and multiple directions are distributed in a centrally symmetrical or axially symmetrical manner as much as possible, such as a triangular distribution with three vertices of an equilateral triangle, or a cross or a cross-shaped distribution, so that the influence of the direction on the shooting can evenly cover the surroundings of the product. Considering that the images taken from different directions deviate from the whole board image to be compared, it is necessary to fuse the captured images. The design drawing of the standard PCB panel is introduced as a basis, and the captured image is fused with it into an image with the same top view as the original image.

[0029] In one embodiment, the captured image under the same lighting conditions is fused with the design drawing of the standard PCB panel to obtain several standard sample images, including: Stretching the captured image under the same lighting condition according to the shooting direction, and matching the captured image with the size of the standard PCB panel to obtain a first captured image; Performing initial fusion on the first captured images under the same illumination condition to obtain a fused image; The fused image is fused again with the design drawing of the standard PCB panel to obtain several standard sample images.

[0030] In the specific implementation process, since the standard PCB panel is photographed from multiple different directions, the photographed images are stretched and tilted relative to the whole board image in the front view direction, so in the stage of image fusion, the first step is to stretch and restore the image so that the size of the photographed image matches the size of the standard PCB panel. The image after size matching is recorded as the first photographed image. In the fusion stage, the first photographed image under the same lighting conditions is first fused to fuse the feature performances of the different effects of lighting in different shooting directions, and then the fused image is fused again with the design drawing of the standard PCB panel to transfer these feature performances to the standard design drawing, so that the obtained standard sample image has these feature performances. These feature performances are caused by lighting, but they are not defect features. With the learning of defective samples, the model can effectively distinguish between defect features and features affected by lighting.

[0031] In one embodiment, stretching images captured under the same lighting conditions according to the shooting direction includes: Determine the stretching direction according to the shooting direction; Determine the stretching gradient and stretching range according to the positional relationship between the camera and the standard PCB panel; According to the stretching gradient and the stretching interval, the images captured under the same lighting conditions are stretched along the stretching direction.

[0032] In the specific implementation process, the camera shooting has the characteristics of small objects at a distance and large objects at near, so linear stretching cannot be used alone for stretching and restoration. The embodiment of the present application provides a gradient stretching method, that is, different regions use different stretching ratios. First, the stretching direction needs to be determined. The stretching tilt of the image is caused by different shooting directions, so the stretching direction is determined by the shooting direction. The stretching ratio is adjusted according to the characteristics of the camera shooting. First, the positional relationship between the shooting camera and the photographed PCB panel is determined, and then the stretching gradient and stretching interval are determined according to the positional relationship, that is, the captured image is divided into multiple sections along the shooting direction, that is, different stretching intervals. According to the characteristics of the camera shooting image being small at a distance and large at near, the closer the stretching interval is to the camera, the larger the stretching gradient needs to be, and vice versa. In this way, each part of the image can be stretched to match its imaging characteristics, and the captured image can be restored more accurately, thereby improving the quality of the fused image.

[0033] In one embodiment, under different lighting conditions, after photographing a standard PCB panel from multiple different directions to obtain a plurality of photographed images, the method further includes: Draw the area of ​​interest based on the design drawing of the standard PCB panel; Performing threshold segmentation on the captured image to mark the reflective area in the region of interest to obtain a marked image; Based on a plurality of marked images under the same lighting condition, the reflective area is restored to obtain a first captured image.

[0034] In the specific implementation process, after the features affected by lighting are transferred to the standard image, even if there is the influence of shadows, during the comparison process, if there are defects in the shadow part, it can be determined whether there are defects based on the grayscale value expression and pixel value distribution of the shadow part. For the highlighted reflective part, although the possibility of overlapping with the defective area is reduced, once this happens, it will inevitably lead to missed defects, so the reflective area is removed and restored by the above means. Specifically, first draw the area of ​​interest on the captured image according to the design drawing of the standard PCB panel, that is, determine which parts will reflect light according to the design drawing of the standard PCB panel, such as exposed metal surfaces, smooth coatings or solder joints, and some areas that have been treated with special processes such as gold plating and polishing, and mark these areas of interest by actively drawing.

[0035] The grayscale value of the reflective highlight after grayscale conversion is significantly different from that of the surrounding area, so the reflective area is marked in the region of interest by threshold segmentation. It should be noted that the appearance of reflections is based on the components and will not affect the area outside the components, that is, the marking of the reflective area is always performed in the region of interest. Reflections are affected by direction, and the reflective areas produced on the components by light from different directions are different. A reflective area in a certain image may appear normal in an image taken from another direction. Therefore, the reflective area can be restored by combining the marked images under the same lighting conditions, and the restored image is the first captured image.

[0036] Based on the above steps, the images taken under the same lighting conditions are fused with the design drawings of the standard PCB panel to obtain several standard sample images, including: The first captured image under the same lighting condition is fused with the design drawing of the standard PCB panel to obtain a number of standard sample images.

[0037] In one embodiment, restoring the reflective area based on a plurality of marked images under the same lighting condition to obtain a first captured image includes: Segmenting the plurality of marked images under the same illumination condition to remove the reflective area to obtain a first marked image; One of the first marked images is used as the target image, and the other first marked images are used as non-target images. The reflective area on the target image is restored according to the area of ​​interest on the non-target image to obtain the first captured image.

[0038] In the specific implementation process, in the specific restoration process, the reflective areas on each marked image are first removed according to the mark, and the removed image is recorded as the first marked image. Then, in order to improve the quality of the restored image, a certain image is not used as a fixed standard, but a multi-image restoration method is adopted, each first marked image is used as the target image, and the remaining first marked images are used as non-target images to restore the missing reflective area on the target image. In this way, the first captured image restored under multiple standards can more accurately restore the original information of the reflective area on the image after fusion. If there are defects in the reflective area at the same time, the defects will be retained on some images that have not been removed, and then the defect information can also be restored after fusion, thereby avoiding the model from missing defects in the reflective part.

[0039] In one embodiment, one first marked image is used as the target image and the other first marked images are used as non-target images. After restoring the reflective area on the target image according to the area of ​​interest on the non-target image and obtaining the first captured image, the method further includes: According to the pixel distribution of the region of interest on the first captured image, the pixel value of the region of interest is adjusted to obtain the second captured image; wherein the adjustment of the pixel value includes: increase or decrease of the pixel value and completion of the pixel value.

[0040] In the specific implementation process, in the process of restoring the reflective area, different pixel values ​​may be generated at the same position on the image due to different shooting directions. There is even a small probability that a certain area will be eliminated in all images, and no basis for restoration can be found in the restoration process. Therefore, the embodiment of the present application can further adjust the first captured image after restoration. On the one hand, the pixel values ​​are increased or decreased by the pixel value distribution of the region of interest to avoid uneven pixel values ​​caused by the restoration process and large deviations from the actual value. On the other hand, the missing part is supplemented by the pixel value of the region of interest. Specifically, the pixel value of the missing part can be supplemented according to its distribution, taking into account the size of the pixel value and the change of the pixel value, so that the restored reflective area can be closer to the actual situation and improve the quality of defect detection.

[0041] Based on the above steps, the first captured image under the same lighting condition is fused with the design drawing of the standard PCB panel to obtain several standard sample images, including: The second captured image under the same lighting condition is fused with the design drawing of the standard PCB panel to obtain several standard sample images.

[0042] See attached Figure 3 Based on the same inventive concept as in the above-mentioned embodiment, the embodiment of the present application further provides a product inspection device based on machine vision, comprising: An acquisition module is used to acquire an original image of the PCB panel; wherein the original image is obtained by photographing from directly above the PCB panel; The detection module is used to input the original image into the defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample images include a number of defect sample images and standard sample images, the several standard sample images are obtained under different lighting conditions, and each standard sample image is obtained by fusing the image of a standard PCB panel taken from multiple different directions under the same lighting conditions with the design drawing of the standard PCB panel.

[0043] Those skilled in the art should understand that the division of the various modules in the embodiment is merely a division of logical functions, and in actual application, all or part of them can be integrated into one or more actual carriers, and these modules can be implemented entirely in the form of software called by a processing unit, or entirely in the form of hardware, or in the form of a combination of software and hardware. It should be noted that each module in the product inspection device based on machine vision in this embodiment corresponds one-to-one to each step in the product inspection method based on machine vision in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned product inspection method based on machine vision, which will not be repeated here.

[0044] Based on the same inventive concept as in the aforementioned embodiments, an embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, a product inspection method based on machine vision as provided in an embodiment of the present application is implemented.

[0045] Based on the same inventive concept as in the above-mentioned embodiment, an embodiment of the present application further provides an electronic device, including a processor and a memory, wherein: The memory is used to store computer programs; The processor is used to load and execute a computer program so that the electronic device executes a product detection method based on machine vision as provided in an embodiment of the present application.

[0046] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0047] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0048] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0049] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0050] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0051] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0052] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0053] In summary, the embodiments of the present application provide a product inspection method, device, medium and equipment based on machine vision, the method comprising: obtaining an original image of a PCB panel; wherein the original image is obtained by taking a photo from directly above the PCB panel; inputting the original image into a defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample image comprises a number of defect sample images and a standard sample image, the number of standard sample images are respectively obtained under different lighting conditions, and each standard sample image is obtained by fusing an image of a standard PCB panel taken from multiple different directions under the same lighting conditions with a design drawing of the standard PCB panel. The present application takes images from directly above the PCB panel for identification, ensuring that the camera field of view can evenly and completely cover the PCB panel. The original image taken is input into the defect detection model for identification to obtain the detection image to complete the defect detection. Since the sample images for training the defect detection model include defect sample images and standard sample images, the model can learn the difference between the defect sample images and the standard sample images, thereby realizing the identification of defects. The standard sample images are obtained under different lighting conditions, so that the model can adapt to the identification under different lighting conditions. Each standard sample image is obtained by fusing images of standard PCB panels taken from multiple different directions with the design drawing of the standard PCB panel, which means that some normal features on the standard PCB panel that are affected by ambient light are transferred to the standard design drawing. The fused image is used as a training sample, so that the model can not only identify defect features, but also identify features that are interfered by reflections under different conditions, thereby avoiding misjudgment and omission of defect features, and effectively improving the quality of defect detection on PCB panels.

[0054] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A product detection method based on machine vision, characterized in that: The following steps are involved: Acquire an original image of the PCB panel; wherein the original image is obtained by photographing from directly above the PCB panel; The original image is input into a defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, the sample images include a number of defect sample images and a standard sample image, the several standard sample images are obtained under different lighting conditions, and each of the standard sample images is obtained by fusing an image of a standard PCB panel taken from multiple different directions under the same lighting conditions with a design drawing of the standard PCB panel.

2. The product detection method based on machine vision according to claim 1, characterized in that: Before inputting the original image into the defect detection model to obtain the detection image, the method further includes: Under different lighting conditions, photographing the standard PCB panel from multiple different directions to obtain a number of photographed images; Merging the captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images; The defect detection model is obtained by training based on a number of the defect sample images and the standard sample images.

3. The product detection method based on machine vision according to claim 2, characterized in that: The step of fusing the captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images includes: Stretching the captured image under the same lighting condition according to the shooting direction, and matching the captured image with the size of the standard PCB panel to obtain a first captured image; Performing an initial fusion of the first captured images under the same lighting condition to obtain a fused image; The fused image is fused again with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images.

4. The product detection method based on machine vision according to claim 3, characterized in that: The step of stretching the captured images under the same lighting conditions according to the shooting direction includes: Determine the stretching direction according to the shooting direction; Determine the stretching gradient and the stretching interval according to the positional relationship between the photographing camera and the standard PCB panel; The captured image under the same illumination condition is stretched along the stretching direction according to the stretching gradient and the stretching interval.

5. The product detection method based on machine vision according to claim 2, characterized in that: After photographing the standard PCB panel from multiple directions under different lighting conditions to obtain a plurality of photographed images, the method further includes: According to the design drawing of the standard PCB panel, drawing the region of interest; Performing threshold segmentation on the captured image to mark a reflective area in the region of interest to obtain a marked image; Based on the plurality of the marked images under the same lighting condition, the reflective area is restored to obtain a first captured image; The step of fusing the captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images includes: The first captured image under the same lighting condition is merged with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images.

6. The method for product detection based on machine vision according to claim 5, characterized in that: The method of restoring the reflective area based on the plurality of marked images under the same lighting condition to obtain a first captured image includes: Segmenting the plurality of marked images under the same lighting condition to remove the reflective area to obtain a first marked image; One of the first marked images is used as a target image and the other first marked images are used as non-target images. The reflective area on the target image is restored according to the area of ​​interest on the non-target image to obtain a first captured image.

7. The method for product detection based on machine vision according to claim 6, characterized in that: After taking one of the first marked images as a target image and the other first marked images as non-target images, and restoring the reflective area on the target image according to the area of ​​interest on the non-target image to obtain the first captured image, the method further includes: According to the pixel distribution of the region of interest on the first captured image, the pixel value of the region of interest is adjusted to obtain a second captured image; wherein the adjustment of the pixel value includes: increasing or decreasing the pixel value and completing the pixel value; The step of fusing the first captured image under the same lighting condition with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images includes: The second captured image under the same lighting condition is merged with the design drawing of the standard PCB panel to obtain a plurality of the standard sample images.

8. A product inspection device based on machine vision, characterized in that: include: An acquisition module, used to acquire an original image of the PCB panel; wherein the original image is obtained by photographing from directly above the PCB panel; The detection module is used to input the original image into a defect detection model to obtain a detection image; wherein the defect detection model is obtained based on sample image training, and the sample images include a plurality of defect sample images and a standard sample image, and the plurality of the standard sample images are respectively obtained under different lighting conditions, and each of the standard sample images is obtained by fusing an image of a standard PCB panel taken from a plurality of different directions under the same lighting conditions with a design drawing of the standard PCB panel.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is loaded and executed by the processor, the product detection method based on machine vision as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: comprising a processor and a memory, wherein: The memory is used to store computer programs; The processor is used to load and execute the computer program so that the electronic device executes the product detection method based on machine vision as described in any one of claims 1 to 7.

Images