PCB defect detection method and device based on HSV color space and computing equipment

Abstract

The invention discloses a PCB defect detection method and device based on an HSV color space and computing equipment, and the method comprises the steps: carrying out the coordinate correction of a to-be-detected image based on a template image, and obtaining a to-be-detected image annotation file; converting the to-be-detected image annotation file from an RGB channel into an HSV channel to obtain an HSV channel image; extracting an area between the adjacent bonding pads from the HSV channel image, and determining whether the PCB has a short circuit defect according to color distribution of the area between the adjacent bonding pads; comparing the HSV channel image with the tin-deficient color extraction image to determine whether each bonding pad has a tin-deficient defect or not; extracting a copper region in each bonding pad region from the HSV channel image to determine whether each bonding pad has a missing solder defect, a false solder defect and a copper sheet defect; and detecting whether a pin region of which the middle is bright color and the periphery is dark color exists in the middle of each bonding pad region in the HSV channel image so as to determine whether each bonding pad has a kneeling defect. According to the invention, various defects of the PCB can be accurately and comprehensively detected, and the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and in particular to a PCB board defect detection method based on HSV color space, a PCB board defect detection device and a computing device. Background Art

[0002] With the rapid development of the electronics industry, PCB (printed circuit board) is the core component of electronic equipment, and its manufacturing quality is crucial to the overall performance and reliability of the product. The solder joints on the PCB board are the key link between electronic components and PCB boards, and their quality directly affects the electrical connection and signal transmission performance of the circuit board. Therefore, the inspection of the solder joints on the back of the PCB board has become a key step to ensure the quality of the circuit board. Nowadays, visual inspection methods have occupied a very important position in the industrial manufacturing process. For example, it can be used to detect surface defects of products. Its non-contact and high-precision detection characteristics enable it to replace people to complete special tasks.

[0003] According to the existing AOI inspection equipment, there are three main methods for inspecting solder joints on PCB boards. The first is template matching: the AOI equipment will pre-store the image template of the standard solder joint, and judge whether the solder joint meets the standard by matching the template with the actual collected solder joint image, but its matching effect is affected by many factors, resulting in inaccurate and unreliable inspection results. The second is feature extraction and recognition: the key features of the solder joint (such as shape, size, position, etc.) are extracted using image processing algorithms, and compared with the preset standard values ​​to identify solder joint defects. This method is very sensitive to parameters. The third is machine learning algorithm: with the development of artificial intelligence technology, some AOI inspection equipment has begun to use machine learning algorithms to train detection models to improve the accuracy and efficiency of defect identification. This method requires a large amount of training data and is highly resource-dependent. It can be seen that the existing AOI inspection equipment has more or less defects in the detection methods of PCB board solder joints.

[0004] Therefore, a PCB board defect detection method based on the HSV color space is needed to solve the problems existing in the above technical solutions. Summary of the invention

[0005] To this end, the present invention provides a PCB board defect detection method and a PCB board defect detection device based on the HSV color space to solve or at least alleviate the above problems.

[0006] According to one aspect of the present invention, a PCB defect detection method based on HSV color space is provided, which is executed in a computing device, wherein the PCB has a plurality of pads for welding, and the method comprises: obtaining an image to be detected, a low-tin color extraction image, and a template image of the PCB to be detected; matching feature points of the image to be detected with the template image, so as to perform coordinate correction on the image to be detected based on the template image, and obtain an annotation file of the image to be detected; based on the HSV color space distribution law, converting the annotation file of the image to be detected from RGB channels to HSV channels, and obtaining an HSV channel image; extracting adjacent pads from the HSV channel image The HSV channel image is compared with the tin-scarce color extraction image to determine whether each pad of the PCB board has a tin-scarce defect; the copper-colored area in each pad area is extracted from the HSV channel image, and based on the copper-colored area in each pad area, it is determined whether each pad of the PCB board has an empty soldering defect, a false soldering defect, and a copper skin defect; and the pin area with a bright middle and dark surrounding in the middle of each pad area in the HSV channel image is detected to determine whether each pad of the PCB board has a kneeling defect.

[0007] Optionally, in the PCB board defect detection method based on the HSV color space according to the present invention, the area between adjacent pads is extracted from the HSV channel image, including: extracting a pad dense distribution area from the HSV channel image, the pad dense distribution area includes a plurality of densely distributed pad areas; based on the short circuit mask image, removing the plurality of pad areas from the pad dense distribution area to obtain the area between adjacent pads.

[0008] Optionally, in the PCB board defect detection method based on the HSV color space according to the present invention, determining whether the PCB board has a short circuit defect according to the color distribution of the area between the adjacent pads includes: judging whether there is a color distribution in the area between the adjacent pads; if there is a color distribution, determining that there is a short circuit defect in the area between the adjacent pads of the PCB board, and extracting a red, green and blue three-color image from the HSV channel image, performing threshold segmentation on the red, green and blue three-color image to obtain a threshold segmentation image, and performing contour extraction on the threshold segmentation image to extract the short circuit defect area; if there is no color distribution, determining that there is no short circuit defect on the PCB board.

[0009] Optionally, in the PCB board defect detection method based on the HSV color space according to the present invention, the HSV channel image is compared with the low-tin color extraction image to determine whether each pad of the PCB board has a low-tin defect, including: extracting multiple pad areas from the HSV channel image as multiple comparison areas; for each of the comparison areas, determining the mean square error value between the comparison area and the corresponding area in the low-tin color extraction image; if the mean square error value is greater than the mean square error threshold, extracting a first red area from the comparison area, and comparing the first red area with the corresponding second red area in the low-tin color extraction image to determine whether the pad corresponding to the comparison area has a low-tin defect.

[0010] Optionally, in the PCB board defect detection method based on the HSV color space according to the present invention, the first red area is compared with the corresponding second red area in the low-tin color extraction image to determine whether there is a low-tin defect in the pad corresponding to the comparison area, including: calculating the red pixel difference between the first red area and the second red area; if the red pixel difference between the first red area and the second red area is greater than a preset pixel difference, it is determined that there is a low-tin defect in the pad corresponding to the comparison area, and the first red area is determined as a low-tin defect area.

[0011] Optionally, in the PCB board defect detection method based on the HSV color space according to the present invention, the copper-colored area in each pad area is extracted from the HSV channel image, including: extracting the area in each pad area where the H channel value is distributed between 3 and 8 from the HSV channel image to obtain the copper-colored area in each pad area.

[0012] Optionally, in the PCB board defect detection method based on the HSV color space according to the present invention, based on the copper-colored area in each pad area, determining whether each pad of the PCB board has an empty soldering defect, a false soldering defect, and a copper skin defect, includes: for the copper-colored area in each pad area, determining the area ratio of the copper-colored area in the corresponding pad area, if the area ratio is greater than a first predetermined ratio, and there is a black ring near the pin of the pad area, then determining that the pad corresponding to the pad area has an empty soldering defect; judging whether there is a black void area between the copper-colored area in the pad area and the pin of the pad area, if there is a black void area, further judging whether there is an overlapping area between the circumscribed rectangle of the copper-colored area and the circumscribed rectangle of the black void area, if there is an overlapping area, determining that the pad corresponding to the pad area has a false soldering defect; if the area ratio is greater than a third predetermined ratio and less than the second predetermined ratio, then determining that the pad corresponding to the pad area has a copper skin defect, wherein the third predetermined ratio is less than the second predetermined ratio, and the second predetermined ratio is less than the first predetermined ratio.

[0013] Optionally, in the PCB board defect detection method based on the HSV color space according to the present invention, it is detected whether there is a pin area with a bright middle and dark surrounding in the middle of each pad area in the HSV channel image, so as to determine whether there are kneeling defects in each pad of the PCB board, including: converting the HSV channel image into a grayscale image, and binarizing the grayscale image to obtain a binary image; detecting whether there is a bright area in the middle of each pad area in the binary image, and whether there is a dark area with an average brightness value lower than a brightness threshold around the bright area; if it is detected that there is a bright area in the middle of the pad area and there is a dark area around the bright area, it is determined that there is a pin area in the middle of the pad area, and the pad corresponding to the pad area does not have a kneeling defect; if it is not detected that there is a bright area in the middle of the pad area and there is a dark area around the bright area, it is determined that there is no pin area in the middle of the pad area, and the pad corresponding to the pad area has a kneeling defect.

[0014] Optionally, in the PCB board defect detection method based on the HSV color space according to the present invention, it also includes: marking each defect existing in the PCB board in the image annotation file to be detected to obtain a defect detection result.

[0015] According to one aspect of the present invention, a PCB defect detection device is provided, which is deployed in a computing device, wherein the PCB has a plurality of pads for welding, and the device comprises: an acquisition unit, adapted to acquire an image to be detected, a low-tin color extraction image, and a template image of the PCB to be detected; a matching unit, adapted to match feature points of the image to be detected with the template image, so as to perform coordinate correction on the image to be detected based on the template image to obtain an annotation file of the image to be detected; a conversion unit, adapted to convert the annotation file of the image to be detected from an RGB channel to an HSV channel based on the HSV color space distribution law to obtain an HSV channel image; and a first detection unit, adapted to extract an area between adjacent pads from the HSV channel image. , determining whether the PCB board has a short circuit defect according to the color distribution of the area between the adjacent pads; the second detection unit is suitable for comparing the HSV channel image with the low-tin color extraction image to determine whether each pad of the PCB board has a low-tin defect; the third detection unit is suitable for extracting the copper-colored area in each pad area from the HSV channel image, and based on the copper-colored area in each pad area, determining whether each pad of the PCB board has an empty soldering defect, a false soldering defect, and a copper skin defect; the fourth detection unit is suitable for detecting whether there is a pin area with a bright middle and dark surroundings in the middle of each pad area in the HSV channel image, so as to determine whether each pad of the PCB board has a kneeling defect.

[0016] According to one aspect of the present invention, there is provided a computing device, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be suitable for execution by the at least one processor, and the program instructions include instructions for executing the PCB board defect detection method based on the HSV color space as described above.

[0017] According to one aspect of the present invention, a computer program product is provided, comprising a computer program / instruction, wherein the computer program / instruction implements the method as described above when executed by a processor.

[0018] According to one aspect of the present invention, a readable storage medium storing program instructions is provided. When the program instructions are read and executed by a computing device, the computing device executes the PCB board defect detection method based on the HSV color space as described above.

[0019] According to the technical solution of the present invention, a PCB board defect detection method based on HSV color space is provided. First, the image to be detected, the color extraction image of the PCB board to be detected and the template image are obtained under three-color light. The image to be detected is coordinate-corrected based on the template image to obtain the image annotation file to be detected, and the image annotation file to be detected is converted into an HSV channel image based on the HSV color space. Then, the PCB board is determined to have a short circuit defect based on the color distribution of the area between adjacent pads. By comparing the HSV channel image with the color extraction image of the color extraction, it is determined whether each pad of the PCB board has a tin defect; by extracting the copper color area in each pad area from the HSV channel image, it is determined whether each pad of the PCB board has an empty soldering defect, a false soldering defect, and a copper skin defect based on the copper color area in each pad area; and by detecting whether there is a pin area with a bright color in the middle and a dark color around the middle of each pad area in the HSV channel image, it is determined whether each pad of the PCB board has a kneeling defect. Based on this, the image based on the HSV color space of the present invention can more intuitively present various defects in the PCB board, and then can more accurately and comprehensively detect various defects in the PCB board, with better robustness and higher detection efficiency. In addition, the PCB board defect detection according to the present invention does not need to rely on training data, has low sensitivity to parameters, and is easy to adjust the threshold according to actual detection requirements.

[0020] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to achieve the above and related purposes, some illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. By reading the following detailed description in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present invention will become more apparent. Throughout the present invention, the same reference numerals generally refer to the same parts or elements.

[0022] Figure 1 A schematic diagram of a computing device 100 provided according to an embodiment of the present invention is shown;

[0023] Figure 2 A schematic flow chart of a PCB board defect detection method 200 based on the HSV color space provided according to an embodiment of the present invention is shown;

[0024] Figure 3A schematic diagram of an image to be detected according to an embodiment of the present invention is shown;

[0025] Figure 4 A schematic diagram of a tin-poor color extraction image according to an embodiment of the present invention is shown;

[0026] Figure 5 A schematic diagram of a template image according to an embodiment of the present invention is shown;

[0027] Figure 6 A schematic diagram showing a densely distributed pad area extracted according to an embodiment of the present invention is shown;

[0028] Figure 7 A schematic diagram of a short circuit mask image according to an embodiment of the present invention is shown;

[0029] Figure 8 A schematic diagram showing a solder pad having a tin deficiency defect according to an embodiment of the present invention is shown;

[0030] Fig. 9 A schematic diagram showing a solder pad having a void soldering defect according to an embodiment of the present invention is shown;

[0031] Fig.10 A schematic diagram showing a false soldering defect in a solder pad according to an embodiment of the present invention is shown;

[0032] Fig.11 A schematic diagram showing a copper skin defect on a pad according to an embodiment of the present invention is shown;

[0033] Fig.12 A schematic diagram showing a kneeling defect of a pad according to an embodiment of the present invention is shown;

[0034] Fig.13 A schematic diagram of a PCB defect detection device 1300 provided according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0035] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0036] In view of the problems existing in the existing AOI inspection equipment's inspection method for PCB board solder joints, an embodiment of the present invention proposes a PCB board defect detection method based on the HSV color space. The image based on the HSV color space can more intuitively present various defects in the PCB board, and thus can more accurately and comprehensively detect various defects in the PCB board, with better robustness and higher detection efficiency.

[0037] The PCB board defect detection method based on the HSV color space in the embodiment of the present invention can be executed in a computing device.

[0038] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] Figure 1 FIG. 1 is a schematic diagram of a computing device 100 provided according to an embodiment of the present invention. Figure 1 As shown, in a basic configuration, the computing device 100 includes at least one processing unit 102 and a system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 can be implemented as a processor. The system memory 104 includes, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, the system memory 104 includes an operating system 105.

[0040] According to one aspect, operating system 105 is suitable for controlling the operation of computing device 100, for example. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other application programs, and are not limited to any particular application or system. Figure 1 The basic configuration is shown in FIG. 1 by those components within the dashed lines. According to one aspect, computing device 100 has additional features or functionality. For example, according to one aspect, computing device 100 includes additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 1 1 is illustrated by a removable storage device 109 and a non-removable storage device 110.

[0041] As stated above, according to one aspect, a program module 103 is stored in the system memory 104. According to one aspect, the program module 103 may include one or more application programs, and the present invention does not limit the type of application program, for example, the application program may include: email and contact application programs, word processing application programs, spreadsheet application programs, database application programs, slide show application programs, drawing or computer-aided application programs, web browser application programs, etc.

[0042] According to one aspect, the program module 103 may include multiple program instructions suitable for executing the PCB board defect detection method 200 based on the HSV color space of the present invention, so that the computing device 100 is configured to execute the PCB board defect detection method 200 based on the HSV color space of the present invention.

[0043] According to one aspect, the program module 103 may include a PCB board defect detection device 1300, and the PCB board defect detection device 1300 may be configured to execute the PCB board defect detection method 200 based on the HSV color space of the present invention.

[0044] According to one aspect, the examples may be practiced on a circuit comprising discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or a single chip containing electronic components or a microprocessor. Figure 1 Each or many components shown in can be integrated into a system on a chip (SOC) on a single integrated circuit to practice examples. According to one aspect, such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") into a chip substrate as a single integrated circuit. When operated via SOC, the functions described in this article can be operated via a dedicated logic integrated with other components of the computing device 100 on a single integrated circuit (chip). Embodiments of the present invention can also be practiced using other technologies capable of performing logical operations (such as AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, embodiments of the present invention can be practiced in a general-purpose computer or in any other circuit or system.

[0045] According to one aspect, the computing device 100 may also have one or more input devices 112, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, etc. Output devices 114 may also be included, such as a display, a speaker, a printer, etc. The aforementioned devices are examples and other devices may also be used. The computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuits; Universal Serial Bus (USB), parallel and / or serial ports.

[0046] The term computer-readable medium as used herein includes computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules 103). System memory 104, removable storage device 109, and non-removable storage device 110 are all examples of computer storage media (i.e., memory storage). Computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices, or any other products that can be used to store information and can be accessed by computing device 100. According to one aspect, any such computer storage medium can be a part of computing device 100. Computer storage media do not include carrier waves or other propagated data signals.

[0047] According to one aspect, communication media is implemented by computer readable instructions, data structures, program modules 103, or other data in a modulated data signal (e.g., a carrier wave or other transport mechanism), and includes any information delivery media. According to one aspect, the term "modulated data signal" describes a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0048] In an embodiment according to the present invention, the computing device 100 is configured to execute the PCB board defect detection method 200 based on the HSV color space of the present invention. The computing device 100 includes one or more processors and one or more readable storage media storing program instructions. When the program instructions are configured to be executed by one or more processors, the computing device executes the PCB board defect detection method 200 based on the HSV color space in the embodiment of the present invention.

[0049] Figure 2 A schematic flow chart of a PCB board defect detection method 200 based on HSV color space provided according to an embodiment of the present invention is shown. The PCB board defect detection method 200 based on HSV color space can be executed in a computing device (eg, the aforementioned computing device 100).

[0050] According to the PCB board defect detection method 200 based on the HSV color space of the present invention, the defect types of the PCB board that can be detected include but are not limited to short circuit defects, insufficient tin defects, empty soldering defects, false soldering defects, copper foil defects, and kneeling defects.

[0051] It should be noted that the PCB has a plurality of pads for soldering, and the pads are metal parts on the PCB used for soldering component pins or metal ends.

[0052] like Figure 2 As shown, the PCB board defect detection method 200 based on the HSV color space includes the following steps 210-260.

[0053] First, in step 210, the computing device 100 may obtain an image to be detected, a low-tin color extraction image, and a template image of the PCB to be detected. Here, the image to be detected is an image of the PCB to be detected under three-color light, and the image to be detected is an image of RGB channels.

[0054] here, Figure 3 A schematic diagram of an image to be detected according to an embodiment of the present invention is shown. Figure 4 A schematic diagram of a tin-poor color extraction image according to an embodiment of the present invention is shown. Figure 5 It is a schematic diagram showing a template image according to an embodiment of the present invention. It should be noted that the low tin color extraction image is an image used to detect low tin defects in the image annotation file to be detected, and "low tin" refers to an area with less tin.

[0055] Subsequently, in step 220, the computing device 100 may use a comparison algorithm to match feature points of the image to be detected with the template image, so as to correct the coordinates of the image to be detected based on the template image and obtain a labeling file of the image to be detected (i.e., a json file of the image to be detected).

[0056] It should be noted that the image annotation file to be inspected contains the coordinate position information of each defect type to be inspected and each part to be inspected (including the pad).

[0057] Next, in step 230 , the computing device 100 may convert the image to be detected from RGB channels to HSV channels based on the HSV color space distribution rule to obtain an HSV channel image.

[0058] Subsequently, in step 240, the computing device 100 may extract the area between adjacent pads (including the area between any two adjacent pads) from the HSV channel image, and determine whether there is a short circuit defect on the PCB board based on the color distribution of the area between adjacent pads.

[0059] In some embodiments of the present invention, in step 240, the computing device 100 may extract the area between adjacent pads from the HSV channel image in the following manner: First, the computing device 100 extracts the area with densely distributed pads from the HSV channel image. Figure 6 FIG. 1 shows a schematic diagram of a densely distributed pad area extracted according to an embodiment of the present invention, Figure 7 FIG. 1 is a schematic diagram of a short circuit mask image according to an embodiment of the present invention. It should be noted that the pad area corresponds to the purple area of ​​the short circuit mask image. Figure 6 and Figure 7 As shown, the pad dense distribution area includes a plurality of densely distributed pad areas, and the shape of the pad area can be circular, square, or teardrop-shaped.

[0060] Furthermore, the computing device 100 may remove a plurality of pad regions from the pad densely distributed region based on the short-circuit mask image to obtain regions between adjacent pads (corresponding to the yellow region of the short-circuit mask image).

[0061] It should be noted that, when solder exists in the area between adjacent pads of the PCB board, color distribution will appear under the illumination of three-color light. Based on this, in step 240, after the computing device 100 extracts the area between adjacent pads from the HSV channel image, it can be determined whether there is color distribution in the area between adjacent pads. If there is color distribution in the area between adjacent pads, it means that solder exists in the area between adjacent pads of the PCB board, and it can be determined that there is a short circuit defect in the area between adjacent pads of the PCB board, and the color distribution position is the short circuit defect position. If there is no color distribution in the area between adjacent pads, it means that there is no solder in the area between adjacent pads of the PCB board, and it can be determined that there is no short circuit defect in the PCB board.

[0062] Furthermore, if it is determined that there is a short circuit defect in the area between adjacent pads of the PCB board, the computing device 100 can extract the short circuit defect area in the following manner: extract the red, green and blue three-color image from the HSV channel image, perform threshold segmentation on the red, green and blue three-color image to obtain a threshold segmentation image, and perform contour extraction on the threshold segmentation image to extract the short circuit defect area. The threshold segmentation method for the red, green and blue three-color image is as follows: all brightness values ​​greater than 190 in the red, green and blue three-color image are set to 255, and all brightness values ​​less than 190 in the red, green and blue three-color image are set to 0, so as to achieve threshold segmentation of the red, green and blue three-color image.

[0063] Next, in step 250 , the computing device 100 may compare the HSV channel image with the insufficient tin color extraction image to determine whether there is an insufficient tin defect on each pad of the PCB board.

[0064] It should be noted that due to the small amount of tin on the PCB, the surface is flat and appears red under three-color light. Therefore, if there is a red part in the HSV channel image of the PCB, it means that the PCB may have a small amount of tin defect. It should also be pointed out that when the H channel value in the HSV channel image is distributed between 0-34 or 150-180, it will appear red, which is more intuitive than the RBG three-channel with light invariance.

[0065] In some embodiments, the computing device 100 compares the HSV channel image with the low-tin color extraction image in the following specific manner: First, the computing device 100 can extract multiple pad areas from the HSV channel image as multiple comparison areas, wherein each pad area is respectively used as a comparison area. For example, the size of the comparison area can be determined as 15×15 based on the pad size. Furthermore, the computing device 100 can determine the mean square error value (MSE value) between the comparison area and the corresponding area in the low-tin color extraction image for each comparison area, and determine whether the mean square error value (MSE value) between the comparison area and the corresponding area in the low-tin color extraction image is greater than the mean square error threshold (MSE threshold). If the mean square error value between the comparison area and the corresponding area in the low-tin color extraction image is greater than the mean square error threshold, a first red area can be further extracted from the comparison area, and the first red area in the comparison area is compared with the corresponding second red area in the low-tin color extraction image to determine whether the pad corresponding to the comparison area has a low-tin defect. Specifically, for the comparison area where the mean square error value of the corresponding area in the low-tin color extraction image is greater than the mean square error threshold, the red pixel difference between the first red area in the comparison area and the corresponding second red area in the low-tin color extraction image can be calculated, and whether the red pixel difference is greater than the preset pixel difference; if the red pixel difference is greater than the preset pixel difference, it can be determined that there is a low-tin defect in the solder pad corresponding to the comparison area, and the first red area in the comparison area can be determined as a low-tin defect area.

[0066] here, Figure 8 A schematic diagram showing a solder pad having a tin deficiency defect according to an embodiment of the present invention is shown, in which the red area (first red area) is the tin deficiency defect area.

[0067] In some embodiments, the preset mean square error threshold is, for example, 80. The preset pixel difference value is, for example, 0.2. However, the present invention is not limited thereto, and it should be understood that the preset mean square error threshold and the preset pixel difference value can be adjusted according to actual use requirements.

[0068] Subsequently, in step 260, the computing device 100 can extract the copper-colored areas in each pad area from the HSV channel image, and then, based on the copper-colored areas in each pad area, determine whether each pad of the PCB board has an empty soldering defect, a false soldering defect, or a copper skin defect.

[0069] It should be noted that in the embodiments of the present invention, empty soldering, i.e., the situation where the pad is not soldered at all, will result in a large copper-colored area on the pad. Fake soldering, i.e., the situation where part of the pad is not soldered, will result in a partial copper-colored area on the pad, and there will be a black void area between the partial copper-colored area and the pin.

[0070] In some embodiments, the copper-colored area is an area where the H channel value is distributed between 3 and 8. In step 250, the computing device 100 can obtain the copper-colored area in each pad area by extracting the area where the H channel value is distributed between 3 and 8 in each pad area from the HSV channel image. Optionally, in the process of extracting the copper-colored area in each pad area, the copper-colored area with a smaller area can also be filtered out, for example, the copper-colored area with an area of ​​less than or equal to 4% in the pad area can be filtered out.

[0071] Fig. 9 A schematic diagram showing a solder pad having an empty soldering defect according to an embodiment of the present invention is shown. In some embodiments, based on the copper-colored area in each solder pad area, it is possible to determine whether each solder pad of the PCB board has an empty soldering defect in the following manner: for each copper-colored area in the solder pad area, firstly, the area ratio of the copper-colored area in the corresponding solder pad area can be determined, and it is determined whether the area ratio is greater than a first predetermined ratio. If the area ratio of the copper-colored area in the corresponding solder pad area is greater than the first predetermined ratio, it is possible to further determine whether there is a black ring near the pin of the solder pad area (the pin is located in the middle of the solder pad area). If there is a black ring near the pin of the solder pad area, refer to Fig. 9 , it can be determined that the pad corresponding to the pad area has a blank soldering defect. In one embodiment, the first predetermined proportion can be, for example, 50%.

[0072] Fig.10A schematic diagram showing a solder joint defect in a pad according to an embodiment of the present invention is shown. In some embodiments, based on the copper-colored area in each pad area, it can be determined whether each pad of the PCB board has a solder joint defect in the following manner: For the copper-colored area in each pad area, it can be first determined whether there is a black hole area between the copper-colored area in the pad area and the pin of the pad area. Here, the area with a grayscale lower than 30 can be defined as a black hole area. If there is a black hole area between the copper-colored area and the pin of the pad area, it can be further determined whether there is an overlapping area between the circumscribed rectangle of the copper-colored area and the circumscribed rectangle of the black hole area. If there is an overlapping area (that is, the copper-colored area is connected to the black hole area), refer to Fig.10 , it can be determined that the pad corresponding to the pad area has a false soldering defect.

[0073] Fig.11 A schematic diagram of a pad having a copper skin defect according to an embodiment of the present invention is shown. In some embodiments, based on the copper-colored area in each pad area, it can be determined whether each pad of the PCB board has a copper skin defect in the following manner: for the copper-colored area in each pad area, firstly, the area ratio of the copper-colored area in the corresponding pad area can be determined, and it is determined whether the area ratio is greater than a third predetermined ratio and less than a second predetermined ratio. If the area ratio is greater than the third predetermined ratio and less than the second predetermined ratio, it can be determined that the pad corresponding to the pad area has a copper skin defect.

[0074] It should be noted that the third predetermined proportion is less than the second predetermined proportion, and the second predetermined proportion is less than the first predetermined proportion. The third predetermined proportion may be greater than or equal to 4%, and the first predetermined proportion may be 50%.

[0075] Finally, in step 270, the computing device 100 can determine whether each pad of the PCB board has a kneeling defect by detecting whether there is a pin area in the middle of each pad area in the HSV channel image (the pin area is a bright area with dark surroundings).

[0076] The kneeling foot is in the specified area (the middle area of ​​the pad). In an embodiment of the present invention, it can be determined whether the pad has a kneeling foot defect by judging whether there is a pin in the middle area of ​​the pad. If there is a pin in the middle area of ​​the pad, it is determined that the pad does not have a kneeling foot defect. Conversely, if there is no pin in the middle area of ​​the pad, it is determined that the pad has a kneeling foot defect.

[0077] It should be noted that under the illumination of three colors of light, due to the existence of the pin, there will be a slope around it and it will appear dark blue, and because the surface of the pin is flat, it will appear bright red. Therefore, the area with a bright center and dark surroundings in each pad area of ​​the HSV channel image can be determined as the pin area. In other words, in the embodiment of the present invention, the pin area is the area with a bright center and dark surroundings.

[0078] Based on this, in step 270, if it is detected from the HSV channel image that there is a pin area with a bright center and dark surroundings in the middle of the pad area, it can be determined that the pad corresponding to the pad area does not have a kneeling defect. Conversely, if it is detected from the HSV channel image that there is no pin area with a bright center and dark surroundings in the middle of the pad area, it can be determined that the pad corresponding to the pad area has a kneeling defect. Here, Fig.12 A schematic diagram showing a kneeling defect on a pad according to an embodiment of the present invention is shown.

[0079] In some embodiments, the computing device 100 can detect whether there is a pin area in the middle of each pad area in the HSV channel image in the following manner: First, the HSV channel image can be converted into a grayscale image, and the grayscale image can be binarized to obtain a binary image. Furthermore, it can be detected whether there is a bright area in the middle of each pad area in the binary image, and whether there is a dark area with an average brightness value lower than the brightness threshold around the bright area. If it is detected that there is a bright area in the middle of the pad area and there is a dark area around the bright area, it is determined that there is a pin area in the middle of the pad area, and the pad corresponding to the pad area does not have a kneeling foot defect. Conversely, if it is not detected that there is a bright area in the middle of the pad area and there is a dark area around the bright area, it is determined that there is no pin area in the middle of the pad area, and the pad corresponding to the pad area has a kneeling foot defect.

[0080] In one implementation, the grayscale image can be binarized in the following manner: all grayscale values ​​greater than 140 in the grayscale image are set to 255, and all grayscale values ​​less than 140 in the grayscale image are set to 0, thereby obtaining a binary image. Furthermore, the outline of the bright area located in the middle of the pad area can be extracted from the binary image, and it is determined whether the average brightness value of the surrounding area of ​​the bright area outline (for example, the area extending outward by 10 pixels) is less than the brightness threshold. If the average brightness value of the surrounding area of ​​the bright area outline is less than the brightness threshold, it is determined that there is a dark area around the bright area, so that it can be determined that the pad corresponding to the pad area does not have a kneeling defect. Conversely, if the average brightness value of the surrounding area of ​​the bright area outline is greater than or equal to the brightness threshold, it is determined that there is no dark area around the bright area, so that it can be determined that the pad corresponding to the pad area has a kneeling defect.

[0081] It can be understood that the dark area in the embodiment of the present invention refers to an area where the average brightness value is lower than the brightness threshold. In one embodiment, the brightness threshold can be, for example, 100. However, it should be understood that the brightness threshold can be set by those skilled in the art according to actual detection requirements, and the present invention does not limit the specific value of the brightness threshold.

[0082] In addition, in some embodiments of the present invention, after the various defects of the PCB board are detected, the various defects existing in the PCB board can be annotated in the image annotation file to be detected (including annotating the defect type and defect position) to obtain the defect detection result. It can be understood that the defect types existing in the PCB detected according to the present invention can include one or more of short circuit defects, insufficient tin defects, empty soldering defects, false soldering defects, copper skin defects, and kneeling defects.

[0083] Fig.13 A schematic diagram of a PCB board defect detection device 1300 provided according to an embodiment of the present invention is shown. The PCB board defect detection device 1300 can be deployed in a computing device 100, and the PCB board defect detection device 1300 is configured to execute the PCB board defect detection method 200 based on the HSV color space of the present invention.

[0084] like Fig.13 As shown, in an embodiment of the present invention, a PCB board defect detection device 1300 includes an acquisition unit 1310, a matching unit 1320, a conversion unit 1330, a first detection unit 1340, a second detection unit 1350, a third detection unit 1360, and a fourth detection unit 1370 which are sequentially communicated and connected.

[0085] The acquisition unit 1310 is used to acquire the image to be inspected, the image of the PCB board to be inspected under three-color light, the image of the color extraction of the low-tin, and the template image.

[0086] The matching unit 1320 is used to match feature points of the image to be detected with the template image, so as to correct the coordinates of the image to be detected based on the template image and obtain a labeling file of the image to be detected.

[0087] The conversion unit 1330 is used to convert the to-be-detected image annotation file from RGB channels to HSV channels based on the HSV color space distribution rule to obtain an HSV channel image.

[0088] The first detection unit 1340 is used to extract the area between adjacent pads from the HSV channel image, and determine whether the PCB board has a short circuit defect according to the color distribution of the area between the adjacent pads.

[0089] The second detection unit 1350 is used to compare the HSV channel image with the low-tin color extraction image to determine whether each pad of the PCB board has a low-tin defect.

[0090] The third detection unit 1360 is used to extract the copper-colored area in each pad area from the HSV channel image, and determine whether each pad of the PCB board has an empty soldering defect, a false soldering defect, or a copper skin defect based on the copper-colored area in each pad area.

[0091] The fourth detection unit 1370 is used to detect whether there is a pin area with a bright center and dark surroundings in the middle of each pad area in the HSV channel image, so as to determine whether there is a kneeling defect in each pad of the PCB board.

[0092] It should be noted that the acquisition unit 1310, the matching unit 1320, the conversion unit 1330, the first detection unit 1340, the second detection unit 1350, the third detection unit 1360, and the fourth detection unit 1370 are respectively used to execute the aforementioned steps 210 to 270. Here, the specific execution logic of each unit can be found in the description of steps 210 to 270 in the method 200 above, which will not be repeated here.

[0093] According to the PCB board defect detection method 200 based on the HSV color space in the embodiment of the present invention, first, the image to be detected, the color extraction image of the PCB board to be detected and the template image are obtained under three-color light, the coordinate correction of the image to be detected is performed based on the template image to obtain the image annotation file to be detected, and the image annotation file to be detected is converted into the HSV channel image based on the HSV color space. Then, it is determined whether the PCB board has a short circuit defect according to the color distribution of the area between adjacent pads. By comparing the HSV channel image with the color extraction image of the tin shortage, it is determined whether each pad of the PCB board has a tin shortage defect; by extracting the copper color area in each pad area from the HSV channel image, it is determined whether each pad of the PCB board has an empty soldering defect, a false soldering defect, and a copper skin defect based on the copper color area in each pad area; and by detecting whether there is a pin area with a bright color in the middle and a dark color around in the middle of each pad area in the HSV channel image, it is determined whether each pad of the PCB board has a kneeling defect. Based on this, the image based on the HSV color space of the present invention can more intuitively present various defects in the PCB board, and then can more accurately and comprehensively detect various defects in the PCB board, with better robustness and higher detection efficiency. In addition, the PCB board defect detection according to the present invention does not need to rely on training data, has low sensitivity to parameters, and is easy to adjust the threshold according to actual detection requirements.

[0094] The various techniques described herein may be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or some aspects or parts of the method and apparatus of the present invention may be in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present invention.

[0095] In the case where the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code; the processor is configured to execute the PCB board defect detection method based on the HSV color space of the present invention according to the instructions in the program code stored in the memory.

[0096] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer readable instructions, data structures, program modules or other data. Communication media generally embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transfer medium. Any combination of the above is also included within the scope of readable media.

[0097] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the examples of the present invention. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to implement the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0098] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0099] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0100] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0101] Unless otherwise specified, the use of ordinal numbers "first", "second", "third", etc. to describe common objects merely indicates that different instances of similar objects are involved and is not intended to imply that the objects so described must have a given order in time, space, order or in any other manner.

Claims

1. A PCB board defect detection method based on HSV color space is performed in a computing device, wherein the PCB board has a plurality of pads for soldering, and the method comprises: Obtain the image to be inspected, the image of the color extraction of the PCB board to be inspected, and the template image; Matching feature points of the image to be detected with the template image, so as to correct the coordinates of the image to be detected based on the template image, and obtain a labeling file of the image to be detected; Based on the HSV color space distribution law, the image annotation file to be detected is converted from RGB channel to HSV channel to obtain an HSV channel image; Extracting the area between adjacent pads from the HSV channel image, and determining whether the PCB board has a short circuit defect according to the color distribution of the area between the adjacent pads; Comparing the HSV channel image with the low-tin color extraction image to determine whether each pad of the PCB board has a low-tin defect; Extracting the copper-colored area in each pad area from the HSV channel image, and determining whether each pad of the PCB board has an empty soldering defect, a false soldering defect, or a copper skin defect based on the copper-colored area in each pad area; It is detected whether there is a pin area with a bright center and dark surrounding in the middle of each pad area in the HSV channel image, so as to determine whether there is a kneeling defect in each pad of the PCB board.

2. The method of claim 1, wherein: Extracting the area between adjacent pads from the HSV channel image includes: Extracting a pad densely distributed area from the HSV channel image, wherein the pad densely distributed area includes a plurality of pad areas that are densely distributed; Based on the short circuit mask image, the plurality of pad regions are removed from the pad densely distributed region to obtain regions between adjacent pads.

3. The method according to claim 1 or 2, wherein: Determining whether the PCB board has a short circuit defect according to the color distribution of the area between the adjacent pads includes: Determine whether there is color distribution in the area between the adjacent pads; If there is a color distribution, it is determined that there is a short circuit defect in the area between adjacent pads of the PCB board, and a red, green and blue three-color image is extracted from the HSV channel image, a threshold segmentation is performed on the red, green and blue three-color image to obtain a threshold segmentation image, and a contour extraction is performed on the threshold segmentation image to extract the short circuit defect area; If there is no color distribution, it is determined that the PCB board does not have a short circuit defect.

4. The method according to any one of claims 1 to 3, wherein: Comparing the HSV channel image with the insufficient tin color extraction image to determine whether each pad of the PCB board has insufficient tin defects, including: Extracting a plurality of pad areas from the HSV channel image as a plurality of comparison areas; For each of the comparison regions, determining a mean square error value between the comparison region and a corresponding region in the tin-poor color extraction image; If the mean square error value is greater than the mean square error threshold, a first red area is extracted from the comparison area, and the first red area is compared with a corresponding second red area in the low-tin color extraction image to determine whether there is a low-tin defect in the pad corresponding to the comparison area.

5. The method of claim 4, wherein: Comparing the first red area with the corresponding second red area in the low-tin color extraction image to determine whether a solder pad corresponding to the comparison area has a low-tin defect, including: Calculate the difference between the first red area and the second red area; If the red pixel difference between the first red area and the second red area is greater than the preset pixel difference, it is determined that there is a tin deficiency defect in the pad corresponding to the comparison area, and the first red area is determined as a tin deficiency defect area.

6. The method according to any one of claims 1 to 5, wherein: Extracting the copper color area in each pad area from the HSV channel image includes: The regions where the H channel values ​​are distributed between 3 and 8 in each pad region are extracted from the HSV channel image to obtain the copper-colored regions in each pad region.

7. The method according to any one of claims 1 to 6, wherein: Based on the copper-colored areas in each pad area, determining whether each pad of the PCB board has a hollow soldering defect, a false soldering defect, or a copper skin defect, including: For the copper-colored area in each pad area, determine the area ratio of the copper-colored area in the corresponding pad area, if the area ratio is greater than a first predetermined ratio, and there is a black ring near the pin of the pad area, it is determined that the pad corresponding to the pad area has an empty soldering defect; Determine whether there is a black void area between the copper area in the pad area and the pin of the pad area, and if there is a black void area, further determine whether there is an overlapping area between the circumscribed rectangle of the copper area and the circumscribed rectangle of the black void area, and if there is an overlapping area, determine that the pad corresponding to the pad area has a false soldering defect; If the area ratio is greater than the third predetermined ratio and less than the second predetermined ratio, it is determined that the pad corresponding to the pad area has a copper foil defect, wherein the third predetermined ratio is less than the second predetermined ratio, and the second predetermined ratio is less than the first predetermined ratio.

8. The method according to any one of claims 1 to 7, wherein: Detecting whether there is a pin area with a bright center and dark surroundings in the middle of each pad area in the HSV channel image, so as to determine whether each pad of the PCB board has a kneeling defect, including: Convert the HSV channel image into a grayscale image, and perform binarization on the grayscale image to obtain a binarized image; Detect whether there is a bright area in the middle of each pad area in the binary image, and whether there is a dark area with an average brightness value lower than a brightness threshold around the bright area; If it is detected that there is a bright area in the middle of the pad area and a dark area around the bright area, it is determined that there is a pin area in the middle of the pad area, and the pad corresponding to the pad area does not have a kneeling defect; If no bright area is detected in the middle of the pad area and a dark area is detected around the bright area, it is determined that no pin area exists in the middle of the pad area and a pad corresponding to the pad area has a kneeling defect.

9. A PCB defect detection device, deployed in a computing device, wherein the PCB has a plurality of pads for soldering, the device comprising: An acquisition unit, adapted to acquire an image to be inspected, a low-tin color extraction image, and a template image of the PCB board to be inspected; A matching unit, adapted to match feature points of the image to be detected with the template image, so as to correct the coordinates of the image to be detected based on the template image, and obtain a labeling file of the image to be detected; A conversion unit, adapted to convert the image annotation file to be detected from RGB channels to HSV channels based on the HSV color space distribution law to obtain an HSV channel image; A first detection unit is adapted to extract an area between adjacent pads from the HSV channel image, and determine whether the PCB board has a short circuit defect according to a color distribution of the area between the adjacent pads; A second detection unit is adapted to compare the HSV channel image with the low-tin color extraction image to determine whether each pad of the PCB board has a low-tin defect; A third detection unit is adapted to extract the copper-colored area in each pad area from the HSV channel image, and determine whether each pad of the PCB board has an empty soldering defect, a false soldering defect, or a copper skin defect based on the copper-colored area in each pad area; The fourth detection unit is adapted to detect whether there is a pin area with a bright center and dark surroundings in the middle of each pad area in the HSV channel image, so as to determine whether there is a kneeling defect in each pad of the PCB board.

10. A computing device comprising: at least one processor; and A memory storing program instructions, wherein the program instructions are configured to be processed by the at least one processor, and the program instructions include instructions for processing the method according to any one of claims 1 to 8.

Images