A computer vision-based circuit board inspection system and method

Abstract

This application relates to a computer vision-based circuit board inspection system and method. The computer vision-based circuit board inspection system includes a gripping device and a computer vision inspection device. The gripping device includes a main body, a picking mechanism and an inspection box mounted on the main body, with the picking mechanism slidably connected to the main body. The picking mechanism grips and holds the circuit board and transports it into the inspection box. The inspection box acquires images of the circuit board and sends them to the computer vision inspection device. The computer vision inspection device uses deep learning algorithms to inspect the images and obtain defect detection results for the circuit board. This application can improve the overall efficiency of replacing circuit boards to be inspected, improve the efficiency of the entire production process, and effectively identify circuit board defects.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and in particular to a circuit board inspection system and method based on computer vision. Background Technology

[0002] Printed circuit boards (PCBs) are a core component of numerous underlying hardware circuits, and their quality determines the stability of the overall hardware system. With the continuous evolution of manufacturing processes, PCBs exhibit richer colors and greater background complexity, making defect detection more difficult. Although workers may notice some minor issues when inspecting newly manufactured PCBs, these defects are difficult to accurately identify with the naked eye. Traditional PCB inspection methods, including visual inspection and bed-of-nails testing, are widely used, but these methods are labor-intensive and inefficient. Furthermore, under current technological conditions, the process of picking and placing PCBs is relatively cumbersome, requiring manual intervention for clamping and placement. This not only affects the efficiency of replacing PCBs to be inspected but also limits the number of products that inspection equipment can inspect in the same amount of time. Thus, this not only reduces the efficiency of the production process but may also adversely affect the inspection results.

[0003] Therefore, the inventors provide a circuit board inspection system and method based on computer vision. Summary of the Invention

[0004] (1) Technical problems to be solved

[0005] This application provides a computer vision-based circuit board inspection system and method. The technical problem to be solved is that traditional circuit board inspection methods are labor-intensive and inefficient, and the process of picking and placing circuit boards is relatively cumbersome, which reduces the efficiency of the production process.

[0006] (2) Technical solution

[0007] In a first aspect, this application provides a computer vision-based circuit board inspection system, including a gripping device and a computer vision inspection device; the gripping device includes a main body, and a picking mechanism and an inspection box disposed on the main body, the picking mechanism being slidably connected to the main body; the picking mechanism is used to grip and hold the circuit board and transport the circuit board into the inspection box; the inspection box is used to acquire images of the circuit board and send the images to the computer vision inspection device; the computer vision inspection device is used to detect defects in the images using deep learning algorithms to obtain defect detection results for the circuit board.

[0008] Furthermore, the retrieval mechanism includes a back plate and a clamping plate; the back plate is slidably connected to the main body for transporting the circuit board into the testing box; the clamping plate is connected to the back plate for gripping and holding the circuit board.

[0009] Furthermore, the object retrieval mechanism also includes a first hydraulic rod and a second hydraulic rod; the first hydraulic rod is disposed on the clamping plate and is used to adjust the distance between the two sides of the clamping plate; the clamping plate is connected to the back plate through the second hydraulic rod, and the second hydraulic rod is used to control the extension of the clamping plate.

[0010] Furthermore, the object retrieval mechanism also includes an electromagnet block; the electromagnet block is disposed on the clamping plate and is used to attract the circuit board.

[0011] Furthermore, the object retrieval mechanism also includes a first motor, a column, and a first placement plate; the first placement plate is connected to the column and is used to place the circuit board; the column is driven to rotate by the first motor to rotate the first placement plate directly below the clamping plate.

[0012] Furthermore, a camera is installed at the top inside the testing box to capture images of the circuit board.

[0013] Furthermore, lighting lamps are installed on both sides inside the testing chamber to illuminate the interior of the testing chamber.

[0014] Furthermore, the testing box is equipped with a lifting mechanism, which includes a second motor, a first gear, a second gear, a threaded rod, a slider, and a second placement plate. The first gear is driven to rotate by the second motor. The second gear meshes with the first gear and is driven to rotate by the first gear. The threaded rod is connected to the second gear and is driven to rotate by the second gear. The slider is mounted on the threaded rod and is driven to slide up and down by the threaded rod. The second placement plate is connected to the slider and is used to place the circuit board, and is driven to move up and down by the slider.

[0015] Furthermore, the computer vision inspection device includes an image acquisition module, an image preprocessing module, an image recognition module, and an image output module; the image acquisition module is used to acquire images of the circuit board collected by the inspection box; the image preprocessing module is used to preprocess the images; the image recognition module is used to detect the images using deep learning algorithms to obtain the defect detection results of the circuit board; and the image output module is used to output the defect detection results.

[0016] Secondly, this application provides a computer vision-based circuit board inspection method, implemented based on the computer vision-based circuit board inspection system described above, comprising:

[0017] The retrieval mechanism grabs and clamps the circuit board, and then moves the circuit board into the testing box;

[0018] The inspection box acquires images of the circuit board and sends the images to the computer vision inspection device;

[0019] Computer vision inspection devices use deep learning algorithms to detect defects in images and obtain the results of circuit board defect detection.

[0020] (3) Beneficial effects

[0021] The above-mentioned technical solution of this application has the following advantages:

[0022] The computer vision-based circuit board inspection system provided in the first aspect of this application is equipped with a picking mechanism. This structure enables the circuit board to be clamped and fixed and transported into the inspection box, thereby improving the overall efficiency of replacing the circuit board to be inspected and improving the efficiency of the entire production process. Through deep learning algorithms, defects in the circuit board can be effectively identified.

[0023] It is understandable that the beneficial effects of the second aspect mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 A schematic diagram of the computer vision-based circuit board inspection system provided in this application;

[0026] Figure 2 A schematic diagram of the computer vision-based circuit board inspection system provided in this application;

[0027] Figure 3 A schematic diagram of the overall structure of the gripping device provided in this application;

[0028] Figure 4 A schematic diagram of the internal structure of the gripping device provided in this application;

[0029] Figure 5 This is a schematic diagram of the internal structure of the gripping device provided in this application.

[0030] Reference numerals: 1. Body; 2. Detection box; 3. Back plate; 4. Clamping plate; 5. First hydraulic rod; 6. Second hydraulic rod; 7. Electromagnetic block; 8. First motor; 9. Column; 10. First shelf; 11. Camera; 12. Lighting lamp; 13. Second motor; 14. First gear; 15. Second gear; 16. Threaded rod; 17. Slider; 18. Second shelf. Detailed Implementation

[0031] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0032] It should be understood that, when used in this application specification and appended claims, the term "comprising" indicates the presence of the described feature, integral, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, in the description of this application specification and appended claims, the terms "first," "second," "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0033] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0034] With the emergence of computer vision technology, related technologies have gradually been introduced into the field of circuit board inspection. Currently, infrared thermal imaging technology and deep learning-based methods are widely used in circuit board inspection. However, infrared thermal imaging technology faces high cost challenges in terms of equipment investment and algorithm development. In comparison, deep learning technology is less expensive and has higher detection accuracy. It is worth noting that using computer vision methods for circuit board inspection requires ensuring that the circuit board is placed flat. Improper placement or obstruction by other objects may lead to false detections, thus affecting the accuracy of the inspection. Therefore, appropriate gripping and placement devices play a crucial role in this process.

[0035] Currently, printed circuit board (PCB) defect detection faces two major challenges. First, defects on PCBs are often tiny, making it difficult to extract sufficient features for detection with the naked eye. Second, to monitor PCBs in real-time on-site, a PCB inspection device that is fast, cost-effective, and highly accurate is needed. Therefore, appropriate gripping and placement devices are essential in PCB inspection.

[0036] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0037] like Figures 1 to 5 As shown, the computer vision-based circuit board inspection system provided in this embodiment includes a gripping device and a computer vision inspection device. The gripping device includes a body 1, a picking mechanism and an inspection box 2 disposed on the body 1, and the picking mechanism is slidably connected to the body 1. The picking mechanism is used to grip and hold the circuit board and transport the circuit board into the inspection box 2. The inspection box 2 is used to acquire images of the circuit board and send the images to the computer vision inspection device. The computer vision inspection device is used to detect the images through deep learning algorithms to obtain the defect detection results of the circuit board.

[0038] In some embodiments, the picking mechanism includes a back plate 3 and a clamping plate 4; the back plate 3 is slidably connected to the body 1 for transporting the circuit board into the testing box 2; the clamping plate 4 is connected to the back plate 3 for gripping and clamping the circuit board.

[0039] In some embodiments, the object retrieval mechanism further includes a first hydraulic rod 5 and a second hydraulic rod 6; the first hydraulic rod 5 is disposed on the clamping plate 4 and is used to adjust the distance between the two sides of the clamping plate 4; the clamping plate 4 is connected to the back plate 3 through the second hydraulic rod 6, and the second hydraulic rod 6 is used to control the extension of the clamping plate 4.

[0040] In some embodiments, the object-grabbing mechanism further includes an electromagnet block 7; the electromagnet block 7 is disposed on the clamping plate 4 and is used to adsorb the circuit board.

[0041] In some embodiments, the object retrieval mechanism further includes a first motor 8, a column 9, and a first placement plate 10; the first placement plate 10 is connected to the column 9 and is used to place the circuit board; the column 9 is driven to rotate by the first motor 8 and is used to rotate the first placement plate 10 to directly below the clamping plate 4.

[0042] In some embodiments, a camera 11 is disposed above the inside of the testing box 2, and the camera 11 is used to acquire images of the circuit board.

[0043] In some embodiments, lighting lamps 12 are provided on both sides inside the test box 2, and the lighting lamps 12 are used to illuminate the inside of the test box 2.

[0044] In some embodiments, the testing box 2 is further provided with a lifting mechanism, which includes a second motor 13, a first gear 14, a second gear 15, a threaded rod 16, a slider 17, and a second placement plate 18; the first gear 14 is driven to rotate by the second motor 13; the second gear 15 meshes with the first gear 14 and is driven to rotate by the first gear 14; the threaded rod 16 is connected to the second gear 15 and is driven to rotate by the second gear 15; the slider 17 is disposed on the threaded rod 16 and is driven to slide up and down by the threaded rod 16; the second placement plate 18 is connected to the slider 17, is used to place the circuit board, and is driven to move up and down by the slider 17.

[0045] In some embodiments, the computer vision inspection device includes an image acquisition module, an image preprocessing module, an image recognition module, and an image output module; the image acquisition module is used to acquire images of the circuit board collected by the inspection box; the image preprocessing module is used to preprocess the images; the image recognition module is used to detect the images using deep learning algorithms to obtain defect detection results of the circuit board; and the image output module is used to output the defect detection results.

[0046] In the application, the computer vision-based circuit board inspection system includes a gripping device and a computer vision inspection device. The gripping device includes a main body 1, a picking mechanism, and an inspection box 2. The picking mechanism includes a back plate 3, a first hydraulic rod 5 and a second hydraulic rod 6, a clamping plate 4, an electromagnetic module, a first motor 8, a column 9, and a first placement plate 10. The inspection box 2 contains a camera 11, a lighting lamp 12, and a lifting mechanism. The lifting mechanism includes a second motor 13, a first gear 14, a second gear 15, a threaded rod 16, a slider 17, and a second placement plate 18. The computer vision inspection device can be a computer with software installed, including modules for image acquisition, image preprocessing, image recognition, and image output. Image acquisition involves capturing images using the camera 11, transmitting and storing the captured images in the computer, and then performing preprocessing operations, including binarization and image segmentation. Image recognition uses the YOLO V8 algorithm with an attention mechanism. Image output includes outputting the results of defect identification.

[0047] Specifically, the gripping device includes a main body 1, a gripping mechanism, and a detection box 2. The detection box 2 is fixedly connected to the top of the main body 1. A camera 11 is located inside the detection box 2 from the top. Illumination lamps 12 are fixedly connected to both sides inside the detection box 2. During detection, considering the problem of insufficient light, illumination lamps 12 are provided to illuminate the inside of the detection box 2, providing sufficient light for more accurate detection results. To make the gripping and detection of circuit boards more efficient and accurate, this application designs a gripping mechanism. This gripping mechanism is slidably connected to the top of the main body 1 and has the following characteristics and components:

[0048] Backplate 3: This is the main part of the retrieval mechanism. It slides above the main body 1 and is responsible for transporting the circuit board to the detection box 2. Clamping plate 4: The distance between its two sides can be adjusted by the first hydraulic rod 5 to accommodate circuit boards of different sizes, ensuring stable clamping. The clamping plate 4 is fixedly connected to the extension end of the second hydraulic rod 6. Second hydraulic rod 6: Fixed to one side of the backplate 3, it controls the clamping plate 4 to extend forward so that the circuit board can be stably placed above the second placement plate 18. The distance between the camera 11 and the second placement plate 18 can be adjusted according to the thickness of the circuit board to ensure optimal detection results. Electromagnetic block 7: Fixedly connected to the clamping plate 4. When energized, it attracts the circuit board, further enhancing clamping stability. First motor 8 and column 9: The first motor 8 is fixedly connected to one side of the main body 1, and its output end is fixedly connected to the column 9. First placement plate 10: Fixedly connected to the column 9. In use, items to be detected, such as circuit boards, are placed on top of it. After the first motor 8 is started, it controls the column 9 to rotate, thereby causing the first shelf 10 to change angle until it rotates to be directly below the clamping plate 4. This design structure ensures the stability and accuracy of the circuit board during the picking, clamping, and testing processes.

[0049] To achieve the lifting function, this application incorporates a lifting mechanism inside the testing box 2. The composition and principle of this mechanism are as follows:

[0050] Second motor 13: Fixedly connected to both sides inside the detection box 2. First gear 14: Fixedly connected to the output end of the second motor 13. Second gear 15: Meets with the first gear 14. When the second motor 13 starts, it controls the rotation of the first gear 14, thereby driving the second gear 15. Threaded rod 16: Fixedly connected to the central shaft of the second gear 15. Therefore, when the second gear 15 rotates, the threaded rod 16 also rotates. Slider 17: This is a part that passes through and is threadedly connected to the threaded rod 16, ensuring synchronous rotation with the threaded rod 16. Second shelf 18: Fixedly connected to the slider 17. When the slider 17 moves, it can drive the second shelf 18 to move arbitrarily in the vertical direction. This design structure ensures that the second shelf 18 can be raised and lowered smoothly and accurately in the vertical direction.

[0051] Before the inspection begins, the conveyor places the item to be inspected (e.g., a circuit board) above the first shelf 10. After the first motor 8 is started, it drives the column 9 to rotate, causing the first shelf 10 to change angle until it is directly below the clamping plate 4. At this point, the electromagnet 7 is energized, attracting the circuit board. To ensure stable clamping of the circuit board, the distance between the two sides of the clamping plate 4 is adjusted by the first hydraulic rod 5 to match the size of the circuit board. The back plate 3 begins to slide above the body 1, moving the circuit board to the position of the inspection box 2. Immediately afterwards, the second hydraulic rod 6 controls the clamping plate 4 to extend forward, allowing the circuit board to be stably placed on the second shelf 18. The distance between the camera 11 and the second shelf 18 can be adjusted according to the thickness of the circuit board. After the second motor 13 is started, the first gear 14 begins to rotate, driving the meshing second gear 15, causing the threaded rod 16 to rotate. This allows the slider 17 to move the second shelf 18 vertically. To ensure the accuracy of the test, when there is insufficient light, a lighting lamp 12 is installed to illuminate the inside of the test chamber 2, ensuring sufficient light during the test process and thus obtaining more accurate test results.

[0052] The overall workflow of the computer vision-based circuit board inspection system provided in this application is as follows: the gripping device grips the circuit board, the camera captures the image, the software processes and recognizes the image, the software outputs the results, the software is installed on a computer device, and the results are saved locally on the computer.

[0053] Regarding the software algorithm, a dataset containing 1000 images of printed circuit boards with various defects was used. Based on the authoritative and widely used IPC standard, five defect types were selected for identification: open circuit, short circuit, notch, fake copper, and via. In the image recognition stage, YOLO v8 is a single-stage object detection algorithm. The YOLO v8 network model mainly consists of four parts: input, backbone, neck, and head. Its input includes an image preprocessing stage, scaling the input image to the network's input size and performing normalization. To improve model accuracy and performance, an attention mechanism is introduced, which helps the model focus more on key regions in the image, thereby improving detection accuracy. Furthermore, to make the model lighter and accelerate inference, this application also employs model quantization technology. This not only improves the accuracy of YOLO v8 but also makes it more suitable for resource-constrained devices. The model quantization method involves compressing the model by reducing the number of bits in the weights and activation representations, reducing the model from 32-bit floating-point to 8-bit integer. This model algorithm achieved an accuracy of 97% after training.

[0054] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application.

[0055] This embodiment also provides a computer vision-based circuit board inspection method, implemented based on the computer vision-based circuit board inspection system described in the above embodiment, including: a picking mechanism grasping and clamping the circuit board and transporting it into the inspection box; the inspection box acquiring an image of the circuit board and sending the image to a computer vision inspection device; the computer vision inspection device using a deep learning algorithm to detect the image and obtain the defect detection result of the circuit board.

[0056] The circuit board inspection system and method based on computer vision provided in this application includes a picking mechanism. The clamping plate and electromagnet in this structure clamp and fix the circuit board. A first placement plate and a first motor transfer the circuit board to the clamping plate. After the first motor starts, the column rotates, the first placement plate rotates to directly below the clamping plate, the electromagnet is energized, and the circuit board is attracted. Control and adjustment of the first hydraulic rod can make the circuit board clamped more stable. This application can improve the overall efficiency of replacing circuit boards to be inspected and improve the efficiency of the entire production process. This application includes a printed circuit board defect detection software based on an improved YOLO V8 algorithm, which can effectively identify five types of defects in printed circuit boards: open circuit, short circuit, notch, false copper, and via.

[0057] Those skilled in the art will recognize that the device and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0058] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A circuit board inspection system based on computer vision, characterized in that, The device includes a gripping device and a computer vision inspection device. The gripping device includes a main body, a picking mechanism and an inspection box disposed on the main body, and the picking mechanism is slidably connected to the main body. The picking mechanism is used to grip and hold a circuit board and transport the circuit board into the inspection box. The inspection box is used to acquire an image of the circuit board and send the image to the computer vision inspection device. The computer vision inspection device is used to detect defects in the image using a deep learning algorithm to obtain the defect detection result of the circuit board. A camera is installed at the top inside the testing box to capture images of the circuit board. The testing box also includes a lifting mechanism comprising a second motor, a first gear, a second gear, a threaded rod, a slider, and a second shelf. The first gear is driven to rotate by the second motor. The second gear meshes with the first gear and is driven to rotate by the first gear; The threaded rod is connected to the second gear and is driven to rotate by the second gear; The slider is mounted on the threaded rod and is driven by the threaded rod to slide up and down; the second shelf is connected to the slider and is used to place the circuit board, and is driven by the slider to move up and down; the lifting mechanism is used to adjust the distance between the camera and the second shelf; The retrieval mechanism includes a back plate and a clamping plate; the back plate is slidably connected to the main body for transporting the circuit board into the testing box; the clamping plate is connected to the back plate for gripping and holding the circuit board; the retrieval mechanism also includes a first hydraulic rod and a second hydraulic rod; the first hydraulic rod is disposed on the clamping plate for adjusting the distance between the two sides of the clamping plate to accommodate circuit boards of different sizes; the clamping plate is connected to the back plate via the second hydraulic rod, and the second hydraulic rod is used to control the extension of the clamping plate so that the circuit board is stably placed on the second placement plate.

2. The computer vision-based circuit board inspection system as described in claim 1, characterized in that, The object retrieval mechanism also includes an electromagnet block; the electromagnet block is disposed on the clamping plate and is used to attract the circuit board.

3. The computer vision-based circuit board inspection system as described in claim 1, characterized in that, The object retrieval mechanism further includes a first motor, a column, and a first placement plate; the first placement plate is connected to the column and is used to place the circuit board; the column is driven to rotate by the first motor, which is used to rotate the first placement plate to directly below the clamping plate.

4. The computer vision-based circuit board inspection system as described in claim 1, characterized in that, Lights are installed on both sides inside the testing box to illuminate the interior of the testing box.

5. The computer vision-based circuit board inspection system as described in claim 1, characterized in that, The computer vision inspection device includes an image acquisition module, an image preprocessing module, an image recognition module, and an image output module. The image acquisition module is used to acquire images of the circuit board collected by the inspection box. The image preprocessing module is used to preprocess the images. The image recognition module is used to detect the images using a deep learning algorithm to obtain defect detection results for the circuit board. The image output module is used to output the defect detection results.

6. A circuit board inspection method based on computer vision, characterized in that, The circuit board inspection system based on computer vision as described in any one of claims 1 to 5 includes: The picking mechanism grabs and clamps the circuit board, and transports the circuit board into the testing box; The inspection box acquires images of the circuit board and sends the images to a computer vision inspection device; The computer vision inspection device uses a deep learning algorithm to detect defects in the image and obtain the defect detection results of the circuit board.

Images