Printed circuit board (PCB) positioning and multifunctional testing integrated device based on visual identification

Abstract

The invention provides a PCB positioning and multifunctional testing integrated device based on visual identification. The device comprises an integrated controller, an image processing unit, a visual identification unit, a PCB positioning unit, a PCB testing unit and a mechanical arm control unit. The image processing unit fuses the initial PCB images at different acquisition angles to obtain a fused PCB image; the visual identification unit determines the mark position of the positioning mark in the fused PCB image; the PCB positioning unit converts the initial attitude information and the initial position information to a world coordinate system according to a conversion relation between the image coordinate system and the world coordinate system to obtain target attitude information and target position information of the PCB in the world coordinate system; the PCB testing unit is used for carrying out multifunctional testing on the PCB; and the mechanical arm control unit conveys the PCB to the next process through the conveying belt. According to the invention, the positioning accuracy of the PCB and the testing efficiency of the PCB are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles (UAVs), and in particular to a PCB positioning and multifunctional testing integrated device based on visual recognition. Background Art

[0002] In modern electronics manufacturing, integrated methods for high-precision PCB board positioning and multi-functional testing are crucial for ensuring product quality and improving production efficiency. Currently, the widely used integrated methods mostly rely on contact sensors or a single mechanical positioning method. Contact sensors require direct contact with the PCB board to obtain positioning information. However, the contact process takes a long time, and there are subjective deviations in the positioning information obtained through contact, which reduces the positioning accuracy of the PCB board and the testing efficiency of the PCB board. A single mechanical positioning method is limited by a fixed structure and preset parameters, making it difficult to accurately match the subtle differences between PCB boards of different batches and specifications. This results in large positioning deviations, reducing the positioning accuracy of the PCB board and the testing efficiency of the PCB board. Summary of the Invention

[0003] The present invention provides a PCB board positioning and multifunctional testing integrated device based on visual recognition, aiming to improve the positioning accuracy of the PCB board and the testing efficiency of the PCB board.

[0004] In a first aspect, the present invention provides an integrated device for PCB board positioning and multifunctional testing based on visual recognition, comprising an integrated controller, an image processing unit, a visual recognition unit, a PCB board positioning unit, a PCB board testing unit, and a robotic arm control unit; the integrated controller is respectively connected to the image processing unit, the visual recognition unit, the PCB board positioning unit, the PCB board testing unit, and the robotic arm control unit to manage each unit;

[0005] An image processing unit is configured to capture multi-angle images of the PCB board to be tested on the conveyor belt using an industrial camera, obtain at least three initial PCB board images at different acquisition angles, and fuse the initial PCB board images at different acquisition angles to obtain a fused PCB board image;

[0006] a visual recognition unit, configured to determine a marking position of the positioning mark on the PCB board identified in the fused PCB board image;

[0007] a PCB board positioning unit, configured to determine initial posture information and initial position information of the PCB board in the image coordinate system based on the mark positions and the geometric relationship, and convert the initial posture information and initial position information into the world coordinate system according to a conversion relationship between the image coordinate system and the world coordinate system, thereby obtaining target posture information and target position information of the PCB board in the world coordinate system;

[0008] A PCB board testing unit, configured to control a test probe carried by a robotic arm to perform a multifunctional test on the PCB board according to the target posture information and the target position information;

[0009] The robot control unit is used to control the robot to remove the test probe if the PCB board passes the test, and to convey the PCB board to the next process through the conveyor belt.

[0010] In a second aspect, the present invention further provides a method for integrating high-precision PCB board positioning and multifunctional testing based on visual recognition, which is implemented based on the device for integrating PCB board positioning and multifunctional testing based on visual recognition described in the first aspect, and the method comprises:

[0011] An industrial camera is used to capture images of the PCB to be tested on the conveyor belt from multiple angles, obtaining at least three initial PCB images at different acquisition angles. The initial PCB images at different acquisition angles are then fused to obtain a fused PCB image.

[0012] Based on the positioning mark on the PCB board identified in the fused PCB board image, determining a marking position of the positioning mark in the fused PCB board image;

[0013] Determining initial posture information and initial position information of the PCB board in the image coordinate system based on the mark position and the geometric relationship, and converting the initial posture information and initial position information to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system, to obtain target posture information and target position information of the PCB board in the world coordinate system;

[0014] Controlling a test probe carried by a robotic arm to perform a multifunctional test on a PCB board according to the target posture information and the target position information;

[0015] If the PCB board passes the test, the robot arm is controlled to move the test probe away and the PCB board is conveyed to the next process via the conveyor belt.

[0016] In a third aspect, the present invention also provides an electronic device comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby realizing any one of the above-mentioned methods for high-precision PCB board positioning and multi-functional testing based on visual recognition.

[0017] In a fourth aspect, the present invention also provides a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, it implements any one of the above-mentioned methods for high-precision PCB board positioning and multi-functional testing based on visual recognition.

[0018] In a fifth aspect, the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned high-precision PCB board positioning and multi-functional testing integrated methods based on visual recognition.

[0019] The embodiment of the present invention provides an integrated device for PCB board positioning and multi-functional testing based on visual recognition. On the one hand, by acquiring PCB board images from at least three different acquisition angles, the detailed features of the PCB board are captured from multiple perspectives, thereby avoiding positioning deviations caused by limited viewing angles. The positioning mark position is then determined by identifying the positioning mark in the image, and the posture information and position information are calculated in combination with the geometric relationship. Therefore, the positioning mark can be accurately identified, and the subtle differences between PCB boards of different batches and different specifications can be adapted to improve the positioning accuracy of the PCB board. On the other hand, non-contact visual recognition technology is used for positioning. After positioning is completed, the robot arm is controlled to carry a test probe to automatically perform multi-functional testing on the PCB board. The entire process does not require human intervention. The robot arm performs the test task quickly and orderly according to the program settings, avoiding the tedious operations and time loss in the contact sensor process, so that the positioning and testing links are closely connected, reducing waiting time and improving the testing efficiency of the PCB board. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic structural diagram of a PCB positioning and multifunctional testing integrated device based on visual recognition provided by the present invention;

[0021] Figure 2 This is a flow chart of the integrated method for high-precision PCB board positioning and multi-functional testing based on visual recognition provided by the present invention;

[0022] Figure 3 An embodiment diagram of an electronic device provided by an embodiment of the present invention;

[0023] Figure 4 An embodiment diagram of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0026] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0027] Optional, see Figure 1 As shown, Figure 1 This is a schematic diagram of the structure of the integrated device for PCB board positioning and multifunctional testing based on visual recognition provided by the present invention. The device includes an integrated controller, an image processing unit, a visual recognition unit, a PCB board positioning unit, a PCB board testing unit, and a robotic arm control unit. Optionally, the integrated controller in this embodiment of the present invention is connected to the image processing unit, the visual recognition unit, the PCB board positioning unit, the PCB board testing unit, and the robotic arm control unit to manage each unit.

[0028] Alternatively, an industrial camera can be positioned above or to the side of a conveyor belt. When the PCB to be tested moves along the conveyor belt to a designated location, the camera initiates a multi-angle image acquisition process. Using pre-set angle parameters, the camera sequentially captures images of the PCB from at least three different angles, generating an initial PCB image at each angle.

[0029] After the image processing unit acquires at least three initial PCB images from different acquisition angles, it uses an image fusion algorithm to fuse these initial PCB images, eliminating image information missing due to differences in viewing angles. This results in a fused PCB image containing complete PCB information, as described in steps 101 through 104. In one embodiment, three industrial cameras are deployed, positioned directly above the conveyor belt (0°), 45° to the left of the conveyor belt, and 45° to the right of the conveyor belt. When a PCB board enters the image acquisition area, the industrial camera directly above captures a top view of the PCB board as the first initial PCB board image. Next, the industrial camera at 45° to the left captures an image with side view information as the second initial PCB board image. Finally, the industrial camera at 45° to the right captures an image with different side view information as the third initial PCB board image.

[0030] Optionally, the visual recognition unit identifies a positioning mark on the PCB board in the fused PCB board image based on computer vision technology. The positioning mark is typically an identifier with a specific shape, color, or pattern, such as a circular mark or a cross mark. Therefore, the visual recognition unit searches for the positioning mark in the image through image recognition algorithms, such as template matching and feature point detection, and determines its precise coordinate position in the fused PCB board image, i.e., the mark position, as described in detail in steps 201 to 205.

[0031] In one embodiment, four circular positioning marks are provided on the PCB board, and the center position of each mark is used to determine the position and posture of the PCB board. If a template matching algorithm is used, a template image of the circular positioning marks is pre-stored. When processing the fused PCB board image, the template image is traversed and matched in the fused image. By calculating the similarity between the template and the image area, the area with the highest template match is found, and the position of the circular positioning marks is determined. After the four circular positioning marks are found, the pixel coordinates of the center of each mark in the fused PCB board image are recorded, such as the coordinates of Mark 1 are (x1, y1), the coordinates of Mark 2 are (x2, y2), the coordinates of Mark 3 are (x3, y3), and the coordinates of Mark 4 are (x4, y4). These coordinates are the marked positions of the positioning marks in the fused PCB board image.

[0032] Optionally, the PCB positioning unit calculates the PCB's initial posture information (e.g., rotation angle) and initial position information (e.g., center point coordinates) in the image coordinate system based on the positions of the positioning marks in the fused PCB image using geometric relationships. For example, the PCB's rotation angle can be determined by calculating the distance and angular relationship between the positioning marks; and the PCB's center point position in the image coordinate system can be determined by calculating the center coordinates of multiple positioning marks.

[0033] Furthermore, after obtaining the initial posture information and initial position information, the PCB board positioning unit converts the initial posture information and initial position information to the world coordinate system according to the conversion relationship between the pre-calibrated image coordinate system and the world coordinate system (such as calibrating the coordinate system through a calibration plate to obtain the rotation matrix and translation vector), and obtains the target posture information and target position information of the PCB board in the world coordinate system.

[0034] Continuing with the above embodiment, four circular positioning marks are determined. First, the distance d12 between mark 1 and mark 2, the distance d23 between mark 2 and mark 3, and the angle θ between the line connecting mark 1 and mark 2 and the horizontal axis are calculated. The rotation angle of the PCB board in the image coordinate system is calculated based on the geometric relationship. For example, the calculated rotation angle is α. Next, the center coordinates (x0, y0) of the four circular positioning marks are calculated as the initial position information of the PCB board in the image coordinate system. Then, the conversion relationship between the pre-calibrated image coordinate system and the world coordinate system is called, where the rotation matrix is R and the translation vector is T. Through the coordinate conversion formula: P world =R*P image +T, convert the initial posture information α and the initial position information (x0, y0) to the world coordinate system to obtain the target posture information (such as the rotation angle α' relative to the world coordinate system) and the target position information (such as the coordinates (x', y', z') in the world coordinate system) of the PCB board in the world coordinate system.

[0035] Optionally, the PCB test unit controls the movement of a robotic arm based on the PCB's target posture and position information in the world coordinate system. The robotic arm, equipped with test probes, performs multifunctional tests, such as electrical performance and signal integrity, on various test points on the PCB according to pre-set test procedures. During testing, test data is collected in real time, and the test results are judged as qualified according to test standards.

[0036] In one embodiment, there are 10 test points on the PCB board, which are used to test different electrical performance and signal integrity. The PCB board test unit controls the robotic arm to accurately move the test probe to the first test point based on the target posture information and target position information. The robotic arm is driven by a servo motor and adjusts its posture in real time during movement to ensure that the test probe is vertically aligned with the test point. When the test probe contacts the test point, the PCB board test unit begins to perform electrical performance testing, such as measuring parameters such as resistance and voltage. After the test is completed, the robotic arm moves the test probe to the next test point and repeats the above test process until the test of all 10 test points is completed. During the test process, the PCB board test unit compares the collected test data with the preset test standards to determine whether the test results of each test point are qualified.

[0037] Optionally, if all tests on the PCB board pass, the robotic arm control unit controls the robotic arm to move the test probe away to avoid interference with subsequent processes.

[0038] The robotic arm control unit then activates the conveyor belt to move the PCBs that pass the test to the next process. If the PCB fails the test, the robotic arm control unit controls the robotic arm to remove the PCB from the conveyor belt and place it in the defective product area for subsequent repair or scrapping.

[0039] In one embodiment, after all tests on a PCB are completed and the PCB is deemed to have passed, the robotic arm control unit reverses the motor of the robotic arm, lifting and removing the test probes. The robotic arm control unit then sends a start signal to the conveyor belt's drive motor, commencing the conveyor belt's operation and transporting the tested PCB to the next process step, such as the next assembly station. If, during the testing process, the test results of any test point fail to meet the test criteria, the robotic arm control unit controls the robotic arm to pick up the PCB and move it to a storage area for defective products, awaiting further processing by staff.

[0040] The embodiment of the present invention obtains PCB board images from at least three different acquisition angles, captures the detailed features of the PCB board from multiple perspectives, avoids positioning deviations caused by limited viewing angles, and then determines the mark position by identifying the positioning mark in the image, and calculates the posture information and position information in combination with the geometric relationship. Therefore, the positioning mark can be accurately identified, and it can adapt to the subtle differences between PCB boards of different batches and different specifications, thereby improving the positioning accuracy of the PCB board. On the other hand, non-contact visual recognition technology is used for positioning. After positioning is completed, the robot arm equipped with a test probe is controlled to automatically perform multi-function testing on the PCB board. The entire process does not require human intervention. The robot arm performs the test task quickly and orderly according to the program settings, avoiding the tedious operations and time loss in the contact sensor process, so that the positioning and testing links are closely connected, reducing waiting time and improving the testing efficiency of the PCB board.

[0041] In one embodiment, steps 101 to 104 are described as follows:

[0042] Step 101 : performing image segmentation based on the edge transparency channel threshold of each pixel in the initial PCB board image at each acquisition angle to obtain a target PCB board image at each acquisition angle.

[0043] Optionally, an edge transparency channel is used to identify whether a pixel is located at the edge or background of an image. The integrated device compares the edge transparency channel value of each pixel with a pre-set threshold. If the edge transparency channel value of a pixel is greater than the threshold, the pixel is determined to belong to the background or invalid edge area. If the edge transparency channel value of a pixel is less than or equal to the threshold, the pixel is determined to belong to the valid area of the PCB board. After determining all pixels, the pixels belonging to the valid area of the PCB board are retained, and the background and invalid edge areas are removed to obtain the target PCB board image for each acquisition angle.

[0044] Continuing with the above example, taking the first top view as an example, each pixel in the image contains edge transparency channel value information. For example, if the critical value is set to 0.5, and all pixels in the image are traversed, pixel A has an edge transparency channel value of 0.3, which is less than the critical value, so it is retained. Pixel B has an edge transparency channel value of 0.6, which is greater than the critical value, so it is discarded. By processing all pixels in the entire image, the first target PCB board image is obtained after removing background and edge interference. Similarly, the same operation is performed on the other two initial PCB board images to obtain the corresponding target PCB board images.

[0045] Step 102 : determining a pixel feature of each pixel in the target PCB board image at each acquisition angle based on the lens optical axis at each acquisition angle and each pixel in the target PCB board image.

[0046] Furthermore, the integrated device analyzes the position information of each pixel in the target PCB image based on the lens optical axis at each acquisition angle, combining it with the position information of each pixel in the target PCB image. The lens optical axis determines the light propagation direction and imaging characteristics. Therefore, the integrated device determines the pixel characteristics of each pixel by calculating the positional relationship of each pixel relative to the lens optical axis, such as parameters such as angle and distance. Specifically, as described in steps 1021 through 1024, the pixel characteristics include, but are not limited to, the pixel's spatial position, light incidence angle, and reflection characteristics, reflecting the pixel's properties in real space.

[0047] Step 103 , performing parallax relationship mapping based on camera parameters at different acquisition angles and pixel features of each pixel in the target PCB board image, to obtain parallax images between different acquisition angles.

[0048] Furthermore, the integrated device uses camera parameters (such as focal length, aperture, sensor size, etc.) of different acquisition angles, combined with the pixel features of each pixel in the target PCB board image, to analyze the position differences of the same object (components on the PCB board, etc.) in the image at different angles, calculates the disparity information between the pixels, and performs disparity relationship mapping on the disparity information between the pixels to generate disparity images between different acquisition angles, wherein the disparity image intuitively shows the difference and correspondence between images at different angles, as specifically described in steps 1031 to 1033.

[0049] Continuing with the above embodiment, the focal lengths of the industrial cameras for the three acquisition angles are f1, f2, and f3, respectively; the apertures are a1, a2, and a3, respectively; and the sensor sizes are s1, s2, and s3, respectively. Taking the top view and the left 45° view as examples, the integrated device calculates the disparity between the pixels at corresponding positions in the two images, such as pixel D in the top view and pixel E in the left 45° view, based on their pixel features and camera parameters. By calculating the disparity of all corresponding pixels in the two images, a disparity image is generated between the top view and the left 45° view. Similarly, disparity images are generated between the top view and the right 45° view, and between the left 45° view and the right 45° view.

[0050] Step 104 determines an image fusion level based on the angle difference between different acquisition angles, and fuses the parallax images between different acquisition angles according to the image fusion level between different acquisition angles to obtain a fused PCB board image.

[0051] Furthermore, the integrated device determines the image fusion level based on the angular difference between different acquisition angles. The greater the angular difference, the higher the fusion level, and the greater the degree of image processing and adjustment during the fusion process. Furthermore, the integrated device fuses the parallax images acquired from different acquisition angles based on the determined image fusion level. During the fusion process, algorithms such as weighted averaging and region-based fusion are used to integrate information from the different parallax images, eliminating image inconsistencies caused by perspective differences, ultimately resulting in a fused PCB image containing complete PCB information.

[0052] Continuing with the above embodiment, among the three acquisition angles, the angle difference between the top view and the left 45° view is 45°, the angle difference between the top view and the right 45° view is also 45°, and the angle difference between the left 45° view and the right 45° view is 90°. According to the preset rules, the fusion level of the top view and the left and right side views is determined to be intermediate, and the fusion level between the left and right side views is determined to be advanced. In the fusion process, for the disparity images of the top view and the left 45° view at the intermediate fusion level, a weighted average algorithm is used to assign appropriate weights to the two images for fusion based on image quality and information richness; for the disparity images of the left 45° view and the right 45° view at the advanced fusion level, a region-based fusion algorithm is used to perform targeted processing on different areas in the image before fusion. After the fusion operation of the three groups of disparity images, a complete fused PCB board image is finally obtained.

[0053] The embodiment of the present invention fully integrates information from images at different angles, eliminating inconsistencies and omissions in image information caused by viewing angle differences and avoiding positioning deviations due to viewing angle limitations. This allows the fused PCB image to fully and accurately present information from all angles of the PCB, providing a high-quality image data foundation for subsequent positioning mark recognition, posture, and position determination. This allows for precise identification of positioning marks, adapting to subtle differences between PCB boards of different batches and specifications, and improving PCB positioning accuracy, further effectively enhancing the accuracy and reliability of PCB testing.

[0054] In one embodiment, steps 1021 to 1024 are described as follows:

[0055] In step 1021, for each target PCB image captured at each acquisition angle, a three-dimensional coordinate system is established, with the lens optical axis as the Z axis and the plane perpendicular to the optical axis and passing through the image center as the XY plane. The original coordinates of each pixel in the target PCB image are mapped to the three-dimensional coordinate system to obtain the mapped coordinates of each pixel. The horizontal and vertical coordinates of the mapped coordinates are determined based on the original coordinates combined with the image resolution and the width and height of the imaging field of view on the XY plane. The vertical coordinate of the mapped coordinates is determined based on the imaging distance of the acquisition device.

[0056] Optionally, for each target PCB image captured at each angle, the integrated device establishes a three-dimensional coordinate system with the lens optical axis at that angle as the principal axis (Z axis). Simultaneously, the center of the target PCB image is located and a plane perpendicular to the optical axis and passing through the image center is constructed as the XY plane. After establishing the three-dimensional coordinate system, the integrated device maps the original coordinates of each pixel in the target PCB image to the three-dimensional coordinate system based on the original coordinates of the target PCB image, the image resolution, and the width and height of the imaging field of view in the XY plane.

[0057] For the horizontal and vertical coordinates in the original coordinates, the integrated device converts them into actual coordinates on the XY plane through the image resolution and the width and height of the imaging field of view; the vertical coordinate is determined according to the imaging distance of the acquisition device, that is, the vertical distance from the object to the optical axis of the lens, so as to obtain the mapped coordinates of each pixel point in the three-dimensional coordinate system.

[0058] Continuing with the above example, the integrated device obtains an image of the target PCB board at a 45° acquisition angle on the left side. This image has a resolution of 1920*1080, an imaging field of view with a width of 200mm and a height of 100mm in the XY plane, and an acquisition device imaging distance of 300mm. For pixel A with coordinates (10, 10) in the upper left corner of the image, the device first calculates its actual horizontal coordinate in the XY plane: 10*(200 / 1920)≈1.04mm, and its actual vertical coordinate: 10*(100 / 1080)≈0.93mm. The vertical coordinate is determined to be 300mm based on the imaging distance, resulting in the mapped coordinates of pixel A in the three-dimensional coordinate system: (1.04, 0.93, 300). The integrated device performs the same calculation for all pixels in the target PCB board image to complete the coordinate mapping.

[0059] Step 1022, calculate the distance from each pixel to the optical axis based on the mapped coordinates of each pixel, and take the optical axis as the reference, calculate the angle between the projection of the vector formed by each pixel and the optical axis in the XY plane and the positive direction of the X axis based on the mapped coordinates of each pixel, and obtain the polar angle at the distance.

[0060] Furthermore, the integrated device calculates the distance from each pixel point to the optical axis (Z axis) based on the mapped coordinates of the pixel point, wherein the distance is calculated using the distance formula between two points in three-dimensional space, and the distance between the projection point of the pixel point on the XY plane and the corresponding point on the optical axis (Z axis) is calculated.

[0061] Furthermore, the integrated device projects the vector formed by each pixel point and the optical axis onto the XY plane with the optical axis as a reference, and calculates the angle between the projection vector and the positive direction of the X axis, which is the polar angle at the distance.

[0062] Continuing with the above example, for the pixel point A with the mapped coordinates (1.04, 0.93, 300) in the target PCB board image at the left side with a 45° acquisition angle, calculate its distance to the optical axis. Next, the angle (polar angle) between the projection of the vector formed by pixel A and the optical axis on the XY plane and the positive direction of the X-axis is calculated. Using the inverse tangent function, arctan(0.93 / 1.04) ≈ 41.8°. The integrated device performs the same calculation for all pixels in the image, determining the distance and polar angle from each pixel to the optical axis.

[0063] Step 1023: Obtain a radial gradient based on the grayscale change rate of the pixel point in the radial direction, and obtain a tangential gradient based on the grayscale change rate of the pixel point in the tangential direction.

[0064] Furthermore, the integrated device analyzes the grayscale change rate of each pixel in the radial and tangential directions. The radial direction refers to the direction from the optical axis toward the pixel, and the tangential direction refers to the direction perpendicular to the radial direction and within the XY plane. The integrated device calculates the grayscale difference between adjacent pixels in the radial and tangential directions and divides it by the distance between the pixels to obtain the grayscale change rate of the pixel in the radial direction, namely the radial gradient. The tangential gradient is obtained similarly. The gradient information reflects the grayscale change characteristics of the pixel in different directions.

[0065] Continuing with the example of pixel A in the target PCB image with a 45° acquisition angle on the left, the integrated device selects adjacent pixels B and C in the radial direction of pixel A and calculates the grayscale difference between them and pixel A. For example, if the grayscale value of pixel A is 120, the grayscale value of pixel B is 115, and the grayscale value of pixel C is 125, and the radial distances between pixel A and B and C are all 0.1 mm, then the radial gradient G r =(125-115) / 0.2=50. In the tangential direction, select adjacent pixel points D and E and perform the same calculation, for example, to obtain the tangential gradient G t = 30. The integrated device performs similar calculations on all pixels in the image to obtain the radial gradient and tangential gradient of each pixel.

[0066] Step 1024 : determining a composite gradient feature based on the radial gradient and the tangential gradient, and determining the distance, polar angle, and composite gradient feature of each pixel as a pixel feature of each pixel.

[0067] Furthermore, the integrated device combines the radial gradient and the tangential gradient into a composite gradient feature. For example, vector synthesis can be used to take the radial gradient and the tangential gradient as two components of a vector, and the magnitude and direction of the synthetic vector are calculated as the composite gradient feature.

[0068] Furthermore, the integrated device determines the pixel feature of the pixel based on the distance from the pixel to the optical axis, the polar angle, and the composite gradient feature, and describes the properties of the pixel in terms of spatial position, grayscale change, etc.

[0069] Continuing with the above example, for the pixel point A in the target PCB board image with a 45° acquisition angle on the left, the distance is 1.4 mm, the polar angle is 41.8°, and the radial gradient G is r =50, tangential gradient G t = 30. Use the vector synthesis formula to calculate the size of the composite gradient feature direction Therefore, the pixel feature of pixel A is (1.4 mm, 41.8°, 58.3, 30.96°). The integrated device performs the same operation on all pixels in the image to determine the pixel feature of each pixel.

[0070] The embodiment of the present invention constructs a multidimensional feature description for each pixel point, including spatial position and grayscale variation characteristics, clarifies the spatial orientation relationship of the pixel point relative to the optical axis, and deeply explores the grayscale variation information of the pixel point in different directions. It provides rich and accurate data support for the subsequent step of performing parallax relationship mapping based on pixel point features. It can more accurately reflect the correspondence and differences of pixel points in images with different acquisition angles, avoid positioning deviations caused by viewing angle limitations, and make the fused PCB board image more clear and accurate in presenting the details and overall features of the PCB board. Therefore, it can accurately identify positioning marks, improve the positioning accuracy of the PCB board, and further improve the accuracy and reliability of PCB board testing.

[0071] In one embodiment, steps 1031 to 1033 are described as follows:

[0072] Step 1031 : Based on the angle differences between the different acquisition angles, the distance, polar angle, and composite gradient features of each pixel in the target PCB board image at different acquisition angles are proportionally mapped, offset-adjusted, and feature-fused to obtain the mapped distance, adjusted polar angle, and fused gradient features of each pixel in the target PCB board image at different acquisition angles.

[0073] Optionally, the integrated device performs targeted processing on the distance, polar angle, and composite gradient features of each pixel in the target PCB image based on the angular difference between different acquisition angles. For distance features, a proportional mapping approach is used to adjust the proportional relationship between the pixel and the optical axis based on the angular difference. For polar angle features, an offset adjustment is performed, where the polar angle is offset accordingly based on the angular difference. For composite gradient features, a feature fusion method is used to integrate the composite gradient features of the same pixel at different acquisition angles to obtain the mapped distance, adjusted polar angle, and fused gradient features for each pixel in the target PCB image at different acquisition angles.

[0074] Continuing with the above embodiment, there are top view, left 45° view and right 45° view. Figure 3 For example, the top view and the left 45° view have an angle difference of 45°. For a pixel P in the top view, the distance is 2 mm, the polar angle is 30°, and the composite gradient signature is (40, 20°). For the corresponding pixel Q in the left 45° view, the distance is 1.8 mm, the polar angle is 25°, and the composite gradient signature is (35, 22°). The integrated device performs proportional mapping on the distance features. Based on the 45° angle difference, the scaling factor k is calculated as 1.8 / 2 = 0.9, resulting in a mapped distance of 2 * 0.9 = 1.8 mm for pixel P. The polar angle feature is offset by 30° - 25° = 5°, resulting in an adjusted polar angle of 30° - 5° = 25° for pixel P. Composite gradient features are fused using a weighted average. For example, the top view has a weight of 0.6, while the left 45° view has a weight of 0.4. The resulting fused gradient feature has a magnitude of 0.6 * 40 + 0.4 * 35 = 38, and a direction of 0.6 * 20° + 0.4 * 22° = 20.8°. The integrated device performs the same processing on all pixels in both views, obtaining the mapped distance, adjusted polar angle, and fused gradient features.

[0075] Step 1032 : Determine the correspondence between each pixel in the target PCB board image at different acquisition angles based on the mapped distance, adjusted polar angle, and fused gradient features of each pixel in the target PCB board image at different acquisition angles.

[0076] Furthermore, the integrated device determines the correspondence between each pixel in the target PCB board image at different acquisition angles based on the mapped distance, adjusted polar angle, and fused gradient features of each pixel in the target PCB board image at different acquisition angles by comparing and matching these features. For example, when the difference in the mapped distance between two pixels is within a certain range, the difference in the adjusted polar angle is within a certain range, and the similarity of the fused gradient features reaches a certain threshold, the two pixels are considered corresponding points.

[0077] Continuing with the top view and the left 45° view as examples, the integrated device traverses the pixels in both views. For pixel P in the top view (mapped distance 1.8mm, adjusted polar angle 25°, fused gradient feature (38, 20.8°)) and pixel Q in the left 45° view (distance 1.8mm, polar angle 25°, composite gradient feature (38, 21°)), since their mapped distances are equal, the adjusted polar angle difference of 21°-20.8°=0.2° is within the set range, the fused gradient feature size difference of 38-38=0, and the direction difference of 21°-20.8°=0.2° is also within the set threshold, so pixel P and pixel Q are determined to be corresponding points.

[0078] Step 1033 , performing parallax relationship mapping based on the correspondence between each pixel in the target PCB board image at different acquisition angles, and obtaining parallax images between different acquisition angles.

[0079] Furthermore, the integrated device performs parallax mapping based on the correspondence between each pixel in the target PCB image at different acquisition angles. Specifically, for each set of corresponding pixels, the positional differences between them in the image are calculated. This positional difference information is then organized and visualized to generate parallax images for different acquisition angles. These parallax images intuitively demonstrate the positional offset of the same object (such as a component on a PCB) in images from different angles.

[0080] Continuing with the above embodiment, after determining the correspondence between all pixels in the top view and the left 45° view, the position difference is calculated for each group of corresponding pixels. For example, the horizontal coordinate difference between corresponding pixels P and Q in the image is 3 pixels, and the vertical coordinate difference is 2 pixels. The device records and processes the position difference information for all corresponding pixels. Using an image generation algorithm, this position difference information is converted into grayscale or color values to generate a disparity image between the top view and the left 45° view. Similarly, disparity images are generated between the top view and the right 45° view, and between the left 45° view and the right 45° view.

[0081] The embodiment of the present invention accurately establishes the correspondence between pixels in the target PCB board image at different acquisition angles, performs proportional mapping, offset adjustment and feature fusion on pixel point features, eliminates the problem of feature inconsistency caused by acquisition angle differences, makes the pixel point features in images at different angles comparable, performs parallax relationship mapping based on the correspondence, and generates a parallax image that clearly shows the differences between images at different angles, providing a key basis for subsequent image fusion. It enables more reasonable integration of image information at different angles during the fusion process, effectively eliminates image overlap and misalignment problems caused by viewing angle differences, and obtains a high-quality, information-complete fused PCB board image. Therefore, positioning marks can be accurately identified, the positioning accuracy of the PCB board is improved, and the accuracy and reliability of PCB board testing are further improved.

[0082] In one embodiment, steps 201 to 205 are described as follows:

[0083] Step 201 : performing region screening on pixel points in the fused PCB image based on the shape features and grayscale difference features of the positioning mark to determine candidate region sub-images of the fused PCB image.

[0084] Optionally, the integrated device pre-stores shape features (such as circle, square, cross, etc.) and grayscale difference features (grayscale contrast information with the surrounding background) of the positioning mark.

[0085] Therefore, after obtaining the fused PCB board image, the integrated device performs regional screening on all pixels in the image based on the shape characteristics and grayscale difference characteristics of the positioning mark. By traversing the pixels in the image, it determines whether the shape and grayscale of each pixel and its neighborhood meet the characteristic requirements of the positioning mark.

[0086] If the pixel point set in a certain area has a high degree of matching with the positioning mark feature in shape and grayscale, the area is determined as a candidate area, and the sub-image corresponding to the area is extracted, namely the candidate area sub-image, as described in steps 2011 to 2014.

[0087] In step 202 , for each pixel in the candidate region sub-image, pixel points are screened based on feature consistency between the pixel points and neighboring pixels to obtain candidate pixels.

[0088] Furthermore, for each candidate region sub-image, the integrated device analyzes each pixel and calculates the feature consistency between each pixel and its neighboring pixels. Features here include grayscale, color, texture, and so on. If the difference between a pixel and its neighboring pixels in these features is small, indicating a high degree of feature consistency, the pixel is retained; conversely, if the difference is large, the pixel is eliminated. In this way, pixels with high feature consistency are screened out from the candidate region sub-image, namely candidate pixels.

[0089] In one embodiment, taking one of the candidate region sub-images as an example, the integrated device processes the pixels in the sub-image. For pixel A, its grayscale value is 205, and the grayscale values of the neighboring pixels are mostly between 200-210, and the color and texture are similar, indicating that the feature consistency between pixel A and the neighboring pixels is high, and it is determined as a candidate pixel; while for pixel B, its grayscale value is 150, which is quite different from the grayscale of the surrounding neighboring pixels, and it is eliminated. The integrated device performs the same judgment on all pixels in the candidate region sub-image to obtain a set of candidate pixels for the sub-image. This operation is performed on all candidate region sub-images to obtain all candidate pixels.

[0090] Step 203 : Based on any pixel point among the candidate pixels as a starting point, the pixels having a connection relationship are connected according to an eight-neighborhood search to obtain a candidate shape outline.

[0091] Furthermore, the integrated device selects any pixel from the candidate pixels as the starting point and uses an eight-neighborhood search algorithm, which searches the eight adjacent pixels of the starting point (upper, lower, left, right, upper left, upper right, lower left, and lower right) to determine whether the adjacent pixel is a candidate pixel. If it is a candidate pixel, it is connected to the starting point, and the adjacent pixel is used as the new starting point to continue searching for candidate pixels within its eight neighborhoods. This process is repeated until there are no candidate pixels that can be connected. Through the above process, the pixels with a connection relationship are connected to form a closed contour, that is, the candidate shape contour. This operation is performed on all candidate pixels to obtain multiple candidate shape contours.

[0092] In one embodiment, pixel C is selected from the candidate pixel set as the starting point. Its eight-neighborhood search reveals that pixels D and E are candidate pixels, and pixel C is connected to D and E. Then, using pixel D as the new starting point, the search continues within its eight-neighborhood search until candidate pixel F is found and connected. This process continues until a closed contour is formed, which is a candidate shape contour. This process is repeated for other candidate pixels to obtain multiple candidate shape contours, such as circular contours and irregular shape contours.

[0093] Step 204 : performing contour screening based on the similarity between the standard shape contour of the positioning marker at different scales and each candidate shape contour to obtain the target shape contour.

[0094] Furthermore, the integrated device pre-stores standard shape contours for positioning markers at different scales. Therefore, for each candidate shape contour, the similarity between it and the standard shape contours at different scales is calculated. This similarity can be calculated using a contour matching algorithm, such as matching based on contour point distances or matching based on Fourier descriptors. By comparing the similarity between the candidate shape contours and the standard shape contours, the candidate shape contour with the highest similarity is selected as the target shape contour.

[0095] In one embodiment, the integrated device stores standard shape contours of circular positioning markers at three scales: small, medium, and large. For a candidate shape contour G, its similarity with the three scale standard circular contours is calculated. Using a matching algorithm based on contour point distance, the similarity between the candidate shape contour G and the small-scale standard circular contour is calculated to be 0.6, the similarity with the medium-scale standard circular contour is 0.85, and the similarity with the large-scale standard circular contour is 0.7. Since the similarity with the medium-scale standard circular contour is the highest, the candidate shape contour G is determined as the target shape contour. The same calculation and screening are performed on all candidate shape contours to obtain the final target shape contour.

[0096] Step 205: Determine the center of gravity position in the target shape outline as the mark position of the positioning mark in the fused PCB board image.

[0097] Furthermore, the integrated device analyzes the target shape's outline and calculates its center of gravity. For a two-dimensional shape outline, this can be calculated using an integral formula or a discretization method (such as weighted averaging the pixel coordinates on the outline). The calculated center of gravity is then used as the location of the positioning marker in the fused PCB image.

[0098] The embodiment of the present invention can accurately locate the position of the marker in the fused PCB board image, thereby accurately identifying the positioning marker and improving the positioning accuracy of the PCB board.

[0099] In one embodiment, steps 2011 to 2014 are described as follows:

[0100] Step 2011: Calculate the shape entropy of each pixel in the fused PCB image within a neighborhood of a first preset size based on the shape feature.

[0101] Optionally, for each pixel in the fused PCB image, the integrated device calculates shape entropy by analyzing the distribution of pixels within a neighborhood of a first preset size, centered on that pixel. Shape entropy measures the degree of similarity between the shape formed by the pixels in the neighborhood and the standard shape of the positioning mark. A lower shape entropy indicates a closer approximation to the standard shape of the positioning mark. The shape entropy of each pixel within its neighborhood is calculated by statistically analyzing the distribution of pixels within the neighborhood and combining it with a method for calculating information entropy.

[0102] In one embodiment, the positioning mark on the PCB board is a circle, and the integrated device sets the first preset size to a 5*5 neighborhood centered on the pixel point. For the pixel point A with coordinates (10, 10) in the fused PCB board image, the distribution of the pixels in its 5*5 neighborhood is analyzed. If the pixels in the neighborhood are mostly concentrated in a circular area with pixel point A as the center, then the shape entropy of pixel point A is calculated to be 0.2 using a specific shape entropy calculation formula (for example, calculating the information entropy based on the distance distribution from the pixel point to the center of the circle); if the pixel points in the neighborhood are distributed in a disorderly manner, the calculated shape entropy may be higher, such as 0.8. This calculation is performed on all pixels in the fused PCB board image to obtain the shape entropy of each pixel point.

[0103] Step 2012: Calculate the grayscale gradient contrast between each pixel in the fused PCB image and the pixels within its neighborhood of a second preset size based on the grayscale difference feature.

[0104] Furthermore, for each pixel in the fused PCB image, the integrated device calculates the grayscale gradient contrast between that pixel and other pixels in its neighborhood of a second preset size. The grayscale gradient contrast reflects the degree of difference in grayscale variation between a pixel and its neighboring pixels. The higher the contrast, the more significant the grayscale difference between the pixel and its surrounding pixels, and the more likely it is to belong to the positioning mark area. The integrated device calculates the grayscale gradient contrast by calculating the difference between the grayscale values of the pixel and its neighboring pixels and performing normalization processing.

[0105] Continuing with the example of pixel A with coordinates (10, 10) in the fused PCB image, the second preset size is set to a 3*3 neighborhood centered on pixel A. For example, the grayscale value of pixel A is 200, and the grayscale values of other pixels in its neighborhood are mostly around 100. Calculate the maximum difference between the grayscale values of pixel A and the pixels in the neighborhood and perform normalization. If the maximum difference is 100, the grayscale gradient contrast obtained after normalization is 0.8. If the grayscale value of the pixels in the neighborhood is slightly different from that of pixel A, the calculated grayscale gradient contrast may be lower, such as 0.2. This calculation is performed for all pixels in the fused PCB image to obtain the grayscale gradient contrast of each pixel.

[0106] Step 2013: Determine the coupling value of each pixel in the fused PCB image based on the shape entropy and grayscale gradient contrast of each pixel.

[0107] Furthermore, the integrated device combines the shape entropy and grayscale gradient contrast of each pixel and determines a coupling value for each pixel using a preset calculation method. The coupling value comprehensively measures the degree to which the pixel conforms to the positioning mark characteristics, reflecting the matching of the pixel with the positioning mark characteristics in terms of both shape and grayscale. Embodiments of the present invention can use methods such as weighted summation to assign different weights to shape entropy and grayscale gradient contrast to calculate the coupling value. For example, if shape is considered more important, shape entropy can be given a higher weight, while grayscale gradient contrast can be given a higher weight.

[0108] In one embodiment, the integrated device assigns a weight of 0.6 to shape entropy and a weight of 0.4 to grayscale gradient contrast. For pixel A, whose shape entropy is 0.2 and grayscale gradient contrast is 0.8, the coupling value of pixel A is calculated as 0.2 * 0.6 + 0.8 * 0.4 = 0.44 using the formula "coupling value = shape entropy * 0.6 + grayscale gradient contrast * 0.4." The same calculation is performed for all pixels in the fused PCB image to obtain the coupling value for each pixel.

[0109] Step 2014: performing region screening based on the coupling value of each pixel in the fused PCB image to determine candidate region sub-images of the fused PCB image.

[0110] Furthermore, the integrated device performs region screening based on the coupling value of each pixel in the fused PCB board image to determine the candidate region sub-image of the fused PCB board image, as specifically described in steps 20141 to 20144.

[0111] The embodiment of the present invention can efficiently and accurately screen out candidate area sub-images in the fused PCB board image based on the shape and grayscale difference characteristics of the positioning mark, effectively eliminating a large amount of irrelevant areas and interference information in the image, greatly narrowing the search range for subsequent positioning mark identification, and laying a solid foundation for more accurate determination of the positioning mark position. Therefore, the positioning mark can be accurately identified, thereby improving the positioning accuracy of the PCB board.

[0112] In one embodiment, steps 20141 to 20144 are described as follows:

[0113] Step 20141, traverse the coupling value of each pixel in the fused PCB board image, and determine whether the coupling value is greater than the coupling values of all pixels in its neighborhood. If so, mark it as a local extreme point.

[0114] Optionally, the integrated device traverses the coupling value of each pixel in the fused PCB image, and examines the coupling values of all pixels in its neighborhood, centered around each pixel. The neighborhood size can be pre-set based on actual needs, such as 3*3, 5*5, etc. If the coupling value of a pixel is greater than the coupling values of all pixels in its neighborhood, it indicates that the pixel has a high coupling advantage in its neighborhood, and the pixel is marked as a local extreme point. The local extreme point represents a relatively prominent position in the image in terms of comprehensive shape and grayscale features, and may be associated with a positioning mark.

[0115] In one embodiment, a 3*3 neighborhood is used as the center of each pixel point for judgment. In the fused PCB board image, for the pixel point P with coordinates (20, 20), its coupling value is 0.6. In its 3*3 neighborhood, the coupling values of the other 8 pixels are 0.3, 0.4, 0.5, 0.45, 0.52, 0.38, 0.42, and 0.55, respectively, all of which are smaller than the coupling value of pixel point P. Therefore, pixel point P is marked as a local extreme point. This traversal and judgment is performed on all pixels in the image to mark all local extreme points.

[0116] In step 20142, local extreme points are clustered according to the shape features of the pixels, so as to classify pixels with similar shapes into one category and obtain candidate region categories.

[0117] Furthermore, possible shape templates or feature descriptions of the positioning marks are pre-stored, and the integrated device performs clustering operations on the marked local extreme points according to the shape features of the pixel points. By calculating the similarity between the local extreme points and these shape templates, the local extreme points with similar shapes are classified into one category to obtain different candidate area categories. Among them, the clustering algorithm can adopt common K-Means clustering, DBSCAN clustering, etc., and the appropriate algorithm is selected according to the number and distribution characteristics of the local extreme points.

[0118] In one embodiment, the positioning mark on the PCB board is circular, and the K-Means clustering algorithm is used, with the number of clusters K set to 3. The degree of similarity between the shape formed by each local extreme point and its neighboring pixels and the circular template is calculated, for example, by calculating the circularity index of the pixel distribution. Local extreme points with similar circularity indexes are grouped together. After clustering, three candidate region categories are obtained, of which category 1 contains local extreme points with higher circularity, and categories 2 and 3 contain local extreme points with larger differences in circularity. In actual applications, category 1 is more likely to correspond to the area where the positioning mark is located.

[0119] Step 20143: For each pixel in the candidate region category, calculate its grayscale mean and variance, and merge the pixels whose grayscale values are within the preset variance range to obtain the candidate region pixels.

[0120] Furthermore, for the pixels in each candidate area category, the integrated device calculates the grayscale mean and variance of all pixels in the category, where the grayscale mean reflects the average grayscale level of the pixels in the category, and the variance reflects the degree of discreteness of the pixel grayscale values.

[0121] Furthermore, a preset variance range is set, and the integrated device merges the pixels whose grayscale values are within the preset variance range, believing that these pixels have high consistency in grayscale features and are more likely to belong to the same target area. The merged pixel set is the candidate area pixel point.

[0122] Continuing with the above example, with a preset variance range of 5-15, take candidate region category 1 as an example. This category contains 100 pixels, and the integrated device calculates the grayscale mean of these pixels to be 180 and the variance to be 10. Pixels with grayscale values between 170-190 (calculated based on the mean and variance ranges) are selected. For example, if 80 pixels meet the criteria, these 80 pixels are merged to obtain a set of candidate region pixels. The same operation is performed for the other candidate region categories to obtain their respective candidate region pixels.

[0123] Step 20144: extract the corresponding sub-image from the fused PCB board image according to the pixel points of the candidate area to obtain the candidate area sub-image.

[0124] Furthermore, the integrated device finds the positions corresponding to the pixel points in the candidate area in the fused PCB board image, extracts the minimum rectangular area containing these pixel points or the area fitted according to the actual shape, and obtains a sub-image of the candidate area.

[0125] In one embodiment, the candidate region pixels are distributed within the rectangular region between coordinates (30, 30) and (80, 80) in the fused PCB image. The integrated device extracts this rectangular region from the fused PCB image to obtain a candidate region sub-image. The same method is used to extract corresponding sub-images for other candidate region pixel sets, ultimately resulting in multiple candidate region sub-images.

[0126] The embodiment of the present invention highlights the locations in the image that have outstanding shape and grayscale comprehensive features by marking local extreme points, eliminates interference areas with mismatched shapes based on shape feature clustering, and calculates the grayscale mean and variance and merges qualified pixel points so that the pixel points in the candidate area have high consistency in grayscale, which is closer to the true features of the positioning mark, providing highly targeted data for the subsequent accurate identification of the positioning mark position. Therefore, the positioning mark can be accurately identified, thereby improving the positioning accuracy of the PCB board.

[0127] Optional, see Figure 2 , Figure 2 The figure is a flow chart of the integrated method for high-precision PCB board positioning and multi-function testing based on visual recognition provided by the present invention. In the embodiment of the present invention, the integrated method for high-precision PCB board positioning and multi-function testing based on visual recognition is executed by an integrated device. Therefore, the integrated method for high-precision PCB board positioning and multi-function testing based on visual recognition includes:

[0128] Step 10: Use an industrial camera to perform multi-angle image acquisition on the PCB board to be tested on the conveyor belt to obtain at least three initial PCB board images at different acquisition angles, and fuse the initial PCB board images at different acquisition angles to obtain a fused PCB board image.

[0129] Optionally, an industrial camera within the integrated device is positioned above or to the side of a conveyor belt. When the PCB to be tested moves along the conveyor belt to a designated location, the industrial camera initiates a multi-angle image acquisition process. Using preset angle parameters, the industrial camera sequentially captures images of the PCB from at least three different angles, capturing an initial PCB image at each angle. After acquisition, the integrated device uses an image fusion algorithm to fuse the initial PCB images from different acquisition angles, eliminating image information lost due to differences in viewing angles. This results in a fused image containing complete PCB information.

[0130] Step 20: Based on the positioning mark on the PCB board identified in the fused PCB board image, determine the marking position of the positioning mark in the fused PCB board image.

[0131] Furthermore, the integrated device uses computer vision technology to identify positioning marks on the PCB board in the fused PCB board image. Positioning marks are typically identifiers with specific shapes, colors, or patterns, such as circular marks or cross marks. Therefore, the integrated device uses image recognition algorithms, such as template matching and feature point detection, to search for the positioning mark in the image and determine its precise coordinate position in the fused PCB board image, namely the mark position.

[0132] Step 30: Determine the initial posture information and initial position information of the PCB board in the image coordinate system based on the marker position and the geometric relationship, and convert the initial posture information and initial position information to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system to obtain the target posture information and target position information of the PCB board in the world coordinate system.

[0133] Furthermore, the integrated device uses geometric relationships to calculate the PCB board's initial posture information (e.g., rotation angle) and initial position information (e.g., center point coordinates) in the image coordinate system based on the positions of the positioning marks in the fused PCB image. For example, by calculating the distance and angular relationship between the positioning marks, the PCB board's rotation angle can be determined; by calculating the center coordinates of multiple positioning marks, the PCB board's center point position in the image coordinate system can be determined.

[0134] Furthermore, after obtaining the initial posture information and initial position information, the integrated device converts the initial posture information and initial position information to the world coordinate system according to the conversion relationship between the pre-calibrated image coordinate system and the world coordinate system (such as calibrating the coordinate system through a calibration plate to obtain the rotation matrix and translation vector), and obtains the target posture information and target position information of the PCB board in the world coordinate system.

[0135] Step 40 : Controlling the test probe carried by the robotic arm to perform a multi-function test on the PCB board according to the target posture information and the target position information.

[0136] Furthermore, the integrated device controls the movement of a robotic arm based on the PCB's target posture and position information in the world coordinate system. The robotic arm, equipped with a test probe, performs multifunctional tests on various test points on the PCB, such as electrical performance and signal integrity, according to pre-set test procedures. During the test process, test data is collected in real time, and the test results are judged as qualified according to test standards.

[0137] Step 50: If the PCB board passes the test, the robot arm is controlled to remove the test probe and the PCB board is transported to the next process via a conveyor belt.

[0138] Furthermore, if all tests on the PCB board pass, the integrated device controls the robotic arm to move the test probe away to avoid interference with subsequent processes.

[0139] Furthermore, the integrated device controls the conveyor belt to start and transfer the PCB boards that pass the test to the next process. If the PCB board fails the test, the integrated device controls the robotic arm to remove the PCB board from the conveyor belt and place it in the defective product area for subsequent repair or scrapping.

[0140] The embodiment of the present invention obtains PCB board images from at least three different acquisition angles, captures the detailed features of the PCB board from multiple perspectives, avoids positioning deviations caused by limited viewing angles, and then determines the mark position by identifying the positioning mark in the image, and calculates the posture information and position information in combination with the geometric relationship. Therefore, the positioning mark can be accurately identified, and it can adapt to the subtle differences between PCB boards of different batches and different specifications, thereby improving the positioning accuracy of the PCB board. On the other hand, non-contact visual recognition technology is used for positioning. After positioning is completed, the robot arm equipped with a test probe is controlled to automatically perform multi-function testing on the PCB board. The entire process does not require human intervention. The robot arm performs the test task quickly and orderly according to the program settings, avoiding the tedious operations and time loss in the contact sensor process, so that the positioning and testing links are closely connected, reducing waiting time and improving the testing efficiency of the PCB board.

[0141] See also Figure 3 , Figure 3 This is a diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, the following steps are implemented:

[0142] An industrial camera is used to capture images of the PCB to be tested on the conveyor belt from multiple angles, obtaining at least three initial PCB images at different acquisition angles. The initial PCB images at different acquisition angles are then fused to obtain a fused PCB image.

[0143] Determine a marking position of the positioning mark in the fused PCB board image based on the positioning mark on the PCB board identified in the fused PCB board image;

[0144] Determine the initial posture information and initial position information of the PCB board in the image coordinate system based on the marker position and geometric relationship, and convert the initial posture information and initial position information to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system to obtain the target posture information and target position information of the PCB board in the world coordinate system;

[0145] Control the test probe carried by the robotic arm to perform multifunctional testing on the PCB board based on the target posture information and target position information;

[0146] If the PCB board passes the test, the robot arm is controlled to remove the test probe and transfer the PCB board to the next process via the conveyor belt.

[0147] See also Figure 4 , Figure 4 Detailed description of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. Figure 4 As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 311 is stored. When the computer program 311 is executed by a processor, the following steps are implemented:

[0148] An industrial camera is used to capture images of the PCB to be tested on the conveyor belt from multiple angles, obtaining at least three initial PCB images at different acquisition angles. The initial PCB images at different acquisition angles are then fused to obtain a fused PCB image.

[0149] Determine a marking position of the positioning mark in the fused PCB board image based on the positioning mark on the PCB board identified in the fused PCB board image;

[0150] Determine the initial posture information and initial position information of the PCB board in the image coordinate system based on the marker position and geometric relationship, and convert the initial posture information and initial position information to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system to obtain the target posture information and target position information of the PCB board in the world coordinate system;

[0151] Control the test probe carried by the robotic arm to perform multifunctional testing on the PCB board based on the target posture information and target position information;

[0152] If the PCB board passes the test, the robot arm is controlled to remove the test probe and transfer the PCB board to the next process via the conveyor belt.

[0153] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the high-precision PCB board positioning and multi-functional testing integrated method based on visual recognition provided by the above methods. The high-precision PCB board positioning and multi-functional testing integrated method based on visual recognition includes:

[0154] An industrial camera is used to capture images of the PCB to be tested on the conveyor belt from multiple angles, obtaining at least three initial PCB images at different acquisition angles. The initial PCB images at different acquisition angles are then fused to obtain a fused PCB image.

[0155] Determine a marking position of the positioning mark in the fused PCB board image based on the positioning mark on the PCB board identified in the fused PCB board image;

[0156] Determine the initial posture information and initial position information of the PCB board in the image coordinate system based on the marker position and geometric relationship, and convert the initial posture information and initial position information to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system to obtain the target posture information and target position information of the PCB board in the world coordinate system;

[0157] Control the test probe carried by the robotic arm to perform multifunctional testing on the PCB board based on the target posture information and target position information;

[0158] If the PCB board passes the test, the robot arm is controlled to remove the test probe and transfer the PCB board to the next process via the conveyor belt.

[0159] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0160] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A PCB board positioning and multifunctional testing integrated device based on visual recognition, characterized in that: The device includes an integrated controller, an image processing unit, a visual recognition unit, a PCB board positioning unit, a PCB board testing unit, and a robotic arm control unit; the integrated controller is connected to the image processing unit, the visual recognition unit, the PCB board positioning unit, the PCB board testing unit, and the robotic arm control unit, respectively, to manage each unit; An image processing unit is configured to capture multi-angle images of the PCB board to be tested on the conveyor belt using an industrial camera, obtain at least three initial PCB board images at different acquisition angles, and fuse the initial PCB board images at different acquisition angles to obtain a fused PCB board image; a visual recognition unit, configured to determine a marking position of the positioning mark on the PCB board identified in the fused PCB board image; a PCB board positioning unit, configured to determine initial posture information and initial position information of the PCB board in the image coordinate system based on the mark positions and the geometric relationship, and convert the initial posture information and initial position information into the world coordinate system according to a conversion relationship between the image coordinate system and the world coordinate system, thereby obtaining target posture information and target position information of the PCB board in the world coordinate system; A PCB board testing unit, configured to control a test probe carried by a robotic arm to perform a multifunctional test on the PCB board according to the target posture information and the target position information; The robot control unit is used to control the robot to remove the test probe if the PCB board passes the test, and to convey the PCB board to the next process through the conveyor belt.

2. The PCB positioning and multifunctional testing integrated device based on visual recognition according to claim 1, characterized in that: Determining a marking position of the positioning mark in the fused PCB image based on a positioning mark on the PCB board identified in the fused PCB board image includes: Performing regional screening on pixel points in the fused PCB image based on the shape characteristics and grayscale difference characteristics of the positioning mark to determine candidate regional sub-images of the fused PCB image; For each pixel in the candidate region sub-image, the pixel is screened based on the feature consistency between the pixel and the neighboring pixel to obtain the candidate pixel. Based on any pixel point among the candidate pixels as the starting point, the pixels with connection relationship are connected according to the eight-neighborhood search to obtain the candidate shape outline; Perform contour screening based on the similarity between the standard shape contour of the positioning mark at different scales and each candidate shape contour to obtain the target shape contour; The center of gravity position in the target shape contour is determined as the mark position of the positioning mark in the fused PCB board image.

3. The PCB positioning and multifunctional testing integrated device based on visual recognition according to claim 2, characterized in that: The performing regional screening on the pixel points in the fused PCB board image based on the shape feature and the grayscale difference feature of the positioning mark to determine the candidate regional sub-image of the fused PCB board image includes: Calculating the shape entropy of each pixel in the fused PCB image within a first preset size neighborhood based on the shape feature; Calculating the grayscale gradient contrast between each pixel in the fused PCB image and the pixels in its neighborhood of a second preset size based on the grayscale difference feature; Determine the coupling value of each pixel point based on the shape entropy and grayscale gradient contrast of each pixel point in the fused PCB board image; Region screening is performed based on the coupling degree value of each pixel point in the fused PCB board image to determine a candidate region sub-image of the fused PCB board image.

4. The PCB positioning and multifunctional testing integrated device based on visual recognition according to claim 3 is characterized in that: The performing region screening based on the coupling value of each pixel in the fused PCB board image to determine a candidate region sub-image of the fused PCB board image includes: Traversing the coupling value of each pixel in the fused PCB board image, determining whether its coupling value is greater than the coupling value of all pixels in its neighborhood, and if so, marking it as a local extreme point; Cluster the local extreme points according to the shape characteristics of the pixels to classify the pixels with similar shapes into one category and obtain the candidate region category; For each pixel in the candidate region category, calculate its grayscale mean and variance, and merge the pixels whose grayscale values are within the preset variance range to obtain the candidate region pixels; A corresponding sub-image is extracted from the fused PCB board image according to the pixel points of the candidate area to obtain the candidate area sub-image.

5. The PCB positioning and multifunctional testing integrated device based on visual recognition according to any one of claims 1 to 4, characterized in that: The fusing of the initial PCB images acquired at different acquisition angles to obtain a fused PCB image includes: Image segmentation is performed based on the edge transparency channel critical value of each pixel in the initial PCB board image at each acquisition angle to obtain the target PCB board image at each acquisition angle; Determine the pixel feature of each pixel in the target PCB board image at each acquisition angle based on the lens optical axis at each acquisition angle and each pixel in the target PCB board image; Based on the camera parameters at different acquisition angles and the pixel features of each pixel in the target PCB board image, parallax relationship mapping is performed to obtain the parallax images between different acquisition angles; An image fusion level is determined based on the angle difference between different acquisition angles, and the parallax images between different acquisition angles are fused according to the image fusion level between different acquisition angles to obtain the fused PCB board image.

6. The PCB positioning and multifunctional testing integrated device based on visual recognition according to claim 5, characterized in that: The step of determining a pixel feature of each pixel in the target PCB board image at each acquisition angle based on the lens optical axis at each acquisition angle and each pixel in the target PCB board image includes: For each acquisition angle of the target PCB image, a three-dimensional coordinate system is established, with the lens optical axis as the Z axis and the plane perpendicular to the optical axis and passing through the center of the image as the XY plane. The original coordinates of each pixel in the target PCB image are mapped to the three-dimensional coordinate system to obtain the mapped coordinates of each pixel. The horizontal and vertical coordinates of the mapped coordinates are determined based on the original coordinates combined with the image resolution and the width and height of the imaging field of view on the XY plane. The vertical coordinate of the mapped coordinates is determined based on the imaging distance of the acquisition device. Calculate the distance from each pixel to the optical axis based on the mapped coordinates of each pixel, and with the optical axis as a reference, calculate the angle between the projection of the vector formed by each pixel and the optical axis in the XY plane and the positive direction of the X axis based on the mapped coordinates of each pixel to obtain the polar angle at the distance; Based on the grayscale change rate of the pixel point in the radial direction, the radial gradient is obtained, and based on the grayscale change rate of the pixel point in the tangential direction, the tangential gradient is obtained; A composite gradient feature is determined based on the radial gradient and the tangential gradient, and the distance, polar angle and composite gradient feature of each pixel are determined as a pixel feature of each pixel.

7. The integrated device for PCB board positioning and multifunctional testing based on visual recognition according to claim 6, characterized in that: The method of performing parallax relationship mapping based on camera parameters at different acquisition angles and pixel features of each pixel in the target PCB board image to obtain parallax images at different acquisition angles includes: Based on the angle differences between different acquisition angles, the distance, polar angle, and composite gradient features of each pixel in the target PCB board image at different acquisition angles are proportionally mapped, offset-adjusted, and feature-fused to obtain the mapped distance, adjusted polar angle, and fused gradient features of each pixel in the target PCB board image at different acquisition angles. Determine the correspondence between each pixel in the target PCB board image at different acquisition angles based on the mapped distance, adjusted polar angle, and fused gradient features of each pixel in the target PCB board image at different acquisition angles; Based on the correspondence between each pixel in the target PCB board image at different acquisition angles, parallax relationship mapping is performed to obtain the parallax images between different acquisition angles.

8. A method for integrating high-precision PCB positioning and multifunctional testing based on visual recognition, implemented based on the device for integrating PCB positioning and multifunctional testing based on visual recognition according to any one of claims 1 to 7, characterized in that: The high-precision PCB board positioning and multi-functional testing integrated method based on visual recognition includes: An industrial camera is used to capture images of the PCB to be tested on the conveyor belt from multiple angles, obtaining at least three initial PCB images at different acquisition angles. The initial PCB images at different acquisition angles are then fused to obtain a fused PCB image. Based on the positioning mark on the PCB board identified in the fused PCB board image, determining a marking position of the positioning mark in the fused PCB board image; Determining initial posture information and initial position information of the PCB board in the image coordinate system based on the mark position and the geometric relationship, and converting the initial posture information and initial position information to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system, to obtain target posture information and target position information of the PCB board in the world coordinate system; Controlling a test probe carried by a robotic arm to perform a multifunctional test on a PCB board according to the target posture information and the target position information; If the PCB board passes the test, the robot arm is controlled to move the test probe away and the PCB board is conveyed to the next process via the conveyor belt.

9. An electronic device comprising: The memory and processor are characterized in that a computer software program is stored on the memory, and when the processor reads and executes the computer software program, the high-precision PCB board positioning and multi-functional testing integrated method based on visual recognition as described in claim 8 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that The storage medium stores a computer software program, which, when executed by the processor, implements the high-precision PCB board positioning and multi-functional testing integrated method based on visual recognition as described in claim 8.

Images