PCB board positioning and multifunctional testing integrated device based on visual recognition

Abstract

The application provides a PCB positioning and multifunctional testing integrated device based on visual recognition, which comprises an integrated controller, an image processing unit, a visual recognition unit, a PCB positioning unit, a PCB testing unit and a mechanical arm control unit; the image processing unit fuses initial PCB images of different collection angles to obtain a fused PCB image; the visual recognition unit determines the mark position of the positioning mark in the fused PCB image; the PCB positioning unit converts initial attitude information and initial position information to a world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system to obtain target attitude information and target position information of the PCB in the world coordinate system; the PCB testing unit performs multifunctional testing on the PCB; and the mechanical arm control unit conveys the PCB to the next process through a conveying belt. The application improves the positioning accuracy of the PCB and the testing efficiency of the PCB.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, and relates to a PCB positioning and multifunctional testing integrated device based on visual recognition. BACKGROUND

[0002] In the modern electronic manufacturing field, high-precision PCB positioning and multifunctional testing integrated methods are crucial to product quality and production efficiency. The current widely used integrated methods mostly rely on contact sensors or single mechanical positioning methods. Contact sensors need to be in direct contact with PCBs to obtain positioning information, but the contact process takes a long time, and the contact positioning information has subjective bias, thereby reducing the positioning accuracy of the PCB and the testing efficiency of the PCB. The single mechanical positioning method is limited by fixed structures and preset parameters, and it is difficult to accurately match the subtle differences of different batches and different specifications of PCBs, resulting in large positioning deviation, which reduces the positioning accuracy of the PCB and the testing efficiency of the PCB. SUMMARY

[0003] The present application provides a PCB positioning and multifunctional testing integrated device based on visual recognition, aiming to improve the positioning accuracy of the PCB and the testing efficiency of the PCB.

[0004] In a first aspect, the present application provides a PCB positioning and multifunctional testing integrated device based on visual recognition, comprising an integrated controller, an image processing unit, a visual recognition unit, a PCB positioning unit, a PCB testing unit and a mechanical arm control unit; the integrated controller is connected with the image processing unit, the visual recognition unit, the PCB positioning unit, the PCB testing unit and the mechanical arm control unit respectively, and manages each unit.

[0005] The image processing unit is configured to perform multi-angle image acquisition on the PCB to be tested on the conveying belt through an industrial camera, obtain initial PCB images at at least three different acquisition angles, and fuse the initial PCB images at different acquisition angles to obtain a fused PCB image.

[0006] The visual recognition unit is configured to determine the mark position of the positioning mark in the fused PCB image based on the positioning mark on the PCB recognized in the fused PCB image.

[0007] The PCB board positioning unit is configured to determine initial attitude information and initial position information of the PCB board in an image coordinate system based on the mark position and geometric relationship, and convert the initial attitude information and the initial position information to a world coordinate system according to a conversion relationship between the image coordinate system and the world coordinate system, to obtain target attitude information and target position information of the PCB board in the world coordinate system.

[0008] The PCB board testing unit is configured to control a test probe carried by the mechanical arm to perform multifunctional testing on the PCB board according to the target attitude information and the target position information.

[0009] The mechanical arm control unit is configured to control the mechanical arm to move the test probe away and convey the PCB board to a next process through the conveying belt if the PCB board testing is passed.

[0010] In a second aspect, the present application further provides a high-precision PCB board positioning and multifunctional testing integrated method based on visual recognition, which is implemented based on the PCB board positioning and multifunctional testing integrated device based on visual recognition in the first aspect, and the method comprises the following steps.

[0011] The industrial camera is used to perform multi-angle image acquisition on the PCB board to be tested on the conveying belt, to obtain initial PCB board images at at least three different acquisition angles, and to fuse the initial PCB board images at different acquisition angles to obtain a fused PCB board image.

[0012] The mark position of the positioning mark on the PCB board in the fused PCB board image is determined based on the positioning mark recognized in the fused PCB board image.

[0013] The initial attitude information and the initial position information of the PCB board in the image coordinate system are determined based on the mark position and the geometric relationship, and the initial attitude information and the initial position information are converted to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system, to obtain the target attitude information and the target position information of the PCB board in the world coordinate system.

[0014] The test probe carried by the mechanical arm is controlled to perform multifunctional testing on the PCB board according to the target attitude information and the target position information.

[0015] If the PCB board testing is passed, the test probe is controlled to be moved away by the mechanical arm, and the PCB board is conveyed to a next process through the conveying belt.

[0016] In a third aspect, the present application also provides an electronic device, comprising: a memory for storing a computer software program; and a processor for reading and executing the computer software program, thereby implementing the above-mentioned any one of the high-precision PCB positioning and multifunctional testing integrated method based on visual recognition.

[0017] In a fourth aspect, the present application also provides a non-transitory computer readable storage medium, wherein the storage medium stores a computer software program, and the computer software program is executed by a processor to implement the above-mentioned any one of the high-precision PCB positioning and multifunctional testing integrated method based on visual recognition.

[0018] In a fifth aspect, the present application also provides a computer program product comprising a computer program, and the computer program is executed by a processor to implement the above-mentioned any one of the high-precision PCB positioning and multifunctional testing integrated method based on visual recognition.

[0019] The high-precision PCB positioning and multifunctional testing integrated device based on visual recognition provided by the embodiments of the present application can capture the detailed features of the PCB from multiple perspectives by acquiring PCB images at at least three different collection angles, avoid positioning deviation caused by perspective limitation, determine the marker position through the positioning marker recognized in the image, and calculate the posture information and position information in combination with the geometric relationship, so that the positioning marker can be accurately recognized, the subtle differences of different batches and different specifications of PCBs can be adapted, and the positioning accuracy of the PCB is improved. On the other hand, the non-contact visual recognition technology is used for positioning, and after the positioning is completed, the mechanical arm carrying the test probe automatically performs multifunctional testing on the PCB. The whole process does not need manual intervention, the mechanical arm executes the test task quickly and orderly according to the program setting, the cumbersome operation and time loss in the process of the contact sensor are avoided, the positioning and testing links are closely connected, the waiting time is reduced, and the testing efficiency of the PCB is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a structural schematic diagram of the high-precision PCB positioning and multifunctional testing integrated device based on visual recognition provided by the present application;

[0021] Figure 2 is a flowchart of the high-precision PCB positioning and multifunctional testing integrated method based on visual recognition provided by the present application;

[0022] Figure 3 is an embodiment diagram of the electronic device provided by the embodiments of the present application;

[0023] Figure 4 is an embodiment diagram of the computer readable storage medium provided by the embodiments of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0025] In the description of the present application, the terms "first", "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly and specifically limited.

[0026] In the description of the present application, the term "for example" is used to indicate "as an example, illustration or explanation". Any embodiment described as "for example" in the present application is not necessarily interpreted as more preferred or more advantageous than other embodiments. The following description is given in order to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can realize the present application without using these specific details. In other examples, well-known structures and processes will not be described in detail in order to avoid unnecessary details making the description of the present application obscure. Therefore, the present application is not intended to be limited to the shown embodiments, but is consistent with the broadest scope in accordance with the principles and characteristics disclosed.

[0027] Optionally, referring to Figure 1 shown, Figure 1 is a structural schematic diagram of a PCB positioning and multifunctional testing integrated device based on visual recognition provided by the present application. The PCB positioning and multifunctional testing integrated device based on visual recognition comprises an integrated controller, an image processing unit, a visual recognition unit, a PCB positioning unit, a PCB testing unit and a mechanical arm control unit. Optionally, the integrated controller in the embodiment of the present application is connected with the image processing unit, the visual recognition unit, the PCB positioning unit, the PCB testing unit and the mechanical arm control unit respectively, and manages each unit.

[0028] Optionally, the industrial camera is arranged above or beside the conveying belt. When the PCB to be tested moves to the specified position with the conveying belt, the industrial camera starts a multi-angle image acquisition program. Through the pre-set angle parameters, the industrial camera sequentially acquires images of the PCB from at least three different angles to obtain an initial PCB image collected at each angle.

[0029] After the image processing unit obtains the initial PCB images of at least three different acquisition angles, the initial PCB images of different acquisition angles are fused by using an image fusion algorithm to eliminate the missing image information caused by the difference in viewing angle, and a fused PCB image containing complete PCB information is obtained, as described in steps 101 to 104. In an embodiment, three industrial cameras are provided, which are arranged above the conveying belt (0°), 45° to the left of the conveying belt, and 45° to the right of the conveying belt, respectively. When the PCB enters the image acquisition area, the industrial camera above acquires a top view of the PCB as the first initial PCB image; then, the industrial camera 45° to the left acquires an image with side information as the second initial PCB image; finally, the industrial camera 45° to the right acquires an image with different side information as the third initial PCB image.

[0030] Optionally, the visual recognition unit recognizes the positioning mark on the PCB in the fused PCB image based on computer vision technology, wherein the positioning mark is usually a mark with a specific shape, color or pattern, such as a circular mark, a cross mark, etc. Therefore, the visual recognition unit searches for the positioning mark in the image by using an image recognition algorithm, such as template matching, feature point detection, etc., and determines the accurate coordinate position of the mark in the fused PCB image, i.e., the mark position, as described in steps 201 to 205.

[0031] In an embodiment, four circular positioning marks are provided on the PCB, and the center position of each mark is used to determine the position and pose of the PCB. For example, a template image of the circular positioning mark is stored in advance by using a template matching algorithm. When the fused PCB image is processed, the template image is iteratively matched in the fused image, the similarity between the template and the image region is calculated, the region with the highest matching degree is found, and the position of the circular positioning mark is determined. When the four circular positioning marks are found, the pixel coordinates of the center of each mark in the fused PCB image are recorded, such as the coordinates of mark 1 (x1, y1), the coordinates of mark 2 (x2, y2), the coordinates of mark 3 (x3, y3), and the coordinates of mark 4 (x4, y4). These coordinates are the mark positions of the positioning marks in the fused PCB image.

[0032] Optionally, the PCB positioning unit calculates the initial pose information (e.g., rotation angle) and initial position information (e.g., center point coordinates) of the PCB in the image coordinate system according to the marker positions of the positioning markers in the fused PCB image, by using geometric relationships. For example, the rotation angle of the PCB can be determined by calculating the distance and angle relationship between the positioning markers; the center point position of the PCB in the image coordinate system can be determined by calculating the center coordinates of the multiple positioning markers.

[0033] Further, after obtaining the initial pose information and initial position information, the PCB positioning unit converts the initial pose information and initial position information to the world coordinate system according to the conversion relationship (e.g., rotation matrix and translation vector obtained by coordinate system calibration using a calibration board) between the image coordinate system and the world coordinate system, to obtain the target pose information and target position information of the PCB in the world coordinate system.

[0034] Continuing with the above embodiment, four circular positioning markers are determined. First, the distance d12 between marker 1 and marker 2, the distance d23 between marker 2 and marker 3, and the angle θ between the line connecting marker 1 and marker 2 and the horizontal axis are calculated. According to the geometric relationship, the rotation angle of the PCB in the image coordinate system is calculated, for example, the rotation angle is calculated to be α. Next, the center coordinates (x0, y0) of the four circular positioning markers are calculated as the initial position information of the PCB in the image coordinate system. Then, the conversion relationship between the image coordinate system and the world coordinate system is called, where the rotation matrix is R and the translation vector is T. By the coordinate conversion formula: P world = R * P image + T, the initial pose information α and the initial position information (x0, y0) are converted to the world coordinate system, to obtain the target pose information (e.g., rotation angle α' relative to the world coordinate system) and target position information (e.g., coordinates (x', y', z') in the world coordinate system) of the PCB in the world coordinate system.

[0035] Optionally, the PCB testing unit controls the movement of the robot arm according to the target pose information and target position information of the PCB in the world coordinate system. The robot arm carries a test probe and performs multi-functional tests (e.g., electrical performance tests, signal integrity tests, etc.) on each test point on the PCB according to a preset test program. During the test process, test data is collected in real time, and the test result is judged according to the test standard.

[0036] In an embodiment, there are 10 test points on the PCB board, which are used to test different electrical properties and signal integrity. The PCB test unit controls the robot arm to move the test probe to the first test point according to the target pose information and target position information. The robot arm is driven by a servo motor and adjusts the pose in real time during movement to ensure that the test probe is vertically aligned with the test point. When the test probe contacts the test point, the PCB test unit starts electrical property testing, such as measuring resistance, voltage, and other parameters. After testing is complete, the robot arm moves the test probe to the next test point and repeats the above testing process until all 10 test points are tested. During testing, the PCB test unit compares the collected test data with the preset test standard to determine whether the test results of each test point are qualified.

[0037] Optionally, if all tests of the PCB board pass, the robot arm control unit controls the robot arm to move the test probe away to avoid interference with subsequent processes.

[0038] Further, the robot arm control unit controls the conveyor to start and convey the PCB board that passed the test to the next process. If the PCB board test fails, the robot arm control unit controls the robot arm to remove the PCB board from the conveyor and place it in the unqualified product area for subsequent repair or scrap processing.

[0039] In an embodiment, after all tests of the PCB board are completed and it is determined that the PCB board passes the test, the robot arm control unit controls the motor of the robot arm to reverse, causing the test probe to lift up and move away. Then, the robot arm control unit sends a start signal to the drive motor of the conveyor, and the conveyor starts to run and convey the PCB board that passed the test to the next process, such as the next assembly station. If during testing, it is found that the test results of a test point do not meet the test standard, the robot arm control unit controls the robot arm to clamp the PCB board and move it to the unqualified product storage area, waiting for the staff to further process.

[0040] The embodiment of the present application can capture the detailed features of the PCB board from multiple perspectives by acquiring PCB board images at at least three different collection angles, avoid positioning deviation caused by perspective limitations, determine the marker position through the positioning marker identified in the image, and calculate the attitude information and position information in combination with the geometric relationship, so that the positioning marker can be accurately identified, the subtle differences of different batches and different specifications of PCB boards can be adapted, and the positioning accuracy of the PCB board is improved. On the other hand, the non-contact visual recognition technology is used for positioning, and after the positioning is completed, the mechanical arm carrying the test probe automatically performs multifunctional testing on the PCB board. The whole process does not need manual intervention, the mechanical arm executes the test task quickly and orderly according to the program setting, avoids the cumbersome operation and time loss in the process of the contact sensor, makes the positioning and testing links closely connected, reduces the waiting time, and improves the testing efficiency of the PCB board.

[0041] In an embodiment, steps 101 to 104 are described as follows:

[0042] Step 101, based on the edge transparent channel critical value of each pixel point in the initial PCB board image of each collection angle, image segmentation is performed to obtain the target PCB board image of each collection angle.

[0043] Optionally, the edge transparent channel is used to identify whether the pixel point is in the image edge or background area. The integrated device compares the edge transparent channel value of each pixel point with the pre-set critical value. If the edge transparent channel value of the pixel point is greater than the critical value, it is determined that the pixel point belongs to the background or edge invalid area. If the edge transparent channel value of the pixel point is less than or equal to the critical value, it is determined that it belongs to the PCB board effective area. After judging all the pixel points, the pixel points belonging to the PCB board effective area are retained, and the background and edge invalid area are removed to obtain the target PCB board image of each collection angle.

[0044] Continuing the above embodiment, taking the first top view as an example, each pixel point in the image contains edge transparent channel value information. For example, the critical value is set to 0.5. All pixel points of the image are traversed. For pixel point A, the edge transparent channel value is 0.3, which is less than the critical value, so it is retained. For pixel point B, the edge transparent channel value is 0.6, which is greater than the critical value, so it is removed. Through the processing of all pixel points of the whole image, the first target PCB board image after removing the background and edge interference is obtained. Similarly, the same operation is performed on the other two initial PCB board images to obtain the corresponding target PCB board images.

[0045] Step 102, based on the lens optical axis of each collection angle and each pixel point in the target PCB board image, the pixel point feature of each pixel point in the target PCB board image of each collection angle is determined.

[0046] Further, the integrated device analyzes the position information of each pixel point in the target PCB image according to the lens optical axis of each collection angle, wherein the lens optical axis determines the propagation direction of the light and the imaging characteristics. Therefore, the integrated device determines the pixel point characteristics of each pixel point by calculating the positional relationship of each pixel point relative to the lens optical axis, such as angle, distance, and other parameters, as described in steps 1021 to 1024, wherein the pixel point characteristics include but are not limited to the spatial position of the pixel point, the light incidence angle, the reflection characteristics, and other attributes reflecting the properties of the pixel point in the actual space.

[0047] Step 103: Based on the camera parameters of different collection angles and the pixel point characteristics of each pixel point in the target PCB image, the parallax relationship mapping is performed to obtain the parallax images between different collection angles.

[0048] Further, the integrated device uses the camera parameters (such as focal length, aperture, sensor size, etc.) of different collection angles, combined with the pixel point characteristics of each pixel point in the target PCB image, to analyze the position difference of the same object (such as elements on the PCB) in the image under different angles, calculate the parallax information between the pixel points, and perform parallax relationship mapping on the parallax information between the pixel points to generate parallax images between different collection angles, wherein the parallax images directly show the differences and corresponding relationships between different angle images, as described in steps 1031 to 1033.

[0049] Continuing the above example, the focal lengths of the three collection angle industrial cameras are f1, f2, and f3, the apertures are a1, a2, and a3, and the sensor sizes are s1, s2, and s3. Taking the top view and the left 45° view as an example, the integrated device calculates the parallax between the corresponding pixel points in the two images, such as pixel point D in the top view and pixel point E in the left 45° view, according to their pixel point characteristics and camera parameters. By calculating the parallax of all corresponding pixel points in the two images, the parallax image between the top view and the left 45° view is generated. Similarly, the parallax images between the top view and the right 45° view, and between the left 45° view and the right 45° view are generated.

[0050] Step 104: Based on the angle difference between different collection angles, the image fusion level is determined, and the parallax images between different collection angles are fused according to the image fusion level between different collection angles to obtain the fused PCB image.

[0051] Further, the integrated device determines the image fusion level according to the angle difference between different collection angles, wherein the greater the angle difference, the higher the fusion level, and the greater the processing and adjustment degree of the image in the fusion process. Further, the integrated device performs a fusion operation on the parallax images between different collection angles according to the determined image fusion level. In the fusion process, the information in different parallax images is integrated through weighted average, region-based fusion algorithm, etc., to eliminate the inconsistency of image information caused by the difference in viewing angles, and finally obtain the fused PCB board image containing complete PCB board information.

[0052] Continuing the above embodiment, among the three collection angles, the angle difference between the top view and the left 45° view is 45°, the angle difference between the top view and the right 45° view is also 45°, and the angle difference between the left 45° view and the right 45° view is 90°. According to the preset rule, it is determined that the fusion level of the top view and the left and right side views is medium, and the fusion level between the left and right side views is high. In the fusion process, for the parallax images of the top view and the left 45° view of the medium fusion level, a weighted average algorithm is used to fuse the two images by giving them appropriate weights according to the image quality and information richness; for the parallax images of the left 45° view and the right 45° view of the high fusion level, a region-based fusion algorithm is used to fuse the images after targeted processing of different regions in the images. After the fusion operation of the three groups of parallax images, a complete fused PCB board image is finally obtained.

[0053] The embodiment of the present application fully integrates the information of images of different angles, eliminates the inconsistency and loss of image information caused by the difference in viewing angles, avoids the positioning deviation caused by the limitation of viewing angle, makes the fused PCB board image present the information of each angle of the PCB board completely and accurately, provides high-quality image data basis for subsequent positioning mark recognition, posture and position determination, and therefore can accurately recognize the positioning mark, can adapt to the subtle differences of different batches and different specifications of PCB boards, improves the positioning accuracy of the PCB board, and further effectively improves the accuracy and reliability of the PCB board test.

[0054] In an embodiment, steps 1021 to 1024 are described as follows:

[0055] Step 1021, for the target PCB board image of each collection angle, taking the lens optical axis as the Z axis and the plane perpendicular to the optical axis and passing through the image center as the XY plane, a three-dimensional coordinate system is established, and the original coordinates of each pixel point in the target PCB board image are mapped to the three-dimensional coordinate system to obtain the mapped coordinates of each pixel point. The horizontal and vertical coordinates in the mapped coordinates are determined based on the original coordinates in combination with the image resolution and the width and height of the imaging field on the XY plane, and the vertical coordinate in the mapped coordinates is determined based on the imaging distance of the collection device.

[0056] Optionally, for each target PCB image of a collection angle, the integrated device establishes the main axis of the three-dimensional coordinate system with the lens optical axis of the angle as the Z axis, and at the same time, finds the center of the target PCB image, and constructs a plane perpendicular to the optical axis and passing through the image center as the XY plane. After the three-dimensional coordinate system is established, the integrated device maps the original coordinates of each pixel point in the target PCB image to the three-dimensional coordinate system according to the original coordinates of the target PCB image, the image resolution, and the width and height of the imaging field on the XY plane.

[0057] For the horizontal and vertical coordinates in the original coordinates, the integrated device converts them into the actual coordinates on the XY plane through the image resolution and the width and height of the imaging field; and the vertical coordinate is determined according to the imaging distance of the collection device, that is, the vertical distance from the object to the lens optical axis, so as to obtain the mapping coordinates of each pixel point in the three-dimensional coordinate system.

[0058] Continuing the above embodiment, the integrated device obtains the target PCB image of the left 45° collection angle, the image resolution of which is 1920*1080, the width of the imaging field on the XY plane is 200mm, the height is 100mm, and the imaging distance of the collection device is 300mm. For the pixel point A with the upper left corner coordinates (10, 10) in the image, the device first calculates the actual horizontal coordinate of the pixel point A on the XY plane: 10*(200 / 1920)≈1.04mm, the actual vertical coordinate: 10*(100 / 1080)≈0.93mm, and the vertical coordinate is determined according to the imaging distance as 300mm, so as to obtain the mapping coordinates of the pixel point A in the three-dimensional coordinate system as (1.04, 0.93, 300). The integrated device performs the same calculation on all pixel points in the target PCB image to complete the coordinate mapping.

[0059] Step 1022, according to the mapping coordinates of each pixel point, the distance of each pixel point to the optical axis is calculated, and according to the mapping coordinates of each pixel point, the angle between the projection of the vector formed by each pixel point and the optical axis on the XY plane and the positive direction of the X axis is calculated, to obtain the polar angle at the distance.

[0060] Further, the integrated device calculates the distance of each pixel point to the optical axis (Z axis) according to the mapping coordinates of each pixel point, wherein the distance is calculated using the distance formula between two points in a three-dimensional space, and the distance between the projection point of the pixel point on the XY plane and the corresponding point on the optical axis (Z axis) is calculated.

[0061] Further, the integrated device projects the vector formed by each pixel point and the optical axis to the XY plane with the optical axis as the reference, and calculates the angle between the projection vector and the positive direction of the X axis, that is, the polar angle at the distance.

[0062] Continuing with the above embodiment, for a pixel point A with a mapped coordinate of (1.04, 0.93, 300) in the target PCB plate image collected at a left 45° angle, the distance of the pixel point A to the optical axis is calculated Then, the angle (polar angle) between the vector formed by the pixel point A and the optical axis in the XY plane and the positive direction of the X axis is calculated, which is arctan(0.93 / 1.04)≈41.8°. The integrated device performs the same calculation on all pixel points in the image to obtain the distance of each pixel point to the optical axis and the polar angle.

[0063] In step 1023, the radial gradient is obtained based on the gray level change rate of the pixel point in the radial direction, and the tangential gradient is obtained based on the gray level change rate of the pixel point in the tangential direction.

[0064] Further, the integrated device analyzes the gray level change rates of each pixel point in the radial direction and the tangential direction, where the radial direction refers to the direction from the optical axis to the pixel point, and the tangential direction refers to the direction perpendicular to the radial direction and in the XY plane. The integrated device calculates the gray level difference between adjacent pixel points in the radial and tangential directions, and then divides the distance between the pixel points to obtain the gray level change rate of the pixel point in the radial direction, i.e., the radial gradient; similarly, the tangential gradient is obtained, where the gradient information reflects the gray level change characteristics of the pixel point in different directions.

[0065] Continuing with the pixel point A in the target PCB plate image collected at a left 45° angle as an example, the integrated device selects adjacent pixel points B and C in the radial direction of the pixel point A, calculates their gray level difference with the pixel point A, for example, the gray level value of the pixel point A is 120, the gray level value of the pixel point B is 115, and the gray level value of the pixel point C is 125, and the distance between the pixel points A and B, C in the radial direction is 0.1 mm. Then the radial gradient G r =(125-115) / 0.2=50. In the tangential direction, adjacent pixel points D and E are selected, and the same calculation is performed, for example, the tangential gradient G t =30 is obtained. The integrated device performs similar calculations on all pixel points in the image to obtain the radial gradient and the tangential gradient of each pixel point.

[0066] In step 1024, the composite gradient feature is determined based on the radial gradient and the tangential gradient, and the distance, the polar angle, and the composite gradient feature of each pixel point are determined as the pixel point feature of each pixel point.

[0067] Further, the integrated device combines the radial gradient and the tangential gradient into a composite gradient feature, for example, the vector synthesis method can be used, the radial gradient and the tangential gradient are taken as two components of a vector, and the size and direction of the synthesized vector are calculated as the composite gradient feature.

[0068] Further, the integrated device determines the pixel point feature of the pixel point by the distance of the pixel point to the optical axis, the polar angle and the composite gradient feature, and describes the attributes of the pixel point in the aspects of spatial position, gray scale change and the like.

[0069] Continuing the above embodiment, for the pixel point A in the target PCB image of the left 45° collection angle, the distance is 1.4 mm, the polar angle is 41.8°, the radial gradient G r = 50, and the tangential gradient G t = 30. The size of the composite gradient feature is calculated by using the vector synthesis formula. direction Therefore, the pixel point feature of the pixel point A is (1.4 mm, 41.8°, 58.3, 30.96°), and the integrated device performs the same operation on all the pixel points in the image to determine the pixel point feature of each pixel point.

[0070] The embodiment of the present application constructs the multi-dimensional feature description containing the spatial position and the gray scale change feature for each pixel point, clearly defines the spatial orientation relationship of the pixel point relative to the optical axis, and deeply mines the gray scale change information of the pixel point in different directions, provides rich and accurate data support for the subsequent step of mapping the parallax relationship based on the pixel point feature, can more accurately reflect the corresponding relationship and difference of the pixel points in the images of different collection angles, avoids the positioning deviation caused by the limited view angle, and makes the fused PCB image more clearly and accurately present the details and overall features of the PCB, so that the positioning mark can be accurately recognized, the positioning accuracy of the PCB is improved, and the accuracy and reliability of the PCB test are further improved.

[0071] In an embodiment, the steps 1031 to 1033 are described as follows:

[0072] In step 1031, the distance, the polar angle and the composite gradient feature of each pixel point in the target PCB image under different collection angles are respectively proportionally mapped, offset adjusted and feature fused based on the angle difference between different collection angles, to obtain the mapped distance, the adjusted polar angle and the fused gradient feature of each pixel point in the target PCB image under different collection angles.

[0073] Optionally, the integrated device processes the distance, polar angle and composite gradient features of each pixel point in the target PCB image according to the angle difference between different acquisition angles. For the distance feature, a proportional mapping method is used to adjust the proportional relationship of the distance from the pixel point to the optical axis according to the angle difference; for the polar angle feature, an offset adjustment is performed to offset the polar angle according to the angle difference; and for the composite gradient feature, the composite gradient features of the same pixel point under different acquisition angles are integrated through feature fusion to obtain the mapped distance, adjusted polar angle and fused gradient feature of each pixel point in the target PCB image under different acquisition angles.

[0074] Continuing the above embodiment, there are a top view, a left 45° view and a right 45° view of the target PCB image under different acquisition angles. Figure Three For example, the angle difference between the top view and the left 45° view is 45°. For a pixel point P in the top view, the distance is 2 mm, the polar angle is 30°, and the composite gradient feature is (40, 20°); for a pixel point Q at the corresponding position in the left 45° view, the distance is 1.8 mm, the polar angle is 25°, and the composite gradient feature is (35, 22°). The integrated device performs proportional mapping on the distance feature, calculates the proportional coefficient k = 1.8 / 2 = 0.9 according to the angle difference of 45°, and then the mapped distance of the pixel point P is 2*0.9 = 1.8 mm; performs offset adjustment on the polar angle feature, and the offset is 30°-25° = 5°, so the adjusted polar angle of the pixel point P is 30°-5° = 25°; and performs fusion on the composite gradient feature, for example, the top view weight is 0.6 and the left 45° view weight is 0.4, and the size of the fused gradient feature is 0.6*40+0.4*35 = 38 and the direction is 0.6*20°+0.4*22° = 20.8°. The integrated device performs the same processing on all pixel points in the two views to obtain the mapped distance, adjusted polar angle and fused gradient feature.

[0075] Step 1032, based on the mapped distance, adjusted polar angle and fused gradient feature of each pixel point in the target PCB image under different acquisition angles, the corresponding relationship between each pixel point in the target PCB image under different acquisition angles is determined.

[0076] Further, based on the mapped distance, adjusted polar angle and fused gradient feature of each pixel point in the target PCB image under different acquisition angles, the integrated device determines the corresponding relationship between each pixel point in the target PCB image under different acquisition angles by comparing and matching these features. For example, when the difference between the mapped distances of two pixel points is within a certain range, the difference between the adjusted polar angles is within a certain range, and the similarity of the fused gradient features reaches a certain threshold, the two pixel points are determined as corresponding points.

[0077] Continuing with the above example of the top view and the left 45° view, the integration device traverses the pixel points in both views. For pixel point P in the top view (1.8 mm after mapping, 25° after adjustment, 38 after fusion of gradient features (20.8°)) and pixel point Q in the left 45° view (1.8 mm, 25°, 38 after fusion of gradient features (21°)), since their distances after mapping are equal, the difference in polar angles after adjustment is 21°-20.8°=0.2° within the set range, and the difference in gradient feature size after fusion is 38-38=0, and the difference in direction is 21°-20.8°=0.2° also within the set threshold, so pixel point P and pixel point Q are determined to be corresponding points.

[0078] At step 1033, disparity relationship mapping is performed based on the correspondence between each pixel point in the target PCB images under different acquisition angles, and disparity images between different acquisition angles are obtained.

[0079] Further, the integration device performs disparity relationship mapping according to the correspondence between each pixel point in the target PCB images under different acquisition angles. Specifically, for each group of corresponding pixel points, the position difference of the corresponding pixel points in the image is calculated, and the position difference information is sorted and visualized to generate disparity images between different acquisition angles. The disparity images intuitively show the position offset of the same object (elements on the PCB, etc.) in different angle images.

[0080] Continuing with the above example, after determining the correspondence of all pixel points in the top view and the left 45° view, the position difference of each group of corresponding pixel points is calculated. For example, the horizontal coordinate difference of corresponding pixel points P and Q in the image is 3 pixels, and the vertical coordinate difference is 2 pixels. The device records and processes the position difference information of all corresponding pixel points, and converts the position difference information into gray value or color value through image generation algorithm to generate the disparity image between the top view and the left 45° view. Similarly, the disparity images between the top view and the right 45° view, and between the left 45° view and the right 45° view are generated.

[0081] The embodiment of the application accurately establishes the corresponding relationship between the pixel points in the target PCB plate image under different collection angles, performs proportional mapping, offset adjustment and feature fusion on the pixel point features, eliminates the feature inconsistency problem caused by the difference in collection angles, makes the pixel point features in different angle images comparable, performs disparity relationship mapping according to the corresponding relationship, and generates a disparity image that clearly shows the difference between different angle images, thereby providing a key basis for subsequent image fusion, making it possible to more reasonably integrate different angle image information in the fusion process, effectively eliminating the image overlap, misplacement and other problems caused by the difference in viewing angles, obtaining a high-quality, information-complete fused PCB plate image, and thus accurately identifying the positioning mark and improving the positioning accuracy of the PCB plate and further improving the accuracy and reliability of the PCB plate test.

[0082] In an embodiment, steps 201 to 205 are described as follows:

[0083] Step 201: Based on the shape feature and the gray difference feature of the positioning mark, the pixel points in the fused PCB plate image are regionally screened to determine the candidate region sub-image of the fused PCB plate image.

[0084] Optionally, the shape feature (such as a circular shape, a square shape, a cross shape, etc.) and the gray difference feature (the gray contrast information with the surrounding background) of the positioning mark are pre-stored in the integrated device.

[0085] Therefore, after obtaining the fused PCB plate image, the integrated device regionally screens all the pixel points in the image based on the shape feature and the gray difference feature of the positioning mark, and determines whether the shape and the gray of each pixel point and its neighborhood meet the feature requirements of the positioning mark by traversing the pixel points in the image.

[0086] If the pixel point set in a certain region has a high matching degree in shape and gray with the feature of the positioning mark, the region is determined as a candidate region, and the sub-image corresponding to the region, i.e., the candidate region sub-image, is extracted, which is specifically described in steps 2011 to 2014.

[0087] Step 202: For each pixel point in the candidate region sub-image, the pixel point is screened based on the feature consistency between the pixel point and the neighborhood pixel points to obtain a candidate pixel point.

[0088] Further, for each candidate region sub-image, the integration device analyzes each pixel point therein and calculates the feature consistency between each pixel point and its neighborhood pixel points, where the features include gray scale, color, texture, etc. If the difference between a pixel point and its neighborhood pixel points in these features is small, it means that they have high feature consistency, and the pixel point is retained; otherwise, the pixel point is removed. In this way, the pixel points with high feature consistency, i.e., candidate pixel points, are selected from the candidate region sub-image.

[0089] In an embodiment, taking one of the candidate region sub-images as an example, the integration device processes the pixel points in the sub-image. For pixel point A, its gray scale value is 205, the gray scale values of its neighborhood pixel points are mostly between 200 and 210, and the color and texture are similar, which means that the feature consistency between pixel point A and its neighborhood pixel points is high, and pixel point A is determined as a candidate pixel point; for pixel point B, its gray scale value is 150, which has a large difference with the gray scale values of its neighborhood pixel points, and pixel point B is removed. The integration device performs the same judgment on all the pixel points in the candidate region sub-image to obtain the candidate pixel point set of the sub-image. The same operation is performed on all the candidate region sub-images to obtain all the candidate pixel points.

[0090] In step 203, based on any one of the candidate pixel points as a starting point, the pixel points with connection relationship are connected according to the eight-neighborhood search to obtain a candidate shape contour.

[0091] Further, the integration device selects any one of the candidate pixel points as a starting point and uses the eight-neighborhood search algorithm, which searches the eight neighboring pixel points (up, down, left, right, upper left, upper right, lower left, and lower right) of the starting point and judges whether the neighboring pixel points are candidate pixel points. If they are candidate pixel points, they are connected with the starting point, and the neighboring pixel points are taken as new starting points to continue searching the candidate pixel points in their eight neighborhoods, and the process is repeated until there is no candidate pixel point to be connected. Through the above process, the pixel points with connection relationship are connected to form a closed contour, i.e., a candidate shape contour, and the same operation is performed on all the candidate pixel points to obtain multiple candidate shape contours.

[0092] In an embodiment, pixel point C is selected as a starting point in the candidate pixel point set, and the eight neighborhoods of the starting point are searched to find that pixel points D and E are candidate pixel points, and pixel point C is connected with pixel points D and E. Then, pixel point D is taken as a new starting point to continue searching its eight neighborhoods, and candidate pixel point F is found and connected. This process is continuously performed to finally form a closed contour, i.e., a candidate shape contour, and the above process is repeated on other candidate pixel points to obtain multiple candidate shape contours, such as a circular contour and an irregular shape contour.

[0093] Step 204, profile screening is performed based on the similarity between the standard shape profile of the positioning mark under different scales and each candidate shape profile, and a target shape profile is obtained.

[0094] Further, the integrated device pre-stores the standard shape profile of the positioning mark under different scales. Therefore, for each candidate shape profile, the similarity between it and the standard shape profile under different scales is calculated, wherein the similarity calculation can adopt a profile matching algorithm, such as matching based on profile point distance, matching based on Fourier descriptor, etc. By comparing the similarity of the candidate shape profile and the standard shape profile, the candidate shape profile with the highest similarity is selected as the target shape profile.

[0095] In an embodiment, the integrated device stores the standard shape profile of the circular positioning mark under small, medium and large scales. For a certain candidate shape profile G, the similarity between it and the standard circular profile under three scales is calculated respectively. Through the profile point distance-based matching algorithm, the similarity between the candidate shape profile G and the standard circular profile under small scale is 0.6, the similarity between the candidate shape profile G and the standard circular profile under medium scale is 0.85, and the similarity between the candidate shape profile G and the standard circular profile under large scale is 0.7. Since the similarity with the standard circular profile under medium scale is the highest, the candidate shape profile G is determined as the target shape profile. The same calculation and screening are performed on all candidate shape profiles to obtain the final target shape profile.

[0096] Step 205, the barycenter position in the target shape profile is determined as the mark position of the positioning mark in the fused PCB board image.

[0097] Further, the integrated device analyzes the target shape profile and calculates the barycenter position of the profile. For a shape profile on a two-dimensional plane, the barycenter position can be calculated by integral formula or discretization method (such as weighted average of pixel point coordinates on the profile), and the calculated barycenter position is determined as the mark position of the positioning mark in the fused PCB board image.

[0098] The embodiment of the present application can accurately locate the position of the mark in the fused PCB board image, so as to accurately identify the positioning mark and improve the positioning accuracy of the PCB board.

[0099] In an embodiment, steps 2011 to 2014 are described as follows:

[0100] Step 2011, the shape entropy of each pixel point in the fused PCB board image in its first preset size neighborhood is calculated based on the shape feature.

[0101] Optionally, for each pixel point in the fused PCB image, the integration device analyzes the distribution of pixel points in a first preset size neighborhood centered on the pixel point to calculate shape entropy, wherein the shape entropy is used to measure the similarity between the shape formed by the pixel points in the neighborhood and the standard shape of the positioning mark. The lower the shape entropy value, the closer the shape formed by the pixel points in the neighborhood to the standard shape of the positioning mark. In the calculation, the distribution of pixel points in the neighborhood is counted, and the shape entropy of each pixel point in its neighborhood is obtained by combining the calculation method of information entropy.

[0102] In an embodiment, the positioning mark on the PCB is a circle, and the integration device sets the first preset size as a 5*5 neighborhood centered on the pixel point. For a pixel point A with coordinates (10, 10) in the fused PCB image, the distribution of pixel points in the 5*5 neighborhood of the pixel point A is analyzed. If the pixel points in the neighborhood are mostly concentrated in the circular region with the pixel point A as the center, the shape entropy of the pixel point A is calculated to be 0.2 by a specific shape entropy calculation formula (for example, calculating the information entropy according to the distance distribution of the pixel points to the center). If the pixel points in the neighborhood are distributed in disorder, the calculated shape entropy may be higher, such as 0.8. The shape entropy of each pixel point in the fused PCB image is obtained by performing such calculation.

[0103] Step 2012, based on the gray difference feature, the gray gradient contrast between each pixel point in the fused PCB image and the pixel points in its second preset size neighborhood is calculated.

[0104] Further, for each pixel point in the fused PCB image, the integration device calculates the gray gradient contrast between the pixel point and other pixel points in its second preset size neighborhood, wherein the gray gradient contrast reflects the difference in gray change between the pixel point and its neighborhood pixel points. The higher the contrast, the more obvious the gray difference between the pixel point and the surrounding pixel points, and the more likely it belongs to the positioning mark region. The integration device obtains the gray gradient contrast by calculating the difference between the pixel point and its neighborhood pixel points and performing normalization processing.

[0105] Continuing to take the pixel point A with the coordinate (10, 10) in the fused PCB board image as an example, a second preset size is set as a 3*3 neighborhood centered on the pixel point A. For example, the gray value of the pixel point A is 200, and the gray values of other pixel points in the neighborhood are mostly around 100. The maximum difference of the gray values of the pixel point A and the pixel points in the neighborhood is calculated, and normalized processing is performed. For example, if the maximum difference is 100, the normalized gray gradient contrast is 0.8. If the gray values of the pixel points in the neighborhood and the pixel point A have small differences, the calculated gray gradient contrast can be low, such as 0.2. The gray gradient contrast of each pixel point in the fused PCB board image is calculated in this way.

[0106] In step 2013, the coupling degree value of each pixel point in the fused PCB board image is determined based on the shape entropy and the gray gradient contrast of each pixel point.

[0107] Further, the integration device combines the shape entropy and the gray gradient contrast of each pixel point, and determines the coupling degree value of each pixel point through a pre-designed calculation manner. The coupling degree value is used to comprehensively measure the degree to which the pixel point meets the positioning mark feature, and reflects the matching of the pixel point in terms of shape and gray with the positioning mark feature. In the embodiment of the present application, a weighted summation or the like can be used to assign different weights to the shape entropy and the gray gradient contrast, and calculate the coupling degree value. For example, if the shape feature is considered to be more important, a higher weight can be assigned to the shape entropy, and vice versa.

[0108] In an embodiment, the integration device assigns a weight of 0.6 to the shape entropy, and a weight of 0.4 to the gray gradient contrast. For the pixel point A, the shape entropy is 0.2, and the gray gradient contrast is 0.8. The coupling degree value of the pixel point A is calculated through the formula "coupling degree value = shape entropy*0.6 + gray gradient contrast*0.4", which is 0.2*0.6 + 0.8*0.4 = 0.44. The coupling degree values of all the pixel points in the fused PCB board image are calculated in the same way.

[0109] In step 2014, the candidate region sub-image of the fused PCB board image is determined based on the coupling degree value of each pixel point in the fused PCB board image.

[0110] Further, the integration device determines the candidate region sub-image of the fused PCB board image according to the coupling degree value of each pixel point in the fused PCB board image, which is specifically described in steps 20141 to 20144.

[0111] The embodiment of the present application can efficiently and accurately screen out the candidate region sub-image in the fused PCB plate image based on the shape and gray difference characteristics of the positioning mark, effectively eliminates a large number of irrelevant regions and interference information in the image, greatly reduces the search range of subsequent positioning mark identification, lays a solid foundation for more accurately determining the positioning mark position, and therefore can accurately identify the positioning mark, and improves the positioning accuracy of the PCB plate.

[0112] In an embodiment, steps 20141 to 20144 are described as follows:

[0113] Step 20141, traverse the coupling degree value of each pixel point in the fused PCB plate image, and judge whether the coupling degree value is greater than the coupling degree values of all pixel points in its neighborhood, if yes, mark it as a local extreme point.

[0114] Optionally, the integrated device traverses the coupling degree value of each pixel point in the fused PCB plate image, and examines the coupling degree values of all pixel points in its neighborhood with each pixel point as the center, wherein the neighborhood size can be pre-set according to actual needs, for example, 3*3, 5*5, etc. If the coupling degree value of a certain pixel point is greater than the coupling degree values of all pixel points in its neighborhood, it means that the pixel point has a higher coupling degree advantage in its neighborhood, and it is marked as a local extreme point, wherein the local extreme point represents a relatively prominent position in the image in terms of shape and gray comprehensive characteristics, which may be associated with the positioning mark.

[0115] In an embodiment, a 3*3 neighborhood centered on each pixel point is used for judgment. In the fused PCB plate image, for the pixel point P with coordinates (20, 20), its coupling degree value is 0.6, and in its 3*3 neighborhood, the coupling degree values of the other 8 pixel points are 0.3, 0.4, 0.5, 0.45, 0.52, 0.38, 0.42, and 0.55, all of which are less than the coupling degree value of the pixel point P. Therefore, the pixel point P is marked as a local extreme point. All pixel points in the image are traversed and judged in this way to mark all local extreme points.

[0116] Step 20142, cluster the local extreme points according to the shape characteristics of the pixel points, so as to classify the pixel points with similar shapes into a class, and obtain a candidate region class.

[0117] Further, a shape template or feature description of the positioning mark is pre-stored, and the integrated device clusters the marked local extreme points according to the shape features of the pixel points, and classifies the local extreme points with similar shapes into a class by calculating the similarity of the local extreme points and the shape templates, to obtain different candidate region classes, wherein the clustering algorithm can adopt common K-Means clustering, DBSCAN clustering, and the like, and a suitable algorithm is selected according to the number and distribution characteristics of the local extreme points.

[0118] In an embodiment, the positioning mark on the PCB is circular, and the K-Means clustering algorithm is adopted, and the clustering number K is set to 3. The similarity of the shape formed by each local extreme point and its neighborhood pixel points with the circular template is calculated, for example, by calculating the circularity index of the pixel point distribution. The local extreme points with similar circularity indexes are classified into a class, and after clustering, three candidate region classes are obtained, wherein class 1 contains local extreme points with higher circularity, and classes 2 and 3 contain local extreme points with greater differences in circularity. In actual application, class 1 is more likely to correspond to the region where the positioning mark is located.

[0119] In step 20143, for the pixel points in each candidate region class, the gray mean and variance of the pixel points are calculated, and the pixel points with a gray value within a preset variance range are merged to obtain candidate region pixel points.

[0120] Further, for the pixel points in each candidate region class, the integrated device calculates the gray mean and variance of all pixel points in the class, wherein the gray mean reflects the average level of the gray of the pixel points in the class, and the variance reflects the dispersion degree of the gray values of the pixel points.

[0121] Further, a preset variance range is set, and the integrated device merges the pixel points with a gray value within the preset variance range, considers that these pixel points have high consistency in the gray feature, and are more likely to belong to the same target region, and the merged pixel point set is the candidate region pixel points.

[0122] Continuing the above embodiment, the preset variance range is 5-15, and taking candidate region class 1 as an example, the class contains 100 pixel points, and the integrated device calculates that the gray mean of the pixel points is 180 and the variance is 10. The pixel points with a gray value within the range of 170-190 (calculated according to the mean and variance range) are selected, for example, there are 80 pixel points that meet the condition, and the 80 pixel points are merged to obtain a candidate region pixel point set. The same operation is performed on other candidate region classes to obtain respective candidate region pixel points.

[0123] Step 20144, according to the candidate region pixel points, a corresponding sub-image is extracted from the fused PCB board image, and a candidate region sub-image is obtained.

[0124] Further, the integrated device finds the positions corresponding to the candidate region pixel points in the fused PCB board image according to the candidate region pixel points, extracts a minimum rectangular region containing the pixel points or a region fitted according to the actual shape, and obtains a candidate region sub-image.

[0125] In an embodiment, the candidate region pixel points are distributed in a rectangular region with coordinates (30, 30)-(80, 80) in the fused PCB board image, and the integrated device extracts the rectangular region from the fused PCB board image to obtain a candidate region sub-image. For other candidate region pixel point sets, the corresponding sub-image is extracted in the same way, and finally a plurality of candidate region sub-images are obtained.

[0126] The embodiment of the application highlights the position that is prominent in the shape and gray scale comprehensive features of the image by marking the local extreme points, excludes the interference region of shape mismatching based on the clustering of shape features, and makes the candidate region pixel points have higher consistency in gray scale by calculating the gray mean and variance and merging the pixel points meeting the conditions, so that the candidate region pixel points are closer to the real features of the positioning marker, thereby providing high-targeted data for subsequent accurate recognition of the positioning marker position, and thus the positioning marker can be accurately recognized, and the positioning accuracy of the PCB board is improved.

[0127] Optionally, referring to Figure 2 , Figure 2 is a flowchart of the high-precision PCB board positioning and multifunctional test integrated method based on visual recognition provided by the application. In the embodiment of the application, the execution subject of the high-precision PCB board positioning and multifunctional test integrated method based on visual recognition is an integrated device, therefore, the high-precision PCB board positioning and multifunctional test integrated method based on visual recognition comprises:

[0128] Step 10, multi-angle image acquisition of the PCB board to be tested on the conveying belt is performed by an industrial camera, at least three initial PCB board images of different acquisition angles are obtained, and the initial PCB board images of different acquisition angles are fused to obtain a fused PCB board image.

[0129] Optionally, the industrial camera in the integrated device is arranged above or beside the conveying belt, and when the PCB to be tested moves to the designated position with the conveying belt, the industrial camera starts the multi-angle image acquisition program. Through the preset angle parameters, the industrial camera sequentially acquires images of the PCB from at least three different angles, and acquires one initial PCB image at each angle. After the acquisition is completed, the integrated device fuses the initial PCB images at different acquisition angles by using an image fusion algorithm, eliminates the missing image information caused by the difference in viewing angles, and obtains a fused PCB image containing complete PCB information.

[0130] Step 20, based on the positioning marks on the PCB identified in the fused PCB image, the mark positions of the positioning marks in the fused PCB image are determined.

[0131] Further, the integrated device identifies the positioning marks on the PCB in the fused PCB image based on computer vision technology, wherein the positioning marks are usually marks with specific shapes, colors or patterns, such as circular marks, cross marks, etc. Therefore, the integrated device searches for the positioning marks in the image by using image recognition algorithms such as template matching, feature point detection, etc., and determines the accurate coordinate positions of the positioning marks in the fused PCB image, i.e. the mark positions.

[0132] Step 30, based on the mark positions and the geometric relationship, the initial pose information and the initial position information of the PCB in the image coordinate system are determined, and the initial pose information and the initial position information are converted to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system, to obtain the target pose information and the target position information of the PCB in the world coordinate system.

[0133] Further, the integrated device calculates the initial pose information (such as rotation angle) and the initial position information (such as center point coordinates) of the PCB in the image coordinate system according to the mark positions of the positioning marks in the fused PCB image and the geometric relationship. For example, by calculating the distance and angle relationship between the positioning marks, the rotation angle of the PCB can be determined; by calculating the center coordinates of multiple positioning marks, the center point position of the PCB in the image coordinate system can be determined.

[0134] Further, after obtaining the initial pose information and the initial position information, the integrated device converts the initial pose information and the initial position information to the world coordinate system according to the conversion relationship (such as by using a calibration board to calibrate the coordinate systems to obtain a rotation matrix and a translation vector) between the image coordinate system and the world coordinate system, to obtain the target pose information and the target position information of the PCB in the world coordinate system.

[0135] Step 40, according to the target posture information and the target position information, the mechanical arm carries the test probe to control the PCB board to carry out multifunctional test.

[0136] Further, the integrated device controls the mechanical arm to move according to the target posture information and the target position information of the PCB board in the world coordinate system. The mechanical arm carries the test probe to carry out multifunctional test on each test point on the PCB board according to the preset test program, such as electrical performance test, signal integrity test, etc. During the test process, the test data is collected in real time, and whether the test result is qualified is judged according to the test standard.

[0137] Step 50, if the PCB board test is passed, the mechanical arm is controlled to move away the test probe, and the PCB board is conveyed to the next process through the conveying belt.

[0138] Further, if all tests of the PCB board are passed, the integrated device controls the mechanical arm to move away the test probe to avoid interference to the subsequent process.

[0139] Further, the integrated device controls the conveying belt to start to convey the test passed PCB board to the next process. If the PCB board test is not passed, the integrated device controls the mechanical arm to remove the PCB board from the conveying belt and place it to the unqualified product area for subsequent repair or scrap processing.

[0140] The embodiment of the application can capture the detailed features of the PCB board from multiple perspectives by acquiring PCB board images at least three different collection angles, avoid positioning deviation caused by limited perspective, and determine the marker position through the positioning marker identified in the image and calculate the posture information and position information combined with the geometric relationship, so that the positioning marker can be accurately identified, the subtle differences of different batches and different specifications of PCB boards can be adapted, and the positioning accuracy of the PCB board is improved. On the other hand, the non-contact visual recognition technology is used for positioning, after the positioning is completed, the mechanical arm carries the test probe to automatically carry out multifunctional test on the PCB board, the whole process does not need manual intervention, the mechanical arm executes the test task quickly and orderly according to the program setting, avoids the cumbersome operation and time loss in the process of the contact sensor, makes the positioning and test link closely, reduces the waiting time, and improves the test efficiency of the PCB board.

[0141] Please refer to Figure 3 , Figure 3 The embodiment of the electronic device provided by the embodiment of the application is shown in the figure. Figure 3 As shown in the figure, the embodiment of the application provides an electronic device 300, which includes a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, the following steps are implemented:

[0142] acquire initial PCB board images of at least three different acquisition angles through multi-angle image acquisition of the PCB board to be tested on the conveying belt by an industrial camera, and fuse the initial PCB board images of different acquisition angles to obtain a fused PCB board image;

[0143] determine a mark position of the positioning mark in the fused PCB board image based on the positioning mark on the PCB board recognized in the fused PCB board image;

[0144] determine initial attitude information and initial position information of the PCB board in an image coordinate system based on the mark position and geometric relationship, and convert the initial attitude information and the initial position information to a world coordinate system according to a conversion relationship between the image coordinate system and the world coordinate system to obtain target attitude information and target position information of the PCB board in the world coordinate system;

[0145] control a test probe carried by a mechanical arm to perform multifunctional testing on the PCB board according to the target attitude information and the target position information;

[0146] if the PCB board passes the test, control the mechanical arm to move the test probe away, and convey the PCB board to a next process through the conveying belt.

[0147] Please refer to Figure 4 , Figure 4 The embodiment of the computer readable storage medium provided by the embodiment of the application is shown in the figure. As shown in Figure 4 The embodiment provides a computer readable storage medium 400, which stores a computer program 311, and the computer program 311 is executed by a processor to realize the following steps:

[0148] acquire initial PCB board images of at least three different acquisition angles through multi-angle image acquisition of the PCB board to be tested on the conveying belt by an industrial camera, and fuse the initial PCB board images of different acquisition angles to obtain a fused PCB board image;

[0149] determine a mark position of the positioning mark in the fused PCB board image based on the positioning mark on the PCB board recognized in the fused PCB board image;

[0150] determine initial attitude information and initial position information of the PCB board in an image coordinate system based on the mark position and geometric relationship, and convert the initial attitude information and the initial position information to a world coordinate system according to a conversion relationship between the image coordinate system and the world coordinate system to obtain target attitude information and target position information of the PCB board in the world coordinate system;

[0151] control a test probe carried by a mechanical arm to perform multifunctional testing on the PCB board according to the target attitude information and the target position information;

[0152] If the PCB board test is passed, the control arm moves the test probe away, and the PCB board is conveyed to the next process through the conveyor belt.

[0153] In another aspect, the present application also provides a computer program product, the computer program product comprising a computer program, the computer program being stored on a non-transitory computer readable storage medium, and the computer program being executable by a processor to cause the computer to perform the method of high-precision PCB board positioning and multifunctional test integration based on visual recognition provided by the above-mentioned methods, the method of high-precision PCB board positioning and multifunctional test integration based on visual recognition comprising:

[0154] acquiring at least three initial PCB board images of different acquisition angles through multi-angle image acquisition of the PCB board to be tested on the conveyor belt by an industrial camera, and fusing the initial PCB board images of different acquisition angles to obtain a fused PCB board image;

[0155] determining a mark position of the positioning mark in the fused PCB board image based on the positioning mark on the PCB board recognized in the fused PCB board image;

[0156] determining initial attitude information and initial position information of the PCB board in the image coordinate system based on the mark position and the geometric relationship, and converting the initial attitude information and the initial position information to the world coordinate system according to the conversion relationship between the image coordinate system and the world coordinate system to obtain target attitude information and target position information of the PCB board in the world coordinate system;

[0157] controlling the test probe carried by the mechanical arm to perform multifunctional test on the PCB board according to the target attitude information and the target position information;

[0158] If the PCB board test is passed, the control arm moves the test probe away, and the PCB board is conveyed to the next process through the conveyor belt.

[0159] The apparatus embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0160] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0161] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A PCB board positioning and multi-functional testing integrated device based on vision recognition, characterized in that, The device includes an integrated controller, an image processing unit, a vision recognition unit, a PCB board positioning unit, a PCB board testing unit, and a robotic arm control unit; the integrated controller is connected to the image processing unit, the vision recognition unit, the PCB board positioning unit, the PCB board testing unit, and the robotic arm control unit respectively, and manages each unit; The image processing unit is used to acquire multi-angle images of the PCB board to be tested on the conveyor belt using an industrial camera, obtain at least three initial PCB board images from different acquisition angles, and fuse the initial PCB board images from different acquisition angles to obtain a fused PCB board image. A visual recognition unit is used to determine the position of the positioning mark in the fused PCB board image based on the positioning mark on the PCB board identified in the fused PCB board image. The PCB board positioning unit is used to determine the initial pose information and initial position information of the PCB board in the image coordinate system based on the marked position and geometric relationship, and to transform the initial pose information and initial position information to the world coordinate system according to the transformation relationship between the image coordinate system and the world coordinate system, so as to obtain the target pose information and target position information of the PCB board in the world coordinate system. The PCB board testing unit is used to control the test probes mounted on the robotic arm to perform multi-functional tests on the PCB board based on the target posture information and the target position information. The robotic arm control unit is used to control the robotic arm to remove the test probes and transfer the PCB board to the next process via the conveyor belt if the PCB board passes the test. The step of fusing initial PCB board images from different acquisition angles to obtain a fused PCB board image includes: Image segmentation is performed based on the edge transparency channel threshold of each pixel in the initial PCB board image at each acquisition angle to obtain the target PCB board image at each acquisition angle; Based on the lens optical axis at each acquisition angle and each pixel in the target PCB board image, determine the pixel features of each pixel in the target PCB board image at each acquisition angle. Based on the camera parameters at different acquisition angles and the pixel features of each pixel in the target PCB board image, a parallax relationship mapping is performed to obtain parallax images between different acquisition angles. The image fusion level is determined based on the angle difference between different acquisition angles, and the parallax images between different acquisition angles are fused according to the image fusion level between different acquisition angles to obtain the fused PCB board image. The process of determining the pixel features of each pixel in the target PCB board image at each acquisition angle, based on the lens optical axis at each acquisition angle and each pixel in the target PCB board image, includes: For each acquisition angle of the target PCB board image, a three-dimensional coordinate system is established with the lens optical axis as the Z-axis and the plane perpendicular to the optical axis and passing through the center of the image as the XY plane. The original coordinates of each pixel in the target PCB board image are mapped to the three-dimensional coordinate system to obtain the mapped coordinates of each pixel. The horizontal and vertical coordinates of the mapped coordinates are determined based on the original coordinates combined with the image resolution and the width and height of the imaging field of view on the XY plane. The vertical coordinate of the mapped coordinates is determined based on the imaging distance of the acquisition device. The distance from each pixel to the optical axis is calculated based on the mapped coordinates of each pixel. Then, using the optical axis as a reference, the angle between the vector formed by each pixel and the optical axis projected onto the XY plane and the positive direction of the X-axis is calculated based on the mapped coordinates of each pixel. The polar angle at the distance is obtained. The radial gradient is obtained based on the rate of gray-level change of the pixel in the radial direction, and the tangential gradient is obtained based on the rate of gray-level change of the pixel in the tangential direction. The composite gradient features are determined based on the radial gradient and the tangential gradient, and the distance, polar angle and composite gradient features of each pixel are determined as the pixel features of each pixel. The parallax relationship is mapped based on camera parameters from different acquisition angles and pixel features of each pixel in the target PCB board image to obtain parallax images between different acquisition angles, including: Based on the angle difference between different acquisition angles, the distance, polar angle and composite gradient features of each pixel in the target PCB board image under different acquisition angles are proportionally mapped, offset adjusted and feature fused respectively to obtain the mapped distance, adjusted polar angle and fused gradient features of each pixel in the target PCB board image under different acquisition angles. Based on the mapped distance, adjusted polar angle, and fused gradient features of each pixel in the target PCB board image under different acquisition angles, the correspondence between each pixel in the target PCB board image under different acquisition angles is determined. Based on the correspondence between each pixel in the target PCB board image under different acquisition angles, a parallax relationship mapping is performed to obtain the parallax image between different acquisition angles.

2. The integrated PCB board positioning and multi-functional testing device based on vision recognition according to claim 1, characterized in that, Based on the positioning marks on the PCB board identified in the fused PCB board image, the position of the positioning marks in the fused PCB board image is determined, including: Based on the shape features and grayscale difference features of the positioning marks, the pixels in the fused PCB board image are filtered to determine the candidate region sub-images of the fused PCB board image. For each pixel in a candidate region sub-image, pixel selection is performed based on the feature consistency between the pixel and its neighboring pixels to obtain candidate pixels. Starting from any pixel among the candidate pixels, connect the pixels with connectivity using an eight-neighbor search to obtain the candidate shape contour. The target shape contour is obtained by screening the contours based on the similarity between the standard shape contour of the positioning mark and each candidate shape contour at different scales. The centroid position in the target shape contour is determined as the mark position of the positioning mark in the fused PCB board image.

3. The integrated PCB board positioning and multi-functional testing device based on vision recognition according to claim 2, characterized in that, The step of filtering pixels in the fused PCB board image based on the shape features and grayscale difference features of the positioning marks to determine candidate sub-images of the fused PCB board image includes: Based on the shape features, the shape entropy of each pixel in the fused PCB board image within its first preset size neighborhood is calculated. Based on the gray-level difference features, calculate the gray-level gradient contrast between each pixel in the fused PCB board image and its inner pixels in a second preset size neighborhood. The coupling degree value of each pixel is determined based on the shape entropy and grayscale gradient contrast of each pixel in the fused PCB board image. Based on the coupling degree value of each pixel in the fused PCB board image, region filtering is performed to determine the candidate region sub-images of the fused PCB board image.

4. The integrated PCB board positioning and multi-functional testing device based on visual recognition according to claim 3, characterized in that, The step of performing region filtering based on the coupling degree value of each pixel in the fused PCB board image to determine candidate region sub-images of the fused PCB board image includes: Traverse the coupling degree value of each pixel in the fused PCB board image, and determine whether its coupling degree value is greater than the coupling degree value of all pixels in its neighborhood. If so, mark it as a local extreme point. Local extreme points are clustered based on the shape features of pixels to group pixels with similar shapes into one category, thus obtaining candidate region categories; For each pixel in the candidate region category, calculate its grayscale mean and variance, and merge pixels whose grayscale values ​​are within the preset variance range to obtain candidate region pixels; Based on the candidate region pixels, the corresponding sub-images are extracted from the fused PCB board image to obtain the candidate region sub-images.

5. A high-precision PCB board positioning and multi-functional testing integrated method based on vision recognition, implemented based on the PCB board positioning and multi-functional testing integrated device based on vision recognition as described in any one of claims 1 to 4, characterized in that, The integrated method for high-precision PCB board positioning and multi-functional testing based on visual recognition includes: The PCB board to be tested on the conveyor belt is captured from multiple angles by an industrial camera, and at least three initial PCB board images from different acquisition angles are obtained. The initial PCB board images from different acquisition angles are then fused to obtain a fused PCB board image. Based on the positioning marks on the PCB board identified in the fused PCB board image, the position of the positioning marks in the fused PCB board image is determined; Based on the marked positions and geometric relationships, the initial pose information and initial position information of the PCB board in the image coordinate system are determined, and the initial pose information and initial position information are transformed to the world coordinate system according to the transformation relationship between the image coordinate system and the world coordinate system, so as to obtain the target pose information and target position information of the PCB board in the world coordinate system. The test probe mounted on the robotic arm is controlled to perform multi-functional tests on the PCB board based on the target posture information and the target position information. If the PCB board passes the test, the robotic arm is controlled to remove the test probes and the PCB board is conveyed to the next process via the conveyor belt. The step of fusing initial PCB board images from different acquisition angles to obtain a fused PCB board image includes: Image segmentation is performed based on the edge transparency channel threshold of each pixel in the initial PCB board image at each acquisition angle to obtain the target PCB board image at each acquisition angle; Based on the lens optical axis at each acquisition angle and each pixel in the target PCB board image, determine the pixel features of each pixel in the target PCB board image at each acquisition angle. Based on the camera parameters at different acquisition angles and the pixel features of each pixel in the target PCB board image, a parallax relationship mapping is performed to obtain parallax images between different acquisition angles. The image fusion level is determined based on the angle difference between different acquisition angles, and the parallax images between different acquisition angles are fused according to the image fusion level between different acquisition angles to obtain the fused PCB board image. The process of determining the pixel features of each pixel in the target PCB board image at each acquisition angle, based on the lens optical axis at each acquisition angle and each pixel in the target PCB board image, includes: For each acquisition angle of the target PCB board image, a three-dimensional coordinate system is established with the lens optical axis as the Z-axis and the plane perpendicular to the optical axis and passing through the center of the image as the XY plane. The original coordinates of each pixel in the target PCB board image are mapped to the three-dimensional coordinate system to obtain the mapped coordinates of each pixel. The horizontal and vertical coordinates of the mapped coordinates are determined based on the original coordinates combined with the image resolution and the width and height of the imaging field of view on the XY plane. The vertical coordinate of the mapped coordinates is determined based on the imaging distance of the acquisition device. The distance from each pixel to the optical axis is calculated based on the mapped coordinates of each pixel. Then, using the optical axis as a reference, the angle between the vector formed by each pixel and the optical axis projected onto the XY plane and the positive direction of the X-axis is calculated based on the mapped coordinates of each pixel. The polar angle at the distance is obtained. The radial gradient is obtained based on the rate of gray-level change of the pixel in the radial direction, and the tangential gradient is obtained based on the rate of gray-level change of the pixel in the tangential direction. The composite gradient features are determined based on the radial gradient and the tangential gradient, and the distance, polar angle and composite gradient features of each pixel are determined as the pixel features of each pixel. The parallax relationship is mapped based on camera parameters from different acquisition angles and pixel features of each pixel in the target PCB board image to obtain parallax images between different acquisition angles, including: Based on the angle difference between different acquisition angles, the distance, polar angle and composite gradient features of each pixel in the target PCB board image under different acquisition angles are proportionally mapped, offset adjusted and feature fused respectively to obtain the mapped distance, adjusted polar angle and fused gradient features of each pixel in the target PCB board image under different acquisition angles. Based on the mapped distance, adjusted polar angle, and fused gradient features of each pixel in the target PCB board image under different acquisition angles, the correspondence between each pixel in the target PCB board image under different acquisition angles is determined. Based on the correspondence between each pixel in the target PCB board image under different acquisition angles, a parallax relationship mapping is performed to obtain the parallax image between different acquisition angles.

6. An electronic device, comprising: The memory and processor are characterized in that the memory stores a computer software program, and when the processor reads and executes the computer software program, it realizes the integrated method for high-precision PCB board positioning and multi-functional testing based on vision recognition as described in claim 5.

7. A non-transitory computer-readable storage medium, characterized in that, The storage medium stores a computer software program, which, when executed by a processor, implements the integrated method for high-precision PCB board positioning and multi-functional testing based on visual recognition as described in claim 5.

Images