Machine vision-based 5g antenna pose estimation method, device and medium

Abstract

The application discloses a 5G antenna pose estimation method and device based on machine vision and a medium, and the method comprises the following steps: 1, detecting an antenna in an image and extracting an ROI region image; 2, performing feature extraction and feature matching on the ROI region antenna image; 3, calculating the pixel coordinates of feature points in the original image; 4, calculating the initial value of the coordinates of the 5G antenna feature points in the base coordinate system of a mechanical arm based on the principle of triangulation; 5, using the LM algorithm to solve the optimal coordinates of the feature points by taking the repositioning error as a target function; 6, fitting the optimal coordinate set to obtain the normal vector of the antenna plane by using the SVD decomposition method; 7, establishing an antenna coordinate system and calculating the x, y and z axis direction vectors; and 8, calculating the antenna rotation matrix and the translation vector to obtain the antenna pose. The application can improve the accuracy and efficiency of antenna pose estimation and reduce the system complexity.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of positioning detection, in particular to a 5G antenna pose estimation method, device and medium based on machine vision. BACKGROUND

[0002] With the rapid development of 5G communication technology, the testing of 5G antenna performance becomes particularly critical. In a 5G antenna near-field detection system based on a mechanical arm, the size and height of the sampling surface need to be set according to the working frequency and size of the 5G antenna, and the position of the sampling surface is determined according to the pose of the 5G antenna. The position of the sampling surface determines the sampling trajectory of the magnetic field probe at the end of the mechanical arm. Only when the position and direction of the antenna match the position and direction of the magnetic field probe, can the electromagnetic performance of the antenna be accurately measured.

[0003] Machine vision can complete object recognition, positioning and measurement tasks without contact, and plays a crucial role in the development of today's industrial automation and robotics. Among them, the method based on monocular vision is simple and easy to implement, but it cannot provide the true scale information of the target object; the method based on binocular vision can recover the three-dimensional information of the object, but it has high requirements for environmental lighting and texture, and the stereo matching algorithm is complex; the method based on multi-view vision can provide more comprehensive perspective information, but it has large amount of calculation, high cost, and increased system complexity, which is not conducive to wide application in actual production environment. Therefore, the above methods have certain limitations in actual application. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a 5G antenna pose estimation method, device and medium based on machine vision, in order to improve the accuracy and efficiency of antenna pose estimation, and reduce the system complexity.

[0005] To achieve the above-mentioned purpose, the following technical solutions are adopted in the present application:

[0006] The 5G antenna pose estimation method based on machine vision of the present application is characterized by the following steps:

[0007] S1: When the monocular camera reaches the i-th acquisition point , the monocular camera collects the i-th antenna image at the i-th mechanical arm pose , detects the antenna in using the YOLOv8 model, obtains the i-th ROI region antenna image , and marks the upper left corner coordinates of in as ;

[0008] S2: feature extraction and feature matching are performed on the i-1th and the ith ROI region antenna images and to obtain the 5G antenna feature points not in a line on and respectively corresponding to the pixel pairs , ; wherein, represents the jth 5G antenna feature point; represents the coordinate on , represents the coordinate on ; m represents the total number of feature points, m≥3; and represent the jth pixel pair;

[0009] S3: the pixel coordinates of the m pixel pairs in , , are calculated using formula (1), wherein, represents the coordinate on , represents the coordinate on :

[0010] (1)

[0011] In formula (1), represents the top-left corner coordinate in the i-1th antenna image ;

[0012] S4: based on the principle of triangulation, the coordinate of the jth 5G antenna feature point in the mechanical arm base coordinate system is calculated using formula (2):

[0013] (2)

[0014] In formula (2), is the intrinsic matrix of the monocular camera, and is obtained through camera calibration; , are respectively the rotation matrix and the translation vector of the camera coordinate system relative to the mechanical arm base coordinate system when the monocular camera takes the i-th photo, and are obtained through coordinate transformation; , ​respectively represent the rotation matrix and the translation vector of the monocular camera relative to the base coordinate system of the robot arm at the i-1th time of photographing, and are obtained through coordinate transformation; represent the depth values of the collection points to at the i-1th time of photographing of the monocular camera; represent the depth values of the collection points to at the i-1th time of photographing of the monocular camera;

[0015] S5: iteratively optimize the target function shown in formula (3) by using the LM algorithm, take the calculation result of S4 as the initial value of the LM algorithm, and solve the optimal coordinates of the jth 5G antenna feature point in the base coordinate system of the robot arm;

[0016] (3)

[0017] In formula (3), represents the repositioning error between and ;

[0018] S6: fit the optimal coordinate set to obtain the normal vector of the antenna plane by using the SVD decomposition method; ; wherein T represents transposition;

[0019] S7: take the direction of the normal vector of the antenna plane as the axis direction vector of the antenna coordinate system, denoted as ; select a certain vector direction on the antenna plane as the axis direction vector of the antenna coordinate system, denoted as ; obtain the axis direction vector by calculating the cross product of the axis direction vector and the axis direction vector, denoted as ;

[0020] S8: obtain the transformation matrix of the antenna coordinate system relative to the base coordinate system of the robot arm by using formula (4) , that is, and the pose of the antenna in

[0021] (4)

[0022] In formula (4), is the rotation matrix of the antenna coordinate system relative to the base coordinate system of the robot arm, ​​​A translation vector of the antenna coordinate system relative to the base coordinate system of the robot arm.

[0023] The 5G antenna pose estimation method based on machine vision has the characteristics that the repositioning error in the S5 is obtained by the following steps:

[0024] A: the actual distance between and is calculated by formula (5) : :

[0025] (5)

[0026] B: the repositioning distance between and is calculated by formula (6) according to the cosine theorem : :

[0027] (6)

[0028] C: the repositioning error of formula (7) is obtained according to the actual distance , the repositioning distance :

[0029] (7)。

[0030] The rotation matrix in S8 is calculated by formula (8) :

[0031] (8)

[0032] In S8, a fixed point on the 5G antenna is selected as the origin of the antenna coordinate system, and the coordinates of the origin in the base coordinate system of the robot arm are recorded as the translation vector .

[0033] The electronic device has the characteristics that it includes a memory for storing a computer program and a processor for executing the computer program to realize the 5G antenna pose estimation method based on machine vision.

[0034] The computer readable storage medium has the characteristics that the computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the 5G antenna pose estimation method based on machine vision.

[0035] Compared with the prior art, the present application has the following advantages:

[0036] ​​​1. This invention combines a robotic arm with a triangulation method, using a robotic arm to carry a monocular camera to capture images from different perspectives, effectively replacing traditional binocular or multi-view camera systems, simplifying the system structure, reducing computational load, and lowering hardware costs.

[0037] 2. This invention is simple to implement and can calculate the pose of the 5G antenna under test while the robotic arm carries the probe to perform the task of detecting the radiation performance of the 5G antenna. The calculated pose result is optimized based on the pose information of the robotic arm, thereby improving the accuracy of pose estimation.

[0038] 3. This invention estimates the pose of the 5G antenna under test by analyzing camera images and the pose information of the robotic arm end effector, without relying on prior pose information, thus reducing the system's dependence on prior knowledge. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the overall system structure involved in the present invention;

[0040] Figure 2 This is a flowchart of the pose estimation method of the present invention;

[0041] Figure 3 This is a schematic diagram of the triangulation method in an embodiment of the present invention;

[0042] The following numbers are labeled in the diagram: 1. Monocular camera; 2. Robotic arm; 3. Robotic arm base; 4. Human-computer interaction unit; 5. Control unit; 6. 5G antenna under test; 7. Stage. Detailed Implementation

[0043] In this embodiment, as Figure 1 As shown, a 5G antenna pose estimation device based on machine vision includes: a monocular camera 1, a robotic arm 2, a robotic arm base 3, a human-machine interface unit 4, a control unit 5, a 5G antenna under test 6, and a stage 7. The monocular camera 1 is mounted at the end of the robotic arm 2 in a "eye-on-hand" configuration, with the optical axis of the monocular camera 1 aligned with the Z-axis of the end of the robotic arm 2. The robotic arm 2 carries the monocular camera 1 to an image acquisition position to perform an image acquisition task. The human-machine interface unit 4 monitors the system's operating status. The control unit 5 sends commands and processes data. The 5G antenna under test 6 is placed on the stage 7. In this embodiment, the robotic arm is a 6-DOF robotic arm, and the 5G antenna under test is a rectangular PCB-mounted antenna.

[0044] In this embodiment, as Figure 2 As shown, a 5G antenna pose estimation method based on machine vision includes the following steps:

[0045] S1: When the monocular camera reaches the i-th acquisition point At the i-th robot arm pose , the i-th antenna image is captured , the YOLOv8 model is used to detect the antenna in , to obtain the i-th ROI region antenna image , and the upper left corner coordinates of in are marked as ;

[0046] In this embodiment, before S1, the following preparations need to be completed, including:

[0047] (A) Train the antenna detection model, the process is: use the device to control the robot arm to carry the monocular camera to collect 60 5G antenna images under different angles, and use the labelme tool to label the antenna region of each image in the data set. Randomly select 48 antenna images in the data set as the training set, and the remaining 12 antenna images as the test set. Input the images of the training set and their corresponding labels into the YOLOv8 target detection algorithm for training. After training is completed, apply the model to the test set, and according to the test result, carry out parameter adjustment and optimization to obtain the 5G antenna detection model.

[0048] (B) Calibrate the monocular camera using the checkerboard calibration method to obtain the monocular camera intrinsic matrix as shown in formula (1) , the expression is:

[0049] (1)

[0050] In formula (1), and are the normalized focal lengths on the u-axis and v-axis of the camera image coordinate system respectively, is the focal length of the camera, and respectively represent the size of a unit pixel on the camera image coordinate system, with the unit being , and represent the optical center of the camera, that is, the intersection of the camera optical axis and the image plane.

[0051] (C) Calibrate the robot-camera system using the Tsai method to obtain the hand-eye matrix as shown in formula (2) :

[0052] (2)

[0053] wherein, , respectively represent the i-th When taking a single image, the rotation matrix and translation vector of the camera coordinate system relative to the end effector coordinate system of the robotic arm.

[0054] S2: Antenna images of the (i-1)th and i-th ROI regions and Feature extraction and feature matching are performed. Specifically, for and Grayscale conversion, Gaussian filtering, and binarization were performed. Antenna corner points and surface solder joints were selected as antenna feature points. The Shi-Tomas method was used to detect corner points in the ROI image, and the SIFT descriptor for each corner point was calculated. Then, the Brute-Force method was used to match the feature points, obtaining non-collinear 5G antenna feature points. exist and The corresponding pixel pairs above , ;in, This represents the j-th 5G antenna feature point; express exist coordinates on express exist The coordinates on the graph; m represents the total number of feature points, m≥3; and This represents the j-th pixel pair;

[0055] S3: Calculate the m pixel pairs respectively using equation (3) pixel coordinates , ,in, express exist coordinates on express exist Coordinates on:

[0056] (3)

[0057] In equation (3), express In the (i-1)th antenna image The coordinates of the top left corner;

[0058] S4: As Figure 3 As shown, based on the principle of triangulation, the j-th 5G antenna feature point is calculated using equation (4). Coordinates in the robot arm base coordinate system :

[0059] (4)

[0060] In formula (4), , are respectively a rotation matrix and a translation vector of the monocular camera coordinate system relative to the robot base coordinate system at the i-th time of photographing, and are obtained through coordinate transformation; , are respectively a rotation matrix and a translation vector of the monocular camera coordinate system relative to the robot base coordinate system at the i-1-th time of photographing, and are obtained through coordinate transformation; represents a depth value of the collection point to of the monocular camera at the i-th time of photographing; represents a depth value of the collection point to of the monocular camera at the i-1-th time of photographing; In this embodiment, the rotation matrix

[0061] and the translation vector of the camera coordinate system relative to the robot base coordinate system are calculated by using formula (5):

[0062] (5)

[0063] In formula (5), represents a transformation matrix of the camera coordinate system relative to the robot base coordinate system at the i-th time of photographing.

[0064] S5: iteratively optimize the objective function shown in formula (6) by using the LM algorithm, take the calculation result of S4 as the initial value of the LM algorithm, and solve the optimal coordinates of the j-th 5G antenna feature point in the robot base coordinate system;

[0065] (6)

[0066] In formula (6), represents a repositioning error between and , and is calculated by formula (7):

[0067] (7)

[0068] In this embodiment, the maximum number of iterations of the LM optimization algorithm is set to 500, and the step size is ​​​When the algorithm is executed to the maximum number of iterations or the step size is less than the given accuracy, it is considered that the optimization process reaches the optimal state at this time, the iteration is stopped, and the optimization result is taken as the optimal coordinate of the jth 5G antenna feature point in the base coordinate system of the mechanical arm.

[0069] S6: Fitting the optimal coordinate set by SVD decomposition method Get the normal vector of the antenna plane ; Where T represents transposition;

[0070] S7: The 5G antenna to be tested in this embodiment is a rectangular PCB antenna. When constructing the antenna coordinate system, the direction of the normal vector of the antenna plane is taken as the axis direction vector of the antenna coordinate system, denoted as ; The optimal coordinate of the upper left corner point on the antenna plane is selected , the optimal coordinate of the lower left corner point is , and the direction of the optimal coordinate is taken as the axis direction vector of the antenna coordinate system, denoted as ; The cross product of the axis direction vector and the axis direction vector is calculated to obtain the axis direction vector, denoted as

[0071] S8: Calculate the rotation matrix of the antenna coordinate system relative to the base coordinate system of the mechanical arm using formula (8) :

[0072] (8)

[0073] Select the optimal coordinate of the upper left corner point on the 5G antenna to be tested as the origin of the antenna coordinate system, and mark the coordinate of the origin in the base coordinate system of the mechanical arm as the translation vector of the antenna coordinate system relative to the base coordinate system of the mechanical arm . The transformation matrix of the antenna coordinate system relative to the base coordinate system of the mechanical arm is obtained by formula (9) , that is and The pose of the antenna in

[0074] (9)

[0075] In this embodiment, an electronic device includes a memory for storing a computer program for executing the above method, and a processor for executing the program stored in the memory.

[0076] In this embodiment, a computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above method.​

Claims

1. A 5G antenna pose estimation method based on machine vision, characterized in that, Includes the following steps: S1: When the monocular camera reaches the i-th acquisition point At that time, the monocular camera is used to determine the pose of the i-th robotic arm. The i-th antenna image is acquired. Using the YOLOv8 model to The antennas in the region are detected to obtain the antenna image of the i-th ROI region. and will exist The coordinates of the top left corner are: ; S2: Antenna images of the (i-1)th and i-th ROI regions and Feature extraction and feature matching were performed to obtain non-collinear 5G antenna feature points. exist and The corresponding pixel pairs above , ;in, This represents the j-th 5G antenna feature point; express exist coordinates on express exist The coordinates on the graph; m represents the total number of feature points, m≥3; and This represents the j-th pixel pair; S3: Calculate the m pixel pairs respectively using equation (1) pixel coordinates , ,in, express exist coordinates on express exist Coordinates on: (1) In equation (1), express In the (i-1)th antenna image The coordinates of the top left corner; S4: Based on the principle of triangulation, the j-th 5G antenna feature point is calculated using equation (2). Coordinates in the robot arm base coordinate system : (2) In equation (2), This is the intrinsic parameter matrix of a monocular camera, obtained through camera calibration; , These are the rotation matrix and translation vector of the camera coordinate system relative to the robot arm base coordinate system when the monocular camera takes the i-th picture, respectively, and are obtained through coordinate transformation; , These are the rotation matrix and translation vector of the camera coordinate system relative to the robot arm base coordinate system when the monocular camera takes the i-1th picture, respectively, and are obtained through the coordinate transformation shown in Equation (9); Indicates the monocular camera at the 1st The collection point at the time of the second photo. arrive The depth value; Indicates the monocular camera at the 1st The collection point at the time of the second photo. arrive The depth value; (9) In equation (9), Let represent the transformation matrix of the camera coordinate system relative to the robot arm base coordinate system at the i-th image acquisition point. Represents the hand-eye matrix; S5: The objective function shown in equation (3) is iteratively optimized using the LM algorithm. The calculation result of S4 is used as the initial value of the LM algorithm to solve for the j-th 5G antenna feature point. Optimal coordinates in the robot arm base coordinate system ; (3) In equation (3), express and Relocation error between; The relocation error It is obtained by following these steps: A: Calculate using equation (5) and The actual distance between : (5) B: According to the Law of Cosines, use equation (6) to calculate and Relocation distance between : (6) C: Based on actual distance Relocation distance The relocation error is obtained from equation (7). : (7) S6: Fitting the optimal coordinate set using the SVD decomposition method Obtain the normal vector of the antenna plane. Where T represents transpose; S7: The direction of the normal vector of the antenna plane is the antenna coordinate system. The axial direction vector is denoted as... ; Select a vector direction on the antenna plane as the antenna coordinate system The axial direction vector is denoted as... ; through calculation Sum of axial direction vectors The cross product of the axial direction vectors is obtained The axial direction vector is denoted as... ; S8: Use equation (4) to obtain the transformation matrix of the antenna coordinate system relative to the robot arm base coordinate system. That is and The pose of the mid-antenna: (4) In equation (4), Let be the rotation matrix of the antenna coordinate system relative to the robot arm base coordinate system, and calculate it using equation (8). Let the translation vector of the antenna coordinate system relative to the robot arm base coordinate system be: (8)。 2. The 5G antenna pose estimation method based on machine vision according to claim 1, characterized in that, In S8, a fixed point on the 5G antenna is selected as the origin of the antenna coordinate system, and the coordinates of the origin in the robot arm base coordinate system are used as the translation vector. .

3. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to implement the machine vision-based 5G antenna pose estimation method as described in claim 1 or 2 when executing the computer program.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the machine vision-based 5G antenna pose estimation method as described in claim 1 or 2.

Images