Visual alignment method for vehicle-mounted screen assembly
Through the multi-camera alignment method, the high-precision alignment of vehicle-mounted screen assembly is achieved using the coordination of industrial cameras and robotic arms, and the problem of insufficient alignment accuracy in the prior art is solved, and the accuracy and efficiency of assembly are improved.
Patent Information
- Application Number
- CN202510016914.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-06
- Publication Date
- 2025-02-28
AI Technical Summary
The prior art is difficult to meet the high requirements for vehicle-mounted alignment accuracy, and neither traditional single-vision scheme nor multi-view synthesis scheme can effectively solve the problem of high-precision assembly.
Using a multi-camera alignment method, multiple sets of industrial cameras are set up for calibration and image acquisition through the coordination of vehicles and robotic arms. The camera calibration algorithm is used to obtain the position relationship between cameras and the position relationship between robotic arms bases, so as to achieve alignment matching and alignment parameters between cameras, and then obtain the movement coordinates of the robotic arms through alignment calculations to achieve high-precision vehicle-mounted screen assembly.
It improves the accuracy and accuracy of vehicle-mounted screen assembly, improves assembly efficiency and quality, and meets the needs of high-precision assembly.
Smart Images
Figure CN119540361A_ABST
Abstract
Description
Technical Field
[0001] The present invention is applied to the technical field of vehicle-mounted screen assembly, and in particular relates to a visual alignment method for vehicle-mounted screen assembly. Background Art
[0002] At present, cars are generally equipped with car systems and large display screens to facilitate user interaction and information acquisition. In traditional screen industrial production, the assembly of parts and components of electronic components mostly relies on manual work. In this process, the skill level of workers plays a vital role, which directly determines the final assembly quality of the product. However, for some high-precision assembly, the problems of low efficiency and substandard precision of manual work are becoming more and more obvious, and can no longer meet the requirements of high precision. At present, it has basically turned to automated equipment for assembly. Conventional automated assembly industrial robots generally use teaching or offline programming to perform path planning and motion programming according to the tasks to be completed. This requires that the automated production line must be highly structured, resulting in a lack of flexibility and adaptability to the working environment. For the high-precision screen assembly process, the application of traditional teaching industrial robots is greatly limited. The single vision solution that is commonly used with mobile phone screens, as the size of car-mounted screens becomes larger and larger, has a large difference in size and process compared to mobile phone screens, so the single vision solution can no longer meet the needs of high-precision assembly.
[0003] There are currently two solutions to the above problem. One is to use an industrial camera with a wider viewing angle and the required accuracy, and the other is to use multi-view synthesis. The former will lead to a decrease in pixels and accuracy after the viewing angle becomes wider. In order to ensure high-precision operation requirements, it is necessary to purchase more expensive high-precision equipment. The latter uses multiple industrial cameras and needs to align the images of each camera. The existing technology cannot complete high-precision alignment, and it also cannot meet the assembly requirements of vehicle-mounted screens.
[0004] If a visual alignment method that meets the alignment accuracy requirements of vehicle-mounted screen assembly can be provided, the cost and accuracy issues in the vehicle-mounted screen assembly process can be well resolved. Summary of the invention
[0005] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a visual alignment method for assembling a vehicle-mounted screen with multiple cameras having high alignment accuracy.
[0006] The technical solution adopted by the present invention is: the present invention comprises the following steps: Step S1, fix the screen by means of a carrier, and move the back cover of the screen to the top of the screen by means of a mechanical arm, a calibration plate matching the screen is arranged on the carrier, and multiple groups of industrial cameras are arranged under the carrier, and the multiple groups of industrial cameras respectively acquire images of corresponding parts of the calibration plate, the screen and the back cover; Step S2, using a camera calibration algorithm to sequentially obtain the positional relationship between industrial cameras, the positional relationship between the industrial camera and the robotic arm base, and the positional relationship between the industrial camera and the center of the robotic arm rotation axis; Step S3, obtain the center point position of the current screen and the back cover through the industrial camera, and obtain the corresponding coordinate position of the current robotic arm, obtain the angle deviation of the screen and the angle deviation of the back cover through alignment calculation, and perform coordinate calculation after the back cover is rotated by an angle, and then obtain the position deviation between the back cover and the screen, obtain the final alignment coordinates of the robotic arm through the above parameters, and perform alignment assembly after completing the movement.
[0007] As can be seen from the above scheme, by setting the calibration plate to obtain the position relationship between industrial cameras, and the position of the industrial camera and the robot arm base, and the center of the robot arm rotation axis, the alignment matching between the cameras is realized, and the alignment parameters are obtained. After taking pictures of the screen and the back cover in step S3, the alignment calculation is performed according to the obtained alignment parameters, and the moving coordinates of the robot arm are obtained, so as to realize the visual alignment of the vehicle screen assembly based on multiple cameras, improve the accuracy and precision of the screen assembly, and improve the assembly efficiency and quality.
[0008] A preferred solution is that step S2 further includes the following specific steps: Step S2.1, a calibration plate is provided with a plurality of dots distributed in an array, and all industrial cameras acquire images of the calibration plate and record the pixel coordinates of the dots in the image; Step S2.2, obtaining the actual coordinates of the dots in all industrial cameras according to the actual distances of the dots on the calibration plate; Step S2.3, calculating the transformation matrix H from the pixel coordinates of each industrial camera to the actual coordinates; Step S2.4, the robot arm picks up the back cover and moves the center of the back cover to the vicinity of the image center of one of the industrial cameras, and records the image coordinates Nx of the back cover center on the industrial camera and the motion coordinates Px of the robot arm at this time; Step S2.5, make the center of the back cover appear in the middle, middle right, upper right, upper middle, upper left, middle left, lower left, lower middle, and lower right of the industrial camera in sequence to obtain the image coordinates Nx of the center of the back cover and the motion coordinates Px of the corresponding robot arm; Step S2.6, calculating the relationship between the industrial camera coordinate system and the robot arm base coordinate system according to the nine sets of data; Step S2.7, the robot arm then moves the back cover to the inspection, assembly and photography position, and rotates the rotation axis by an angle r while keeping the horizontal position of the robot arm unchanged; Step S2.8, after taking a photo with one of the industrial cameras, identify the features of the back cover and record the pixel coordinates Rx of the back cover on the industrial camera; Step S2.9: Repeat steps S2.8 and S2.9 to obtain multiple pixel coordinates Rx, and obtain the center of the circle where the Rx point is located (X r , Y r ), center (X r , Y r ) is the rotation center coordinate of the robot arm.
[0009] A preferred solution is that step S3 further includes the following specific steps: Step S3.1, extract two straight line features of the screen in one of the industrial camera images, and find the intersection of the two straight lines, which is the screen corner point coordinates in the industrial camera, marked as Dp1. Similarly, extract the screen corner point coordinates dp2, dp3, and dp4 in the remaining industrial cameras; Step S3.2, according to the positional relationship between the industrial cameras obtained in step S2, the coordinates of the remaining industrial cameras are converted into the coordinates in the first group of industrial cameras, that is, the coordinates Dp2, Dp3, Dp4 of the corner points dp2, dp3, dp4 in the coordinate system of the first group of industrial cameras are obtained; Step S3.3, find the intersection of the line connecting Dp1 and Dp3 and the line connecting Dp2 and Dp4 as the screen center point D(x1, y1), and the angle between the line connecting Dp1 and Dp3 and the positive direction of the X-axis as the screen angle Angle; Step S3.4: Similarly, according to steps S3.1 to S3.3, the center point d(x2, y2) and the angle angle of the back cover are obtained; Step S3.5: When taking photos, the coordinates of the robot are (x0, y0, r0); The angular deviation can be obtained: ; Coordinates of the back cover after rotation angle: ; Back cover and screen deviation: ; The coordinates (x, y, r) of the robot arm after final alignment satisfy . BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 It is a work flow chart of the present invention; Figure 2It is a schematic diagram of camera distribution of the present invention. DETAILED DESCRIPTION
[0011] like Figure 1 and Figure 2 As shown, in this embodiment, the present invention includes the following steps: Step S1, fix the screen by a carrier, and move the back cover of the screen to the top of the screen by a mechanical arm. A calibration plate matching the screen is arranged on the carrier, and four groups of industrial cameras are arranged under the carrier. The four groups of industrial cameras are distributed in a matrix and arranged in sequence along the circumferential direction. The four groups of industrial cameras respectively acquire images centered on the four corner points of the calibration plate, the screen and the back cover; Step S2, using a camera calibration algorithm to sequentially obtain the positional relationship between industrial cameras, the positional relationship between the industrial camera and the robotic arm base, and the positional relationship between the industrial camera and the center of the robotic arm rotation axis; The specific step S2 includes the following specific steps: Step S2.1, nine dots distributed in an array are arranged on the calibration board. The size of the calibration board is close to that of the screen and the material is glass. The spacing between the dots is 4 mm. Four groups of industrial cameras simultaneously acquire images of the calibration board and record the pixel coordinates of the dots in the image. The dot coordinates of the first group of industrial cameras are marked as Cam1Mark1, Cam1Mark2...Cam1Mark9, and the dot coordinates of the second group of industrial cameras are marked as Cam2Mark1, Cam2Mark2...Cam2Mark9. Similarly, the dot coordinates of the third and fourth groups of industrial cameras are obtained; Step S2.2, according to the actual distance record of the dots on the calibration plate, obtain the actual coordinates of the dots in all industrial cameras in the calibration plate coordinate system (the dots on the calibration plate are spaced 4 mm apart). The actual coordinates of the corresponding points of the first group of industrial cameras in the calibration plate coordinate system are marked as Cam1Pos1, Cam1Pos2...Cam1Pos9, and the actual coordinates of the corresponding points of the second group of industrial cameras in the calibration plate coordinate system are marked as Cam2Pos1, Cam2Pos2...Cam2Pos9. Similarly, the actual coordinates of the corresponding points of the third group of industrial cameras and the fourth group of industrial cameras in the calibration plate coordinate system are obtained; Step S2.3, calculate the transformation matrix H from the pixel coordinates of each industrial camera to the actual coordinates, denoted as H1, H2, H3, H4 respectively; Specifically, assuming that the camera coordinates Cam1Mark1 are recorded as (x1, y1), and the actual coordinates in the calibration plate coordinate system Cam1Pos1 (X1, Y1), then there is a relationship from the camera coordinates to the calibration plate coordinates: ; The transformation matrix T of a two-dimensional plane can be written as follows: is the parameter to be requested; ; Therefore, the following set of equations can be established: ; By solving the least squares method, we can obtain the parameters of the transformation matrix H. The transformation matrix H of the four groups of industrial cameras is obtained, which are denoted as H1, H2, H3, and H4 respectively; Step S2.4, the robot arm picks up the back cover and moves the center of the back cover to the vicinity of the image center of one of the industrial cameras, and records the image coordinates Nx of the back cover center on the industrial camera and the motion coordinates Px of the robot arm at this time; Step S2.5, make the center of the back cover appear in the middle, middle right, upper right, upper middle, upper left, middle left, lower left, lower middle, and lower right of the industrial camera in sequence to obtain the image coordinates Nx of the center of the back cover and the motion coordinates Px of the corresponding robot arm; Step S2.6, according to step S2.5, nine sets of data (image coordinates Nx of the center of the back cover and motion coordinates Px of the robot arm) are obtained, and the transformation matrix T between the industrial camera coordinate system and the robot arm base coordinate system is calculated by the calculation method of the transformation matrix H in step S2.3; Step S2.7, the robot arm then moves the back cover to the inspection, assembly and photography position, and rotates the rotation axis by an angle r while keeping the horizontal position of the robot arm unchanged; Step S2.8, identifying the features of the back cover after taking a photo with the first group of industrial cameras, and recording the pixel coordinates Rx of the back cover in the first group of industrial cameras; Step S2.9: Repeat steps S2.8 and S2.9 to obtain multiple pixel coordinates Rx, and obtain the center of the circle where the Rx point is located (X r , Y r ), center (X r , Y r ) is the rotation center coordinate of the robot arm; Step S3, obtain the center point position of the current screen and the back cover through the industrial camera, and obtain the corresponding coordinate position of the current robot arm, obtain the angle deviation of the screen and the angle deviation of the back cover through alignment calculation, and perform coordinate calculation after the back cover is rotated by an angle, thereby obtaining the position deviation between the back cover and the screen, and obtain the final alignment coordinates of the robot arm through the above parameters, and perform alignment assembly after completing the movement; Specifically, step S3 includes the following specific steps: Step S3.1, extract two straight line features of the screen in the first group of industrial camera images, and find the intersection of the two straight lines, which is the corner point coordinate of the screen in the first group of industrial cameras, marked as Dp1. Similarly, extract the corner point coordinates of the screen in the second, third and fourth groups of industrial cameras as dp2, dp3 and dp4; Step S3.2, according to the positional relationship between the industrial cameras obtained in step S2, the second, third and fourth groups of industrial camera coordinates are converted into coordinates in the first group of industrial cameras, that is, the transformation matrices H1, H2, H3, H4, the second group of industrial camera coordinates are converted to the first group of industrial camera matrix: H1^(-1) *H2; the third group of industrial camera coordinates are converted to the first group of industrial camera matrix: H1^(-1) *H3; the fourth group of industrial camera coordinates are converted to the first group of industrial camera matrix: H1^(-1) *H4; and then the corner point coordinates dp2, dp3, dp4 are obtained in the coordinate system of the first group of industrial cameras Dp2, Dp3, Dp4; Step S3.3, find the intersection of the line connecting Dp1 and Dp3 and the line connecting Dp2 and Dp4 as the screen center point D(x1, y1), and the angle between the line connecting Dp1 and Dp3 and the positive direction of the X-axis as the screen angle Angle; Step S3.4: Similarly, according to steps S3.1 to S3.3, the center point d(x2, y2) and the angle angle of the back cover are obtained; Step S3.5: When taking photos, the coordinates of the robot are (x0, y0, r0); The angular deviation can be obtained: ; Coordinates of the back cover after rotation angle: ; Back cover and screen deviation: ; The coordinates (x, y, r) of the robot arm after final alignment satisfy: .
[0012] Although the embodiments of the present invention are described with practical solutions, they do not constitute limitations on the meaning of the present invention. For those skilled in the art, it is obvious to modify the implementation scheme and combine it with other solutions based on this description.
Claims
1. A visual alignment method for vehicle screen assembly, characterized in that: It includes the following steps: Step S1, fix the screen by means of a carrier, and move the back cover of the screen to the top of the screen by means of a mechanical arm, a calibration plate matching the screen is arranged on the carrier, and multiple groups of industrial cameras are arranged under the carrier, and the multiple groups of industrial cameras respectively acquire images of corresponding parts of the calibration plate, the screen and the back cover; Step S2, using a camera calibration algorithm to sequentially obtain the positional relationship between industrial cameras, the positional relationship between the industrial camera and the robotic arm base, and the positional relationship between the industrial camera and the center of the robotic arm rotation axis; Step S3, obtain the center point position of the current screen and the back cover through the industrial camera, and obtain the corresponding coordinate position of the current robotic arm, obtain the angle deviation of the screen and the angle deviation of the back cover through alignment calculation, and perform coordinate calculation after the back cover is rotated by an angle, and then obtain the position deviation between the back cover and the screen, obtain the final alignment coordinates of the robotic arm through the above parameters, and perform alignment assembly after completing the movement.
2. The visual alignment method for assembling a vehicle-mounted screen according to claim 1, characterized in that: Step S2 also includes the following specific steps: Step S2.1, a calibration plate is provided with a plurality of dots distributed in an array, and all industrial cameras acquire images of the calibration plate and record the pixel coordinates of the dots in the image; Step S2.2, obtaining the actual coordinates of the dots in all industrial cameras according to the actual distances of the dots on the calibration plate; Step S2.3, calculating the transformation matrix H from the pixel coordinates of each industrial camera to the actual coordinates; Step S2.4, the robot arm picks up the back cover and moves the center of the back cover to the vicinity of the image center of one of the industrial cameras, and records the image coordinates Nx of the back cover center on the industrial camera and the motion coordinates Px of the robot arm at this time; Step S2.5, make the center of the back cover appear in the middle, middle right, upper right, upper middle, upper left, middle left, lower left, lower middle, and lower right of the industrial camera in sequence to obtain the image coordinates Nx of the center of the back cover and the motion coordinates Px of the corresponding robot arm; Step S2.6, calculating the relationship between the industrial camera coordinate system and the robot arm base coordinate system according to the nine sets of data; Step S2.7, the robot arm then moves the back cover to the inspection, assembly and photography position, and rotates the rotation axis by an angle r while keeping the horizontal position of the robot arm unchanged; Step S2.8, after taking a photo with one of the industrial cameras, identify the features of the back cover and record the pixel coordinates Rx of the back cover on the industrial camera; Step S2.9: Repeat steps S2.8 and S2.9 to obtain multiple pixel coordinates Rx, and obtain the center of the circle where the Rx point is located (X r , Y r ), center (X r , Y r ) is the rotation center coordinate of the robot arm.
3. The visual alignment method for assembling a vehicle-mounted screen according to claim 1, characterized in that: Step S3 also includes the following specific steps: Step S3.1, extract two straight line features of the screen in one of the industrial camera images, and find the intersection of the two straight lines, which is the screen corner point coordinates in the industrial camera, marked as Dp1. Similarly, extract the screen corner point coordinates dp2, dp3, and dp4 in the remaining industrial cameras; Step S3.2, according to the positional relationship between the industrial cameras obtained in step S2, the coordinates of the remaining industrial cameras are converted into the coordinates in the first group of industrial cameras, that is, the coordinates Dp2, Dp3, Dp4 of the corner points dp2, dp3, dp4 in the coordinate system of the first group of industrial cameras are obtained; Step S3.3, find the intersection of the line connecting Dp1 and Dp3 and the line connecting Dp2 and Dp4 as the screen center point D(x1, y1), and the angle between the line connecting Dp1 and Dp3 and the positive direction of the X-axis as the screen angle Angle; Step S3.4: Similarly, according to steps S3.1 to S3.3, the center point d(x2, y2) and the angle angle of the back cover are obtained; Step S3.5: When taking photos, the coordinates of the robot are (x0, y0, r0); The angular deviation can be obtained: ; Coordinates of the back cover after rotation angle: ； Back cover and screen deviation: ； The coordinates (x, y, r) of the robot arm after final alignment satisfy: 。