Visual guidance method and system for three-dimensional modeling robot, medium and product

Abstract

The invention discloses a three-dimensional modeling robot visual guidance method and system, a medium and a product, and relates to the field of industrial automation. By the adoption of the technical scheme, accurate measurement of the real-time position and the real-time posture of the workpiece is achieved, the visual guidance system automatically calculates the space position deviation value and the posture deviation value between the real-time position and the real-time posture of the workpiece and the target position and the target posture, and therefore the optimal adjustment path is generated, and the adjustment precision is improved. And the vector is converted into a visual space guiding vector to be displayed on a visual guiding interface of the robot. By means of the non-contact visual guiding mode, it is avoided that markers such as targets are installed on the surface of the workpiece, the assembly preparation workload is reduced, and meanwhile the assembly precision and efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of industrial automation, and in particular to a three-dimensional modeling robot vision guidance method, system, medium and product. Background Art

[0002] With the rapid development of industrial automation, the assembly accuracy of workpieces has a direct impact on product reliability. To ensure assembly accuracy, the spatial position and posture of the workpieces must be precisely guided during the assembly process, which places higher demands on assembly guidance technology.

[0003] Currently, the most common assembly guidance method used in industrial production primarily utilizes positioning and guidance technology based on laser trackers. This involves installing a reflective target on the workpiece surface and using the laser tracker to track the target's position, acquiring the workpiece's spatial coordinates. Based on this spatial coordinate information, a robot adjusts the workpiece's position until it reaches the desired installation location.

[0004] However, the laser tracker requires the installation of a reflective target on the surface of the workpiece, which increases the workload of assembly preparation, and the target may fall off or be damaged during use, increasing the complexity of operation. Summary of the Invention

[0005] The present application provides a three-dimensional modeling robot vision guidance method, system, medium and product for improving assembly accuracy and efficiency.

[0006] In the first aspect, the present application provides a three-dimensional modeling robot vision guidance method, which is applied to a vision guidance system, and the method includes: obtaining a first stereo image pair of a workpiece acquired by a binocular camera, and determining the initial parameters of the binocular camera according to a preset calibration plate; calibrating the internal and external parameters of the binocular camera based on the initial parameters to establish a spatial geometric relationship model of the binocular camera; performing distortion correction and epipolar line correction on the first stereo image pair according to the spatial geometric relationship model to obtain a corrected second stereo image pair; performing feature matching on the second stereo image pair and the three-dimensional model of the standard workpiece to extract a feature point set in the second stereo image pair and a standard feature point set in the three-dimensional model of the standard workpiece; performing three-dimensional space registration on the feature point set and the standard feature point set, and calculating a rigid body transformation matrix as a real-time posture measurement benchmark; and performing three-dimensional space registration on the feature point set and the standard feature point set to obtain a rigid body transformation matrix as a real-time posture measurement benchmark; and performing three-dimensional space registration on the feature point set and the standard feature point set to obtain a rigid body transformation matrix as a real-time posture measurement benchmark; and performing three-dimensional space registration on the feature point set and the standard feature point set, and calculating a rigid body transformation matrix as a real-time posture measurement benchmark; and performing three-dimensional space registration on the feature point set and the standard feature point set. The ambient temperature data of the set is used to determine the real-time position data of the workpiece after thermal compensation of the feature point set. The real-time position data of the workpiece is used to represent the real-time position and real-time posture of the workpiece during the assembly process; the real-time position data of the workpiece is compared with the preset workpiece position data, and the spatial position deviation value and posture deviation value of the workpiece are calculated. The preset workpiece position data is used to represent the target position and target posture of the workpiece during the assembly process; based on the spatial position deviation value and posture deviation value, an optimal adjustment path is generated, and the optimal adjustment path is converted into a spatial guidance vector containing a translation component and a rotation component. The translation component is used to indicate the linear movement direction and distance of the workpiece, and the rotation component is used to indicate the angular adjustment direction and amplitude of the workpiece. The optimal adjustment path is used to adjust the workpiece from the real-time position and real-time posture to the target position and target posture; the spatial guidance vector is displayed in real time on the visual guidance interface.

[0007] By employing this technical solution, the real-time position and attitude of the workpiece can be accurately measured. The vision guidance system automatically calculates the spatial position and attitude deviations between the workpiece's real-time position and attitude and the target position and attitude, generating the optimal adjustment path and converting them into intuitive spatial guidance vectors for display on the vision guidance interface. This contactless vision guidance method eliminates the need to install markers such as targets on the workpiece surface, reducing assembly preparation workload while improving assembly accuracy and efficiency.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the intrinsic and extrinsic parameters of the binocular camera are calibrated based on initial parameters to establish a spatial geometric relationship model of the binocular camera, specifically including: collecting a preset number of calibration plate image pairs, each calibration plate image pair including a left camera image and a right camera image; extracting corner point features in the left camera image and the right camera image to establish a correspondence between the world coordinate system and the image coordinate system; based on the initial parameters and the correspondence, iteratively optimizing the focal length, principal point coordinates, and lens distortion coefficient of the binocular camera to obtain the intrinsic parameters of the binocular camera; based on the intrinsic parameters and the correspondence, calculating the rotation matrix and translation vector between the left camera and the right camera to obtain the extrinsic parameters of the binocular camera; constructing a basic matrix and an essential matrix based on the intrinsic parameters and the extrinsic parameters, and establishing a spatial geometric relationship model of the binocular camera.

[0009] By employing this technical solution, the vision guidance system collects a preset number of calibration plate image pairs and extracts corner features, establishing a precise correspondence between the world coordinate system and the image coordinate system. The vision guidance system iteratively optimizes the focal length, principal point coordinates, and lens distortion coefficients of the binocular camera to obtain the binocular camera's intrinsic parameters. It then calculates the rotation matrix and translation vector between the left and right cameras to obtain the binocular camera's extrinsic parameters. Ultimately, it constructs the fundamental matrix and the intrinsic matrix to form a complete spatial geometric relationship model. This systematic camera calibration process ensures the accuracy of subsequent 3D reconstruction and position measurement, laying a solid foundation for the entire vision guidance system.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the feature point set and the standard feature point set are aligned in three-dimensional space, and a rigid body transformation matrix is ​​calculated as a real-time posture measurement benchmark, specifically including: calculating the first center point coordinates of the feature point set and the second center point coordinates of the standard feature point set; translating the feature point set so that the first center point coordinates coincide with the second center point coordinates to determine the translation parameters; calculating the rotation angle of the translated feature point set relative to the standard feature point set; and generating a rigid body transformation matrix based on the rotation angle and translation parameters as a real-time posture measurement benchmark.

[0011] By adopting the above technical solution, the vision guidance system utilizes a 3D spatial registration technique based on center point alignment. First, the vision guidance system calculates the center point coordinates of the feature point set and the standard feature point set, aligning them and determining the translation parameters. Then, the vision guidance system calculates the rotation angle of the registered feature point set relative to the standard feature point set. Finally, the vision guidance system generates a rigid body transformation matrix based on the rotation angle and translation parameters. This registration method quickly and accurately establishes the spatial transformation relationship between the current pose of the workpiece and the standard pose, providing a reliable benchmark for real-time pose measurement and effectively improving the measurement accuracy and real-time performance of the vision guidance system.

[0012] In combination with some embodiments of the first aspect, in some embodiments, before the step of determining the real-time position data of the workpiece after thermal compensation of the feature point set based on the spatial geometric relationship model, the real-time posture measurement benchmark and the real-time collected ambient temperature data, the method also includes: obtaining the standard feature point position of the workpiece at the standard working temperature and the historical feature point position under the historical ambient temperature data; determining the historical temperature difference based on the historical ambient temperature data and the standard working temperature; determining the historical feature point position difference based on the historical feature point position and the standard feature point position; using the historical temperature difference as input and the historical feature point position difference as output, training the preset model to obtain a thermal deformation prediction model, which is used to represent the impact of temperature changes on the shape and size of the workpiece.

[0013] By employing this technical solution, the vision guidance system quantifies the relationship between temperature changes and changes in the positions of feature points on the workpiece at standard operating temperature and at different historical temperatures. This relationship is then used as input and output to train a pre-set model to generate a thermal deformation prediction model. Because this model is trained based on a wealth of historical workpiece data, it is highly adaptable to specific workpieces and can accurately represent the impact of temperature on workpiece shape and size. This allows for early prediction of deformation based on real-time temperature, enabling early compensation intervention to reduce machining errors and improve precision.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the real-time position data of the workpiece after thermal compensation of the feature point set is determined based on the spatial geometric relationship model, the real-time posture measurement benchmark and the real-time collected ambient temperature data, specifically including: calculating the temperature difference between the ambient temperature data and the standard working temperature; inputting the temperature difference into the thermal deformation prediction model to obtain the thermal deformation displacement of the feature point set; superimposing the thermal deformation displacement on the original coordinates of the feature point set to obtain the thermally compensated feature point set; and converting the thermally compensated feature point set into the real-time position data of the workpiece based on the spatial geometric relationship model and the real-time posture measurement benchmark.

[0015] By adopting the above technical solution, the vision guidance system calculates the temperature difference between the ambient temperature data and the standard operating temperature, uses this temperature difference as the input of the thermal deformation prediction model, accurately obtains the thermal deformation displacement of the feature point set, and superimposes it on the original coordinates of the feature point set to achieve thermal compensation of the feature point set. Based on the spatial geometric relationship model and the real-time posture measurement benchmark, the vision guidance system successfully converts the thermally compensated feature point set into the real-time position data of the workpiece. This series of operations effectively considers the impact of ambient temperature changes on the position change of the workpiece, greatly improving the accuracy of the real-time position data of the workpiece, helping the robot to accurately grasp the state of the workpiece, reduce processing errors caused by thermal deformation, and improve product processing accuracy and production quality.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after comparing the real-time position data of the workpiece with the preset workpiece position data and calculating the spatial position deviation value and posture deviation value of the workpiece, the method also includes: when the spatial position deviation value is less than a preset first threshold and the posture deviation value is less than a preset second threshold, triggering a workpiece in place confirmation signal; and displaying the workpiece in place indication information on the visual guidance interface.

[0017] By adopting the above technical solution, the visual guidance system can accurately determine whether the workpiece is in place based on the spatial position deviation value and the posture deviation value, and trigger the in-place confirmation signal when the conditions are met. This can not only start the subsequent processes in time and improve production efficiency, but also intuitively present the in-place indication information on the visual guidance interface to help the robot know that the workpiece is in place.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the spatial guidance vector is displayed in real time on the visual guidance interface, specifically including: converting the translation component into displacement arrows in three orthogonal directions, the length of the displacement arrows being proportional to the displacement distance; converting the rotation component into rotation arrows in three orthogonal axes, the arc of the rotation arrows being proportional to the rotation angle; superimposing the displacement arrows and the rotation arrows in the three-dimensional display area of ​​the visual guidance interface; and updating the direction and size of the displacement arrows and the rotation arrows in real time.

[0019] By employing this technical solution, the visual guidance system visualizes the abstract spatial guidance vectors: the translational component is converted into displacement arrows in three orthogonal directions whose lengths are proportional to the displacement distance, and the rotational component is converted into rotation arrows whose arcs are proportional to the rotation angle. These are displayed as a superimposed image in the three-dimensional area of ​​the visual guidance interface, with their direction and magnitude updated in real time. This approach allows the robot to visually obtain information about the workpiece's translation and rotation in space, clearly grasping its real-time dynamics, greatly improving operational accuracy and efficiency, effectively reducing operational errors caused by misunderstandings about spatial vectors, and facilitating a smoother production process.

[0020] In a second aspect, an embodiment of the present application provides a visual guidance system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the visual guidance system to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when run on a visual guidance system, enables the visual guidance system to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on a visual guidance system, the visual guidance system executes the method described in the first aspect and any possible implementation of the first aspect.

[0023] It is understood that the visual guidance system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can be referenced to the beneficial effects of the corresponding methods and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By employing the above technical solution, accurate measurement of the workpiece's real-time position and attitude is achieved. The vision guidance system automatically calculates the spatial position and attitude deviations between the workpiece's real-time position and attitude and the target position and attitude, generating an optimal adjustment path. This is converted into intuitive spatial guidance vectors and displayed on the robot's vision guidance interface. This contactless vision guidance method eliminates the need to install markers such as targets on the workpiece surface, reducing assembly preparation workload while improving assembly accuracy and efficiency.

[0025] 2. By adopting the above technical solution, the vision guidance system uses a three-dimensional spatial registration technology based on center point alignment. First, the vision guidance system calculates the center point coordinates of the feature point set and the standard feature point set, aligning them and determining the translation parameters. Then, the vision guidance system calculates the rotation angle of the registered feature point set relative to the standard feature point set. Finally, the vision guidance system generates a rigid body transformation matrix based on the rotation angle and translation parameters. This registration method can quickly and accurately establish the spatial transformation relationship between the current pose of the workpiece and the standard pose, providing a reliable benchmark for real-time pose measurement and effectively improving the measurement accuracy and real-time performance of the vision guidance system.

[0026] 3. By adopting the above technical solution, the vision guidance system calculates the temperature difference between the ambient temperature data and the standard operating temperature, uses this temperature difference as the input of the thermal deformation prediction model, accurately obtains the thermal deformation displacement of the feature point set, and superimposes it on the original coordinates of the feature point set to achieve thermal compensation of the feature point set. Based on the spatial geometric relationship model and the real-time pose measurement benchmark, the vision guidance system successfully converts the thermally compensated feature point set into the real-time position data of the workpiece. This series of operations effectively considers the impact of ambient temperature changes on the position change of the workpiece, greatly improving the accuracy of the real-time position data of the workpiece, helping the robot to accurately grasp the state of the workpiece, reduce processing errors caused by thermal deformation, and improve product processing accuracy and production quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of the visual guidance method for a three-dimensional modeling robot in an embodiment of the present application; Figure 2 This is another flowchart of the visual guidance method for a three-dimensional modeling robot according to an embodiment of the present application; Figure 3 It is a schematic diagram of the physical device structure of the visual guidance system in the embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms used in the following examples of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application, the singular expressions "a", "an", "above", "the", and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations of one or more of the listed items.

[0029] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0030] The following describes the process of the method provided by this implementation in combination with the above scenarios. Figure 1 , which is a flow chart of the three-dimensional modeling robot vision guidance method in an embodiment of the present application.

[0031] S101, obtaining a first stereo image pair of a workpiece captured by a binocular camera, and determining initial parameters of the binocular camera according to a preset calibration plate; Among them, the binocular camera refers to a stereo vision system composed of two parallel industrial cameras, which is used to collect three-dimensional information of the workpiece; the workpiece refers to the parts or components that need to be assembled; the first stereo image pair refers to the left and right images simultaneously collected by the binocular camera; the preset calibration plate refers to a calibration tool with specific geometric features, usually a black and white checkerboard pattern; the initial parameters refer to the basic imaging parameters of the binocular camera, including focal length, principal point coordinates, baseline length, etc.

[0032] Specifically, the vision guidance system controls the binocular cameras to simultaneously capture images of the workpiece, obtaining two images, one left and one right, that encompass the complete view of the workpiece, forming the first stereo image pair. Simultaneously, a pre-set calibration plate is placed within the binocular cameras' field of view. The vision guidance system captures the plate's image and calculates the initial parameters of the binocular cameras by analyzing the image positions of the checkerboard corner points on the plate.

[0033] S102, calibrating the internal and external parameters of the binocular camera based on the initial parameters to establish a spatial geometric relationship model of the binocular camera; Among them, internal and external parameters include intrinsic parameters and extrinsic parameters; intrinsic parameters refer to the inherent imaging parameters of the binocular camera, including focal length, principal point coordinates, lens distortion coefficient, etc.; extrinsic parameters refer to the relative position relationship parameters between two industrial cameras, including rotation matrix and translation vector; spatial geometric relationship model is a mathematical model that describes the geometric characteristics of binocular camera imaging, including basic matrix and essential matrix.

[0034] Specifically, the vision guidance system controls the mechanical device to adjust the position and posture of the preset calibration plate, and uses the binocular camera to collect multiple sets of calibration plate image pairs in different positions and postures. First, the vision guidance system extracts the checkerboard corner features from each set of calibration plate image pairs and establishes the correspondence between the world coordinate system and the image coordinate system. Then, based on the initial parameters of the binocular camera, the vision guidance system uses a nonlinear optimization algorithm to iteratively calculate the intrinsic parameters of the binocular camera to obtain the precise focal length, principal point, and distortion coefficient. After that, the vision guidance system calculates the relative position relationship between the left and right cameras to obtain the extrinsic parameters of the binocular camera. Finally, the vision guidance system constructs the basic matrix and the essential matrix to form a complete spatial geometric relationship model.

[0035] Optionally, in general, calibrating the internal and external parameters of the binocular camera based on the initial parameters to establish a spatial geometric relationship model of the binocular camera can be achieved in the following ways, which are not limited here: collecting a preset number of calibration plate image pairs, each calibration plate image pair includes a left camera image and a right camera image; extracting corner point features in the left camera image and the right camera image to establish a correspondence between the world coordinate system and the image coordinate system; based on the initial parameters and the correspondence, iteratively optimizing the focal length, principal point coordinates and lens distortion coefficient of the binocular camera to obtain the intrinsic parameters of the binocular camera; based on the intrinsic parameters and the correspondence, calculating the rotation matrix and translation vector between the left camera and the right camera to obtain the extrinsic parameters of the binocular camera; constructing the basic matrix and the essential matrix according to the intrinsic parameters and the extrinsic parameters, and establishing the spatial geometric relationship model of the binocular camera.

[0036] Consider a binocular camera setup consisting of two MV-CA023-10GM industrial cameras with a baseline distance of 120 mm and a fixed-focus lens with a focal length of 8 mm. The default calibration pattern is a 12×9 black and white checkerboard with each grid measuring 15 mm × 15 mm.

[0037] (1) Example of intrinsic parameter calibration: By collecting 20 sets of calibration plate images with different poses and extracting corner points, we can calculate: A. Left camera internal parameters: Focal length: fx=1850.6 pixels, fy=1852.3 pixels; Principal point coordinates: cx = 1023.8 pixels, cy = 767.2 pixels; Radial distortion coefficient: k1=-0.1523, k2=0.0892; Tangential distortion coefficient: p1=0.0012, p2=-0.0008; B. Right camera internal parameters: Focal length: fx=1849.8 pixels, fy=1851.9 pixels; Principal point coordinates: cx = 1024.2 pixels, cy = 768.1 pixels; Radial distortion coefficient: k1=-0.1518, k2=0.0885; Tangential distortion coefficient: p1=0.0011, p2=-0.0009; (2) External parameter calibration example: The relative position relationship between the two industrial cameras is calculated: Rotation matrix R: [0.9999 0.0032 -0.0018] [-0.0031 0.9998 0.0025] [0.0019 -0.0024 0.9999] Translation vector T: [119.8526] [0.2851] [0.1523] This means that the right camera is translated about 120mm relative to the left camera in the X direction, and the offsets in the Y and Z directions are very small; (3) Example of spatial geometric relationship model: Based on the above parameters, we can calculate: Fundamental matrix F: [0.0000 -0.0083 0.2851] [0.0085 0.0000 -119.853] [-0.2848 119.852 0.0000] Essential matrix E: [0.0000 -0.1523 0.2851] [0.1518 0.0000 -119.853] [-0.2848 119.852 0.0000] (4) Calibration accuracy assessment: Mean reprojection error: 0.086 pixels; Calibration plate corner reconstruction error: 0.042mm; Baseline length measurement error: 0.147mm.

[0038] S103, performing distortion correction and epipolar correction on the first stereo image pair according to the spatial geometric relationship model to obtain a corrected second stereo image pair; Distortion correction refers to the process of eliminating the deformation of the image caused by lens distortion; epipolar correction refers to the process of adjusting the epipolar lines corresponding to the same-name points in the left and right images to be coplanar and parallel; the second stereo image pair refers to the corrected left and right image pair.

[0039] Specifically, the vision guidance system first corrects the distortion of the first stereo image pair based on the calibrated lens distortion parameters, eliminating nonlinear distortions such as barrel and pincushion distortions. Then, based on the fundamental and essential matrices, the vision guidance system calculates the image correction transformation matrix, projecting the left and right images onto parallel epipolar planes so that the epipolar lines of corresponding points are horizontally collinear. After correction, the same-name points in the second stereo image pair have the same vertical coordinates, greatly simplifying the subsequent feature matching process.

[0040] S104, performing feature matching on the second stereo image pair and the standard workpiece three-dimensional model to extract a feature point set in the second stereo image pair and a standard feature point set in the standard workpiece three-dimensional model; Among them, the standard workpiece three-dimensional model refers to the pre-established three-dimensional geometric model of the workpiece under ideal conditions; feature matching refers to the process of finding corresponding features in the second stereo image pair and the standard workpiece three-dimensional model through computer vision algorithms; the feature point set refers to the set of key point coordinates extracted from the actual workpiece image, such as edge points, corner points or feature salient points; the standard feature point set refers to the set of key point coordinates under ideal conditions extracted from the standard workpiece three-dimensional model.

[0041] Specifically, the vision guidance system first performs feature detection on the rectified second stereo image pair, extracting local feature descriptors such as SIFT and SURF. Next, the vision guidance system extracts feature descriptors of the same type from the 3D model of the standard workpiece. Next, the vision guidance system establishes a correspondence between image feature points and model feature points through similarity matching of feature descriptors. Finally, the vision guidance system uses algorithms such as RANSAC to eliminate incorrect matching points and obtain a reliable set of feature point correspondences. This entire feature matching process needs to account for factors such as projection transformation and scale changes.

[0042] S105, performing three-dimensional space registration on the feature point set and the standard feature point set, and calculating a rigid body transformation matrix as a real-time pose measurement benchmark; Among them, three-dimensional space registration refers to the process of achieving optimal overlap of two sets of three-dimensional point sets through rotation and translation; the rigid body transformation matrix refers to the 4×4 matrix that describes the rotation and translation transformation in three-dimensional space; the real-time posture measurement benchmark is used to represent the transformation relationship between the current position and current posture of the workpiece relative to the standard state.

[0043] Specifically, the vision guidance system first calculates the centroid coordinates of the two feature point sets. It then translates the feature point sets so that their centroids coincide with the centroid of the standard feature point set, determining the translation vector. Next, the vision guidance system uses SVD decomposition or quaternion methods to calculate the optimal rotation matrix, which is then combined with the translation vector to construct the complete rigid body transformation matrix. Finally, the vision guidance system refines the transformation parameters through iterative optimization to minimize the registration error.

[0044] Optionally, in general, the feature point set and the standard feature point set are aligned in three-dimensional space, and the rigid body transformation matrix is ​​calculated as the real-time posture measurement benchmark, which can be achieved in the following way, which is not limited here: calculate the first center point coordinates of the feature point set and the second center point coordinates of the standard feature point set; translate the feature point set so that the first center point coordinates coincide with the second center point coordinates to determine the translation parameters; calculate the rotation angle of the translated feature point set relative to the standard feature point set; generate a rigid body transformation matrix based on the rotation angle and translation parameters as the real-time posture measurement benchmark.

[0045] Assume that the pose of an engine cylinder head is measured and the following data has been obtained: Standard feature point set (10 feature points under ideal conditions, unit: mm): P1: (0.0, 0.0, 0.0); P2: (100.0, 0.0, 0.0); P3: (100.0, 100.0, 0.0); P4: (0.0, 100.0, 0.0); P5: (0.0, 0.0, 50.0); P6: (100.0, 0.0, 50.0); P7: (100.0, 100.0, 50.0); P8: (0.0, 100.0, 50.0); P9: (50.0, 50.0, 0.0); P10: (50.0, 50.0, 50.0); Feature point set (corresponding points extracted from the actual workpiece image, unit: mm): Q1: (10.5, -5.2, 2.1); Q2: (109.8, -2.8, 4.3); Q3: (107.6, 97.5, 2.9); Q4: (8.9, 94.6, 1.2); Q5: (12.3, -3.8, 51.6); Q6: (111.2, -1.5, 54.8); Q7: (109.1, 98.8, 53.2); Q8: (10.7, 95.9, 50.9); Q9: (59.4, 46.2, 3.5); Q10: (60.8, 47.1, 52.7); (1) Registration process calculation: The centroid of the standard feature point set: Pc = (50.0, 50.0, 25.0); The centroid of the feature point set: Qc=(60.0, 46.7, 27.7); Calculate the translation vector T (from Qc to Pc): T = [-10.0, 3.3, -2.7]; (2) Calculate the rotation matrix R after decentering (using the SVD method): R=[0.9986 -0.0523 0.0087] [0.0524 0.9986 -0.0035] [-0.0082 0.0044 0.9999] (3) Construct a 4×4 rigid body transformation matrix M: M=[0.9986 -0.0523 0.0087 -10.0] [0.0524 0.9986 -0.0035 3.3] [-0.0082 0.0044 0.9999 -2.7] [0.0000 0.0000 0.0000 1.0] (4) Registration accuracy evaluation: Point pair average registration error: 0.183mm; Maximum registration error: 0.246mm (occurs at the P7-Q7 point pair); Rotation angle decomposition: Around the X axis: -0.25°; Around the Y axis: 0.47°; Around Z axis: 3.00°; (5) Actual application effect: Position deviation compensation: X direction: -10.0mm; Y direction: 3.3mm; Z direction: -2.7mm; Posture deviation compensation: The main deviation is in the Z-axis rotation (about 3 degrees); The X and Y axis rotation deviations are small (less than 0.5 degrees); (6) Improvements after iterative optimization: The average registration error was reduced to 0.156 mm; The maximum registration error is reduced to 0.212mm.

[0046] S106, determining the real-time position data of the workpiece after thermal compensation of the feature point set based on the spatial geometric relationship model, the real-time position and posture measurement benchmark, and the real-time collected ambient temperature data, where the real-time position data of the workpiece is used to represent the real-time position and real-time posture of the workpiece during the assembly process; Among them, the spatial geometric relationship model refers to the mathematical model that describes the geometric characteristics of binocular camera imaging; the ambient temperature data refers to the real-time temperature measurement value of the current working environment; thermal compensation is used to represent the correction process of the workpiece deformation caused by temperature changes; the workpiece real-time position data refers to the current spatial position and posture information of the workpiece after thermal compensation.

[0047] Specifically, the vision guidance system first obtains the current ambient temperature and calculates the temperature difference from the standard operating temperature. Then, based on a preset deformation table (developed based on historical experience) and the temperature difference, the vision guidance system determines the compensation coefficient for the feature point set at the current ambient temperature to determine the deformation. The vision guidance system then superimposes the deformation on the original feature point coordinates and transforms the thermally compensated feature point set into the world coordinate system using a spatial geometric relationship model. Finally, the vision guidance system combines this with a real-time pose measurement benchmark to calculate the workpiece's real-time position and posture data.

[0048] Suppose a precision mechanical part is assembled: (1) The initial conditions are as follows: Standard operating temperature: 20°C; Material: aviation grade aluminum alloy; Linear expansion coefficient: 23.6×10⁻ 6 / °C; Workpiece basic size: 200mm×150mm×100mm; (2) On-site measurement data: Current ambient temperature: 26.5°C Original feature point coordinates (mm): P1: (0.0, 0.0, 0.0); P2: (200.0, 0.0, 0.0); P3: (200.0, 150.0, 0.0); P4: (0.0, 150.0, 0.0); P5: (0.0, 0.0, 100.0); P6: (200.0, 0.0, 100.0); (3) Thermal compensation calculation process: Temperature difference calculation: ΔT=26.5°C-20°C=6.5°C; Query the compensation coefficient according to the preset deformation table: X-direction compensation coefficient: 1.08 (force direction); Y direction compensation coefficient: 1.05; Z direction compensation coefficient: 1.03; Deformation calculation: Actual expansion in the X direction = 200 mm × 23.6 × 10⁻ 6 ×6.5×1.08=0.033mm; Actual expansion in the Y direction = 150mm × 23.6 × 10⁻ 6 ×6.5×1.05=0.024mm; Actual expansion in the Z direction = 100 mm × 23.6 × 10⁻ 6 ×6.5×1.03=0.016mm; (4) Feature point coordinates after thermal compensation (mm): P1': (0.000, 0.000, 0.000); P2': (200.033, 0.000, 0.000); P3': (200.033, 150.024, 0.000); P4': (0.000, 150.024, 0.000); P5': (0.000, 0.000, 100.016); P6': (200.033, 0.000, 100.016); (5) Use the spatial geometric relationship model to transform to the world coordinate system: Binocular camera parameters: Baseline distance: 120mm; Focal length: 8mm; Principal point offset: (cx=1024, cy=768); Coordinate transformation matrix: R=[0.9998 -0.0175 0.0087] [0.0176 0.9998 -0.0035] [-0.0085 0.0039 0.9999] T=[250.0, 180.0, 500.0] (6) Calculate the final position based on the real-time pose measurement benchmark: Pose measurement reference matrix: M=[0.9986 -0.0523 0.0087 10.5] [0.0524 0.9986 -0.0035 3.2] [-0.0082 0.0044 0.9999 -2.1] [0.0000 0.0000 0.0000 1.0] Final workpiece real-time position data: Spatial position (mm): X=260.533; Y=183.224; Z=497.916; Attitude angle (degrees): Roll=0.251; Pitch=-0.495; Yaw=3.012.

[0049] S107, comparing the real-time workpiece position data with preset workpiece position data to calculate the spatial position deviation value and posture deviation value of the workpiece, the preset workpiece position data being used to represent the target position and target posture of the workpiece during the assembly process; Among them, the preset workpiece position data refers to the ideal three-dimensional coordinate position and posture angle of the workpiece in the assembled state; the spatial position deviation value is used to represent the linear distance difference in the xyz directions between the real-time position of the workpiece and the target position; the posture deviation value is used to represent the angular difference around the xyz axes between the real-time posture of the workpiece and the target posture.

[0050] Specifically, the vision guidance system first reads the preset workpiece position data for the corresponding workpiece model from a pre-stored assembly process database. The vision guidance system then subtracts the xyz coordinate values ​​in the workpiece's real-time position data from the corresponding xyz coordinate values ​​in the preset workpiece position data to obtain spatial position deviations in the three directions. Simultaneously, the vision guidance system subtracts the pose angle values ​​about the xyz axes in the workpiece's real-time position data from the corresponding pose angle values ​​in the preset workpiece position data to obtain pose deviations in the three axes.

[0051] S108. Generate an optimal adjustment path based on the spatial position deviation value and the posture deviation value, and convert the optimal adjustment path into a spatial guidance vector including a translation component and a rotation component, wherein the translation component is used to indicate the linear movement direction and distance of the workpiece, and the rotation component is used to indicate the angular adjustment direction and amplitude of the workpiece. The optimal adjustment path is used to adjust the workpiece from the real-time position and real-time posture to the target position and target posture; Among them, the optimal adjustment path refers to the optimal motion trajectory for adjusting the workpiece from its real-time position and real-time posture to its target position and target posture; the translation component refers to the linear displacement along the xyz directions; the rotation component refers to the rotation angle around the xyz axes; and the spatial guidance vector is used to indicate the direction and distance the workpiece needs to move at each moment.

[0052] Specifically, the vision guidance system first constructs a six-degree-of-freedom state space based on spatial position and attitude deviation values. It then uses the A* algorithm to search for the optimal path from the current state to the target state, while taking into account constraints such as obstacle avoidance and stability. The vision guidance system then discretizes the optimal path into several key path points, calculating the translation vectors and rotation quaternions between adjacent path points. Finally, the vision guidance system converts these into intuitive three-dimensional guidance vectors, including displacement arrows in the x, y, and z directions and rotation arrows around the x, y, and z axes.

[0053] S109: Displaying the spatial guidance vector in real time on the visual guidance interface.

[0054] Among them, the visual guidance interface refers to the human-computer interaction interface used to display workpiece assembly guidance information; real-time display refers to updating the display content at a frequency of not less than 30Hz; the display methods of spatial guidance vectors include two visual forms: displacement arrows and rotation arrows.

[0055] Specifically, the vision guidance system renders a 3D scene onto a display interface, overlaying the scene with straight arrows representing translation and curved arrows representing rotation. The length of the displacement arrow is proportional to the displacement distance and points to the target position; the arc of the rotation arrow is proportional to the rotation angle and points in the direction of rotation. The vision guidance system also distinguishes different types of arrows by color and displays specific numerical values ​​next to the arrows. As the workpiece position changes, the vision guidance system updates the direction and size of these guidance arrows in real time, providing intuitive assembly guidance for the robot.

[0056] Optionally, in general, the real-time display of the spatial guidance vector on the visual guidance interface can be achieved in the following ways, which are not limited here: converting the translation component into displacement arrows in three orthogonal directions, and the length of the displacement arrow is proportional to the displacement distance; converting the rotation component into rotation arrows in three orthogonal axes, and the arc of the rotation arrow is proportional to the rotation angle; superimposing the displacement arrow and the rotation arrow in the three-dimensional display area of ​​the visual guidance interface; and updating the direction and size of the displacement arrow and the rotation arrow in real time.

[0057] By employing this technical solution, the real-time position and attitude of the workpiece can be accurately measured. The vision guidance system automatically calculates the spatial position and attitude deviations between the workpiece's real-time position and attitude and the target position and attitude, generating the optimal adjustment path and converting them into intuitive spatial guidance vectors for display on the vision guidance interface. This contactless vision guidance method eliminates the need to install markers such as targets on the workpiece surface, reducing assembly preparation workload while improving assembly accuracy and efficiency.

[0058] The following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of the three-dimensional modeling robot vision guidance method in an embodiment of the present application.

[0059] After step S105, the following steps may or may not be performed, which is not limited here: S201, obtaining the standard feature point positions of the workpiece under the standard working temperature and the historical feature point positions under the historical ambient temperature data; Among them, the standard working temperature refers to the ideal ambient temperature when the workpiece undergoes normal assembly operations, usually 20°C or 25°C; the standard feature point position refers to the three-dimensional coordinate value of the feature point of the workpiece at the standard working temperature, which is used as a reference position; the historical ambient temperature data refers to the actual ambient temperature data set recorded in the past production process; the historical feature point position refers to the actual position data set of the feature points measured at different historical ambient temperatures on the workpiece; the feature point refers to the key geometric feature point on the surface of the workpiece that is easy to identify and locate.

[0060] Specifically, the vision guidance system retrieves pre-stored data on the positions of standard feature points on the workpiece at standard operating temperatures from a process database. Simultaneously, the system extracts ambient temperature records and corresponding actual feature point positions from a historical database for this model of workpiece during past production. This historical data, which includes the actual positional variations of feature points at different ambient temperatures (e.g., within a range of 15°C to 35°C), provides the foundation for the subsequent development of a thermal deformation prediction model.

[0061] S202, determining a historical temperature difference based on historical ambient temperature data and a standard operating temperature; The historical temperature difference refers to the set of temperature difference values ​​between the historical ambient temperature data and the standard operating temperature; the unit of the historical temperature difference is degrees Celsius (℃).

[0062] Specifically, the vision guidance system subtracts each set of historical ambient temperature data from the standard operating temperature to generate a series of historical temperature difference values. For example, if the historical ambient temperature is 26.5°C and the standard operating temperature is 20°C, the calculated historical temperature difference is 6.5°C. The vision guidance system performs this difference calculation on all historical ambient temperature data to form a complete historical temperature difference data set, which is used to analyze the corresponding relationship between temperature changes and workpiece deformation.

[0063] S203, determining the historical feature point position difference based on the historical feature point position and the standard feature point position; Among them, the historical feature point position difference refers to the set of coordinate difference values ​​between the historical feature point position and the standard feature point position; the historical feature point position difference includes the linear displacement components in the X, Y, and Z directions; the measurement accuracy of the feature point position is usually better than 0.01mm; the standard feature point position is used as the reference coordinate.

[0064] Specifically, the vision guidance system performs vector subtraction on the coordinates of each feature point in each set of historical feature point position data from the corresponding feature point coordinates in the standard feature point position. For each feature point, the displacement difference in the X, Y, and Z directions is calculated. The vision guidance system performs this difference calculation on all feature points and all historical feature point positions, establishing a complete historical feature point position difference dataset, which provides output label data for subsequent training of the thermal deformation prediction model.

[0065] S204: Using historical temperature differences as input and historical feature point position differences as output, a preset model is trained to obtain a thermal deformation prediction model, which is used to represent the effect of temperature changes on the shape and size of the workpiece; Specifically, the visual guidance system first constructs an LSTM-based recurrent neural network, consisting of an input layer, two LSTM hidden layers, a fully connected layer, and an output layer. The input layer inputs historical temperature differences, with the number of hidden layer nodes set to 64 and the number of fully connected layer nodes set to 32. The output layer outputs the position differences of historical feature points.

[0066] The vision guidance system then uses the Adam optimizer with a learning rate of 0.001 and a training batch size of 32. These settings can be customized and are not set here. 80% of the historical data is divided into a training set and 20% into a validation set. Training is performed for 100 epochs, and the model with the highest validation set accuracy is retained. This can also be customized and is not set here. An epoch is the process by which the training dataset passes through the neural network once. In machine learning and deep learning, an epoch is a unit used to measure the number of times the entire training set is repeatedly learned. Specifically, an epoch is completed when the neural network completes a forward computation and backward propagation process, meaning that all data has been processed once. The vision guidance system uses binary cross entropy as the loss function and employs early stopping to prevent overfitting. When the loss function exceeds a preset threshold, model training is considered complete, resulting in a thermal deformation prediction model. Early stopping is a technique used in deep learning and machine learning to prevent overfitting by monitoring the model's performance on the validation set to determine when to stop training.

[0067] Finally, the vision-guided system feeds the input features from the validation set into the thermal deformation prediction model. The model then generates its predicted output, which is then compared with the actual output features from the validation set. Performance metrics such as accuracy, precision, recall, F1 score, and mean squared error (MSE) are used to evaluate the model's performance. Based on the model's performance on the validation set, the model's parameters are adjusted, including the learning rate, model complexity (such as increasing or decreasing the number of neural network layers or nodes), and regularization strength. This process may require multiple iterations, each based on the previous learning results, to optimize the model.

[0068] S205, calculating the temperature difference between the ambient temperature data and the standard operating temperature; Specifically, the vision guidance system performs a subtraction operation on the acquired ambient temperature data and the standard operating temperature to obtain a temperature difference between the ambient temperature data and the standard operating temperature.

[0069] S206, inputting the temperature difference into a thermal deformation prediction model to obtain the thermal deformation displacement of the feature point set; Specifically, the vision guidance system inputs the temperature difference into the thermal deformation prediction model, and the thermal deformation prediction model outputs the thermal deformation displacement of the feature point set under the ambient temperature data.

[0070] S207, superimposing the thermal deformation displacement onto the original coordinates of the feature point set to obtain a thermally compensated feature point set; Specifically, the process of the vision guidance system superimposing the thermal deformation displacement onto the original coordinates of the feature point set to obtain the thermally compensated feature point set can be referred to the example in step S106 and will not be repeated here.

[0071] S208, converting the thermally compensated feature point set into real-time position data of the workpiece according to the spatial geometric relationship model and the real-time pose measurement benchmark; Specifically, the process of the vision guidance system converting the feature point set after thermal compensation into the real-time position data of the workpiece can be referred to the example in step S106, and will not be repeated here.

[0072] S209, comparing the real-time workpiece position data with preset workpiece position data to calculate the spatial position deviation value and posture deviation value of the workpiece, where the preset workpiece position data is used to represent the target position and target posture of the workpiece during the assembly process; For details, please refer to step S107, which will not be described in detail here.

[0073] S210, when the spatial position deviation value is less than a preset first threshold value and the posture deviation value is less than a preset second threshold value, triggering a workpiece in place confirmation signal; Among them, the preset first threshold refers to the maximum allowable spatial position deviation, which is usually set to a value within the range of 0.1mm-1mm; the preset second threshold refers to the maximum allowable posture deviation, which is usually set to a value within the range of 0.1°-1°; the spatial position deviation value refers to the distance difference in the xyz directions between the real-time position of the workpiece and the target position; the posture deviation value refers to the angular difference around the xyz axes between the real-time posture of the workpiece and the target posture; the workpiece in place confirmation signal refers to a digital signal indicating that the workpiece has reached the expected assembly position.

[0074] Specifically, the vision guidance system monitors the workpiece's spatial position deviation and posture deviation in real time. When the spatial position deviation in all three axes (x, y, and z) is less than a preset first threshold (e.g., 0.5 mm), and the posture deviation around all three axes is less than a preset second threshold (e.g., 0.5°), the vision guidance system determines that the workpiece has reached the intended assembly position. At this point, the vision guidance system transmits a workpiece arrival confirmation signal to the host computer or PLC controller via the industrial fieldbus, triggering the start of subsequent assembly steps.

[0075] S211, displaying workpiece arrival indication information on the visual guidance interface; Among them, the visual guidance interface refers to the human-computer interaction interface used to display workpiece assembly guidance information; the workpiece in place indication information refers to the prompt information indicating that the workpiece has reached the expected assembly position through text, icons, colors, etc.; display refers to the presentation of information in a graphical manner on the visual guidance interface.

[0076] Specifically, when the workpiece in place confirmation signal is triggered, the vision guidance system displays clear workpiece in place indication information on the vision guidance interface, including: changing the display color of the preset workpiece three-dimensional model from yellow (under adjustment) to green (in place), displaying the text prompt "Workpiece in place" in a prominent position on the interface, playing prompt sound effects, flashing indicator lights, etc.

[0077] S212: Based on the spatial position deviation value and the posture deviation value, an optimal adjustment path is generated, and the optimal adjustment path is converted into a spatial guidance vector including a translation component and a rotation component, wherein the translation component is used to indicate the linear movement direction and distance of the workpiece, and the rotation component is used to indicate the angular adjustment direction and amplitude of the workpiece. The optimal adjustment path is used to adjust the workpiece from the real-time position and real-time posture to the target position and target posture; For details, please refer to step S108, which will not be described again here.

[0078] S213. Display the spatial guidance vector in real time on the visual guidance interface.

[0079] For details, please refer to step S109, which will not be described again here.

[0080] The following describes the visual guidance system in the embodiment of the present invention from the perspective of hardware processing. Figure 3 , is a schematic diagram of the physical device structure of the visual guidance system in an embodiment of the present application.

[0081] It should be noted that Figure 3 The structure of the visual guidance system shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0082] like Figure 3 As shown, the vision guidance system includes a CPU 301, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 302 or programs loaded from a storage unit 308 into a random access memory (RAM) 303, such as the methods described in the above embodiments. RAM 303 also stores various programs and data required for system operation. CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An I / O interface 305 is also connected to bus 304.

[0083] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, push button switches, and the like; an output section 307 including a liquid crystal display (LCD), an audio output device, indicator lights, and the like; a storage section 308 including a hard disk and the like; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. Removable media 311, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 310 as needed, so that computer programs read from the removable media can be installed in the storage section 308 as needed.

[0084] In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 309 and / or installed from the removable medium 311. When the computer program is executed by the CPU 301, the various functions defined in the present invention are performed.

[0085] It should be noted that specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings.

[0087] Specifically, the visual guidance system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the three-dimensional modeling robot visual guidance method provided by the above embodiment is implemented.

[0088] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the vision guidance system described in the above embodiments, or may exist independently and not be incorporated into the vision guidance system. The storage medium carries one or more computer programs, which, when executed by a processor of the vision guidance system, enable the vision guidance system to implement the vision guidance method for 3D modeling robots provided in the above embodiments.

[0089] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0090] As used in the above embodiments, the term “when” may be interpreted to mean “if” or “after” or “in response to determining that” or “in response to detecting that”, depending on the context. Similarly, the phrases “upon determining that” or “if (stated condition or event) is detected” may be interpreted to mean “if determining that” or “in response to determining that” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0091] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A three-dimensional modeling robot vision guidance method, characterized in that: Applied to a vision guidance system, the method comprises: Acquire a first stereo image pair of the workpiece captured by a binocular camera, and determine initial parameters of the binocular camera according to a preset calibration plate; Calibrate the internal and external parameters of the binocular camera based on the initial parameters to establish a spatial geometric relationship model of the binocular camera; performing distortion correction and epipolar correction on the first stereo image pair according to the spatial geometric relationship model to obtain a corrected second stereo image pair; Performing feature matching on the second stereo image pair and the standard workpiece three-dimensional model to extract a feature point set in the second stereo image pair and a standard feature point set in the standard workpiece three-dimensional model; Performing three-dimensional spatial registration on the feature point set and the standard feature point set, and calculating a rigid body transformation matrix as a real-time pose measurement benchmark; Determining, based on the spatial geometric relationship model, the real-time pose measurement benchmark, and the real-time collected ambient temperature data, real-time position data of the workpiece after thermal compensation of the feature point set, wherein the real-time position data of the workpiece is used to represent the real-time position and real-time pose of the workpiece during the assembly process; Comparing the real-time position data of the workpiece with preset workpiece position data to calculate the spatial position deviation value and posture deviation value of the workpiece, wherein the preset workpiece position data is used to represent the target position and target posture of the workpiece during the assembly process; generating an optimal adjustment path based on the spatial position deviation value and the posture deviation value, and converting the optimal adjustment path into a spatial guidance vector including a translation component and a rotation component, wherein the translation component is used to indicate a linear movement direction and distance of the workpiece, and the rotation component is used to indicate an angular adjustment direction and amplitude of the workpiece, and the optimal adjustment path is used to adjust the workpiece from the real-time position and the real-time posture to the target position and the target posture; The spatial guidance vector is displayed in real time on a visual guidance interface.

2. The method according to claim 1, characterized in that The step of calibrating the internal and external parameters of the binocular camera based on the initial parameters to establish a spatial geometric relationship model of the binocular camera specifically includes: Collecting a preset number of calibration plate image pairs, each calibration plate image pair including a left camera image and a right camera image; Extracting corner features from the left camera image and the right camera image to establish a correspondence between a world coordinate system and an image coordinate system; Iteratively optimizing the focal length, principal point coordinates, and lens distortion coefficient of the binocular camera based on the initial parameters and the corresponding relationship to obtain intrinsic parameters of the binocular camera; Based on the intrinsic parameters and the corresponding relationship, a rotation matrix and a translation vector between the left camera and the right camera are calculated to obtain extrinsic parameters of the binocular camera; According to the intrinsic parameters and the extrinsic parameters, a basic matrix and an essential matrix are constructed, and a spatial geometric relationship model of the binocular camera is established.

3. The method according to claim 1, characterized in that The step of performing three-dimensional spatial registration on the feature point set and the standard feature point set to calculate a rigid body transformation matrix as a real-time pose measurement benchmark specifically includes: Calculating the coordinates of a first center point of the feature point set and the coordinates of a second center point of the standard feature point set; translating the feature point set so that the coordinates of the first center point coincide with the coordinates of the second center point, to determine a translation parameter; Calculating the rotation angle of the translated feature point set relative to the standard feature point set; The rigid body transformation matrix is ​​generated according to the rotation angle and the translation parameter as the real-time posture measurement reference.

4. The method according to claim 1, wherein Before the step of determining the real-time position data of the workpiece after thermal compensation of the feature point set based on the spatial geometric relationship model, the real-time pose measurement benchmark and the real-time collected ambient temperature data, the method further includes: Obtaining standard feature point positions of the workpiece at a standard operating temperature and historical feature point positions under historical ambient temperature data; Determining a historical temperature difference based on the historical ambient temperature data and the standard operating temperature; Determining a historical feature point position difference based on the historical feature point position and the standard feature point position; The historical temperature difference is used as input and the historical feature point position difference is used as output to train the preset model to obtain a thermal deformation prediction model, which is used to represent the impact of temperature changes on the shape and size of the workpiece.

5. The method according to claim 4, characterized in that Determining the real-time position data of the workpiece after thermal compensation of the feature point set based on the spatial geometric relationship model, the real-time pose measurement benchmark, and the real-time collected ambient temperature data specifically includes: Calculating the temperature difference between the ambient temperature data and the standard operating temperature; Inputting the temperature difference into the thermal deformation prediction model to obtain the thermal deformation displacement of the feature point set; Superimposing the thermal deformation displacement onto the original coordinates of the feature point set to obtain a thermally compensated feature point set; The thermally compensated feature point set is converted into the real-time position data of the workpiece according to the spatial geometric relationship model and the real-time pose measurement benchmark.

6. The method according to claim 1, characterized in that After the step of comparing the real-time workpiece position data with the preset workpiece position data and calculating the spatial position deviation value and the posture deviation value of the workpiece, the method further includes: When the spatial position deviation value is less than a preset first threshold value and the posture deviation value is less than a preset second threshold value, a workpiece in place confirmation signal is triggered; The workpiece position indication information is displayed on the visual guidance interface.

7. The method according to claim 1, characterized in that The real-time display of the spatial guidance vector on the visual guidance interface specifically includes: Converting the translation component into displacement arrows in three orthogonal directions, wherein the lengths of the displacement arrows are proportional to the displacement distance; Convert the rotation component into three orthogonal axial rotation arrows, where the arc of the rotation arrows is proportional to the rotation angle; Overlaying and displaying the displacement arrow and the rotation arrow in a three-dimensional display area of ​​the visual guidance interface; The directions and sizes of the displacement arrows and the rotation arrows are updated in real time.

8. A visual guidance system, characterized in that: The visual guidance system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the visual guidance system to execute the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a vision guidance system, the vision guidance system is caused to perform the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a vision guidance system, the vision guidance system is caused to perform the method according to any one of claims 1 to 7.

Images