A three-dimensional space groove track generation method based on multi-view images

Abstract

The present application relates to the technical field of image data processing, and discloses a three-dimensional space groove track generation method based on multi-view images, comprising: acquiring a front view image extraction plane polar coordinate profile, and synchronously collecting a side view image sequence when a workpiece rotates at a constant speed; only extracting height data on a fixed observation line from each frame of the image sequence; generating an angle-height mapping diagram based on the height data and the time sequence thereof; and obtaining corresponding groove heights by querying the mapping diagram through a space polar angle in the plane polar coordinate profile and synthesizing a three-dimensional track, wherein the present application establishes the correspondence between the plane angle and the space height by using time sequence correlation through image data processing, avoids the dependence on complex space geometric calibration of the side view camera, makes the system have high tolerance to device position drift, and the calculation process is efficient.

Description

Technical Field

[0001] This invention relates to a method for generating three-dimensional spatial groove trajectories based on multi-view images, belonging to the field of image data processing technology. Background Technology

[0002] Currently, especially when precision machining or applying adhesive to workpieces with complex three-dimensional contours, automatically generating accurate work trajectories is a fundamental and crucial technical aspect. Utilizing machine vision systems to extract workpiece features from image data and generate paths is a recognized technical direction for improving production efficiency and consistency, replacing manual teaching. To acquire the three-dimensional information of the workpiece, a conventional approach in image processing is to construct a multi-view vision system and, through complex multi-view geometric calibration, pre-establish precise spatial coordinate transformation relationships between each camera, workpiece, and motion mechanism. Then, surface point cloud data is obtained through triangulation or 3D reconstruction. This method performs well in ideal experimental environments, but in real industrial production environments, its accuracy highly depends on the long-term stability of calibration parameters. Mechanical vibrations generated during equipment operation, thermal expansion and contraction due to environmental temperature changes, or daily replacement and wear of tooling fixtures and end tools can all cause the preset calibration parameters to gradually become invalid. Once the calibration relationship drifts, the system's three-dimensional data will deviate, forcing the production line to frequently stop for recalibration and maintenance. This contradicts the goal of high cycle time and high reliability pursued by automated production lines.

[0003] However, even at the software level of data processing, existing technologies have not escaped their dependence on such rigid physical calibrations and have fundamental flaws. For example, Chinese invention patent CN119941801A discloses a three-dimensional spatial trajectory recognition method and system based on multi-view stereo reconstruction. In terms of software flow, the core of this method is still to solve the coordinates by combining multi-view triangulation with camera intrinsic and extrinsic parameters. The validity of this algorithm depends entirely on the long-term stability of the intrinsic and extrinsic parameters obtained by calibrating the camera through camera calibration software and calibration board. This indicates that its data processing method itself is deeply bound to unstable physical calibration, and therefore it is also unable to cope with position drift in real industrial environments and has not solved the fundamental problem.

[0004] Therefore, the technical problem to be solved by this invention is to find a new technical approach at the image data processing level that does not rely on complex and easily failed global geometric calibration, but instead utilizes the information contained in the multi-view image data itself during the acquisition process to establish a reliable data correspondence and generate complex three-dimensional spatial trajectories. Summary of the Invention

[0005] This invention provides a method for generating three-dimensional spatial groove trajectories based on multi-view images. Its main purpose is to solve the problem of how to establish a reliable data correspondence by utilizing the information contained in the multi-view image data itself during the acquisition process, without relying on complex and easily failed global geometric calibration, thereby generating complex three-dimensional spatial trajectories.

[0006] To achieve the above objectives, this invention provides a method for generating three-dimensional spatial groove trajectories based on multi-view images. This method is applied to a system including a robotic arm, a front-view camera, and a side-view camera. The method includes controlling the robotic arm to grasp a product and move it to the working position of the front-view camera, acquiring a front-view image of the product through the front-view camera, processing the front-view image to extract a groove contour point set, and converting the groove contour point set into a set of polar radii with the center of the gripper as the origin. and spatial polar angle It also includes the following steps:

[0007] Step a: While the robotic arm rotates at a constant speed around the center of the gripper, a side-view image of the product is simultaneously acquired using a side-view camera to obtain an image time series. ,in For frame index number, The horizontal image coordinates are... Vertical image coordinates, frame index number With the rotation angle of the robotic arm There is a linear correspondence between them;

[0008] Step b, for image time series Each frame of the image Extract only one preset fixed vertical observation line from the image frame. The groove on the image is perpendicular to the coordinates. and will Convert to physical height ;

[0009] Step c, based on frame indexing Physical height arranged in order Generate a one-dimensional angle-height mapping map. The angle-height mapping establishes the rotation angle of the robotic arm. The corresponding rotation angle With physical height The mapping relationship between them;

[0010] Step d, for a set of spatial polar angles Any spatial polar angle By querying the angle-height mapping map To determine the polar angle of the space Corresponding groove height ;

[0011] Step e, based on polar radius Polar angle and the height of the groove The three-dimensional spatial groove trajectory is synthesized.

[0012] Preferably, in step d, the groove height identified as ,in The function represented by the angle-height mapping. Polar angle With rotation angle The calibration offset between; and, when the query angle Inaccurate Angle-Height Map When using discrete indexing, one-dimensional interpolation is used to calculate the groove height. .

[0013] Preferably, in step b, a preset fixed vertical observation line is used. The vertical line representing the horizontal center of each frame captured by the side-view camera.

[0014] Preferably, in step b, the groove is vertically aligned with the image coordinates. Convert to physical height The process is achieved through a pre-calibrated millimeter / pixel conversion factor.

[0015] Preferably, the groove contour point set is converted into a set of polar radii with the center of the gripper as the origin. and spatial polar angle The steps include: smoothing the groove contour point set and equidistant interpolation.

[0016] Preferably, the method further includes: performing a gripper center calibration step before the robotic arm grasps the product and moves to the working position of the front-view camera; the gripper center calibration step includes: controlling the robotic arm to grasp the calibration object and move to the working position of the front-view camera, acquiring calibration images when the sixth axis of the robotic arm is at 0 degrees and 180 degrees respectively, and analyzing and calculating the position of the gripper center in the coordinate system of the front-view camera.

[0017] Preferably, before processing the front view image to extract the groove contour point set, the method further includes a step of denoising the product front view image; the denoising step includes: calculating the angular gradient of each point in the groove contour point set. ; Loop to check angle gradient Is the maximum value less than the preset angle gradient threshold? If not, delete the contour point corresponding to the maximum value and recalculate the angle gradient until the angle gradient of all remaining contour points is less than the angle gradient threshold. Among them, the angle gradient , For contour points The coordinates.

[0018] Preferably, the method further includes determining the calibration offset. Steps: Determine the calibration offset The steps include: operating the robotic arm to rotate the product so that a known spatial polar angle is found in the product's front view image. The feature point at the location moves to the preset fixed vertical observation line of the side view image. Record the rotation angle of the robotic arm at this point. ; calibrate offset Determined as .

[0019] Preferably, the method is also applied to a system including a dispensing nozzle; the method further includes: acquiring an image of the dispensing nozzle and identifying and calculating the position of the dispensing nozzle. In step e, the three-dimensional spatial groove trajectory is based on the polar radius. Polar angle Groove height and location synthesis.

[0020] Preferably, in step e, the step of synthesizing the three-dimensional spatial groove trajectory further includes determining the spatial polar angle. Determine the orientation information of each point on the three-dimensional spatial groove trajectory so that the glue discharge direction at each point is consistent with the radial direction of that point.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] 1. This solution establishes an image data processing method that utilizes planar contour information extracted from the front view image and a sequence of images synchronously acquired by a side-view camera during the uniform rotation of the workpiece. This method does not perform complex two-dimensional contour analysis on each frame of the side view image, but only extracts one-dimensional height data at specific observation positions in the image. By utilizing the linear correspondence between the acquisition time sequence and the rotation angle of the workpiece, this one-dimensional height value sequence is reconstructed into a mapping map with the rotation angle as the index and the physical height as the value. This data processing method allows any angle coordinate obtained from the front view to directly obtain its corresponding height data by consulting this mapping map, thereby establishing a correspondence between two separate visual acquisition sources based on the inherent temporal correlation of the data.

[0023] 2. The data processing method enables the direct correlation between the height information acquired by the side-view camera and the planar information acquired by the front-view camera through the shared reference system of angle. The accuracy of the system's three-dimensional coordinate synthesis mainly depends on the stability of the planar calibration rotation axis of the front view and the height calibration of the side view. The relative geometric relationship between the spatial installation position of the side-view camera and the rotation axis is no longer the decisive factor affecting the final coordinate accuracy. This makes the system highly tolerant to slight positional drift of the side-view camera caused by environmental vibration or tooling adjustment during deployment and long-term operation, avoiding the vulnerability caused by relying on complex spatial geometric calibration.

[0024] 3. This solution transforms the complex 3D contour reconstruction problem into a data generation and data query process. In the data generation stage, the system only needs to perform edge analysis on the one-dimensional pixel rows in the side view image sequence to construct an angle height mapping map. In the trajectory synthesis stage, the system only needs to calculate the angles of the front view contour points and perform one-dimensional lookup and interpolation of the mapping map. This processing flow avoids feature matching or triangulation operations on multiple 2D side view images during trajectory generation, reducing the complexity of data processing and enabling the system to achieve rapid generation of 3D trajectory points with low computational resource consumption. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the generation of a three-dimensional groove trajectory based on a front view image and a side view time sequence according to the present invention.

[0026] Figure 2 This is a schematic diagram illustrating the data relationship between the frontal contour points and the height mapping points of the present invention;

[0027] Figure 3 The fishbone diagram is a technical step in this invention to generate high-precision, high-resistance three-dimensional trajectories. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the scope of protection of the invention.

[0029] This invention discloses a method for generating three-dimensional spatial groove trajectories based on multi-view images, applied to a system including a robotic arm, a front-view camera, and a side-view camera. Its core technical idea lies in establishing a time-series-based angle-height mapping relationship through a specific image data processing flow, thereby achieving accurate synthesis of planar contour information and spatial height information without relying on complex global three-dimensional geometric calibration. The method first uses the front-view camera to acquire the planar contour of the product and converts it into polar coordinate data with the rotation center as the origin. Then, the side-view camera synchronously acquires an image time series during the product's uniform rotation. Through a one-dimensional feature extraction and temporal reconstruction image data processing method, this image series is compressed and generated into an angle-height mapping map with rotation angle as the index and physical height as the numerical value. Ultimately, the arbitrary spatial polar angle in the planar contour data All can be queried through this mapping graph. To efficiently obtain its corresponding spatial height This process synthesizes a complete three-dimensional trajectory. In a specific engineering practice, the execution of this method begins with establishing the system coordinate reference, that is, calibrating the origin of the polar coordinates of the planar contour. Before the robotic arm grasps the product and moves to the working position of the front-view camera, the system performs a gripper center calibration step. This step involves controlling the robotic arm to grip a standard calibration object, moving it to the working position of the front-view camera, and acquiring calibration images when the sixth axis of the robotic arm is in two states, such as 0 degrees and 180 degrees. By analyzing the images, the center point of the calibration object's features in the two images is calculated. The midpoint of these two center points is then determined as the position of the gripper center in the front-view camera coordinate system. This location It will serve as the sole origin for the transformation of all subsequent planar contour point sets to polar coordinates.

[0030] After establishing the coordinate reference, the robotic arm grasps the product to be measured and moves it to the working position of the front-view camera, acquiring a front-view image of the product through the front-view camera. Given that the original image contour acquired in the industrial field may contain noise spikes caused by burrs or reflections, to ensure the smoothness of subsequent trajectories, the system preferably performs a denoising process before contour extraction. This denoising process is an algorithm based on image data analysis, which first calculates the angular gradient of each point in the groove contour point set. , where the angle gradient , For contour points The coordinates, The value characterizes the degree of change in the local curvature of the contour; the system iteratively determines the angular gradient. Is the maximum value less than a preset angle gradient threshold? For example, the threshold It can be set to 10 degrees according to process requirements; if the current maximum... Value (e.g., 25 degrees) greater than If the maximum value is not found, the contour point corresponding to that value is considered noise and deleted. Then, the angle gradient of the remaining points is recalculated, and this iterative process is repeated until the angle gradient of all remaining contour points is less than the angle gradient threshold. After denoising, the system extracts the groove contour point set using a standard image edge detection algorithm, and then smooths and interpolates the groove contour point set. For example, it uses B-spline fitting and resampling at a fixed arc length (e.g., 0.5 mm) to obtain a set of evenly distributed contour points. Finally, the system will store this set of points. Convert to the previously calibrated gripper center A set of polar radii with the origin and spatial polar angle The conversion formula is: as well as .

[0031] The next step is a crucial step in the method of this invention: acquiring the spatial height and generating the mapping map. This step utilizes image processing techniques to transform a temporal image sequence into a spatial query map. The system executes step a, in which the robotic arm revolves around the center of the gripper at a known, for example, uniform speed of 5 degrees / s. During the rotation, the side-view camera synchronously acquires side-view images of the product at a fixed frame rate, for example, 100Hz, thereby obtaining an image time series. ,in For frame index number, The horizontal image coordinates are... Vertical image coordinates; due to the acquisition frame rate and rotation angular velocity All are constant, frame index number With the rotation angle of the robotic arm There must be a fixed linear correspondence between them, for example This establishes the first re-anchoring relationship between the image frame and the rotation angle; subsequently, the system executes step b, at which point it does not re-anchor each image frame. Instead of performing complex two-dimensional contour analysis, it extracts only a pre-defined, fixed vertical observation line from the image frame. The groove on the image is perpendicular to the coordinates. In a preferred configuration, a preset fixed vertical observation line is used. This refers to the horizontal center and vertical line of each frame captured by the side-view camera. The system only needs to... By performing fast edge detection on this column of pixel data, the pixel coordinates of the groove can be obtained. .

[0032] Preset fixed vertical observation line The specific procedure for determining this is as follows: Control the robotic arm to hold the workpiece and rotate it at least once within the field of view of the side-view camera, and record the workpiece's groove features in the image during this process. The axis refers to the range of motion in the horizontal image coordinate direction; select one of the axes within this range of motion. Coordinate values ​​as ,Should The corresponding vertical observation line at all angles of workpiece rotation Within the specified range, all coordinates can be stably extracted by edge detection algorithms by forming a groove contour. The intersection point, and this intersection point is not obstructed by clamps or other structures; obtain the pixel coordinates of the groove. One specific processing step includes: in a one-dimensional pixel column Up, along The axial direction is analyzed to determine the pixel grayscale value or its first-order gradient, and compared with one or more preset background-workpiece grayscale transition thresholds to search for and locate the upper edge pixel coordinates of the groove. and bottom edge pixel coordinates Then, based on the specific requirements of the process for the trajectory points, Determined as , or To increase the subsequent physical height The resolution can also be adjusted in this step. or Neighboring pixels (e.g.) Within a pixel, sub-pixel precision edge coordinates are calculated using sub-pixel interpolation algorithms such as gray-scale centroid method or Gaussian fitting, and then used for... The determination. Due to It's in pixels; the system needs to convert it to physical height. This is achieved through a pre-calibrated millimeter / pixel conversion factor (e.g., 0.02 mm / pixel), which can be measured by placing a calibration block of known height in the field of view of the side-view camera; next, the system executes step c, based on the frame index. Physical height arranged in order The sequence generates a one-dimensional angle-height mapping. The mapping diagram In terms of data structure, it is a one-dimensional array or lookup table, with its index... That is, indexed by frame The rotation angle obtained by conversion Its value That is, the physical height corresponding to that angle. The process of generating this mapping is essentially to transform a... The three-dimensional image sequence data was reconstructed and compressed into a single image using image processing techniques. The two-dimensional mapping relationship.

[0033] Before querying this mapping, the polar angle of the front view space must be determined. Rotation angle with side view The alignment relationship between them, that is, determining a calibration offset. ;Should The process of determining the value is as follows: Operate the robotic arm to rotate the product so that it is located at a known feature point in the front view (e.g., a point found through image processing at a spatial polar angle). (A marker point), slowly moving to the preset fixed vertical observation line of the side view image. At that location, when the feature point is When detected online, the rotation angle of the robotic arm is immediately recorded. Then the calibration offset That is, it was determined to be ;once Once calibrated, the system can execute step d, for a set of spatial polar angles. Any spatial polar angle By querying the angle-height mapping map Determine the corresponding groove height The specific query function is It should be noted that when querying from different angles... (e.g., 30.27 degrees) Angle-height mapping not precisely hit When using discrete indexes (e.g., stored in 0.1-degree increments), the system employs one-dimensional interpolation (e.g., linear interpolation or cubic spline interpolation) to calculate the precise groove height. Finally, the system executes step e, based on the polar radius. Polar angle And the groove height obtained by table lookup interpolation The system synthesizes three-dimensional spatial groove trajectories. In applications such as adhesive application, the system is also used in systems including adhesive outlets. In this case, the system needs to acquire images of the adhesive outlet using another camera and identify and calculate the location of the outlet. (usually refers to its in) (Tool offset on the axis); and when synthesizing the trajectory, it also includes the spatial polar angle. The purpose of determining the orientation information of each point on the three-dimensional spatial groove trajectory is to ensure that the dispensing direction at each point is consistent with the radial direction at that point. Therefore, the coordinates and orientation of the finally synthesized three-dimensional spatial groove trajectory points in the user coordinate system with the gripper center as the origin can be expressed as (e.g., This generates a complete, accurate, and executable trajectory that includes attitude information.

[0034] Example 1: In a high-speed, continuously operating industrial automated production line, the system of this invention is deployed for precision adhesive application to a workpiece with a complex free-form groove. This production line environment inevitably experiences mechanical vibrations, and the tooling fixtures or camera brackets may experience slight physical positional drift due to daily maintenance, cleaning, or thermal expansion and contraction. This is a known vulnerability of traditional high-precision global geometric calibration schemes, as even sub-millimeter-level calibration failures can lead to trajectory deviations and batch defects. When the side-view camera's mounting bracket is accidentally touched during routine maintenance, causing a slight, unrecorded drift in its spatial position and orientation (external parameters), the traditional solution requires immediate line shutdown and several hours of complex three-dimensional geometric recalibration by professional engineers. The method of this invention, however, demonstrates high tolerance to such real-world industrial disturbances. When processing the next workpiece, the system first acquires a frontal view image using the front-view camera. The two-dimensional calibration relationship of this step, i.e., the gripper center... The relationship with the camera plane remains stable, so the system can still correctly extract the groove profile point set and convert it into a set of unaffected polar radii. and spatial polar angle .

[0035] Subsequently, the workpiece is brought by the robotic arm to the side-view camera where the position has drifted, and the uniform rotation and synchronous acquisition steps begin. At this time, the image data processing mechanism included in the method of this invention starts to operate. The system does not rely on the preset global three-dimensional spatial geometric extrinsic parameters of the side-view camera, which has failed, but as in the specific embodiment, it processes the acquired image time series... Every frame In this process, only the fixed vertical observation line in its own image coordinate system is extracted. The groove on the image is perpendicular to the coordinates. Because of this This refers to the physical drift of the camera relative to its own sensor. The camera's physical drift does not change the internal logic of the image processing; the system can still correctly extract the image from the new perspective after the drift when the workpiece rotates to... At an angle, the physical height of the point directly opposite the camera's observation line. This method transforms a 3D reconstruction problem dependent on a global coordinate system into a 1D mapping generation problem that depends only on a time-angle linear correspondence through image data processing. Based on this, the system instantly generates an angle-height mapping that reflects the actual physical relationship after the current drift. And through the calibrated offset. For each spatial polar angle from the front view Perform table lookup and interpolation operations, i.e. In this step, the spatial polar angle of the planar coordinate system with height coordinates The connection is established at the data level through the shared reference system of angle, namely the angle-height mapping map. The trajectory is dynamically established, rather than relying on a fixed and easily failing physical spatial geometry between two cameras; therefore, the three-dimensional spatial groove trajectory ultimately synthesized by the system is based on... , And newly found The generated image remains accurate, allowing the robotic arm to continue its gluing operation and avoiding production line downtime and recalibration caused by the drift of the side-view camera position.

[0036] Example 2: To objectively verify the accuracy and position tolerance of the method of the present invention in generating a three-dimensional spatial groove trajectory when there is physical position disturbance, the following comparative experiment was set up; the test platform included a six-axis industrial robot arm, an industrial camera (resolution 2048x1536, mounted directly above the workpiece) as a front-view camera, an industrial camera (resolution 1920x1080, mounted to the side of the workpiece) as a side-view camera, and a standard workpiece with a known three-dimensional groove profile (CMM measurement accuracy 0.005mm) calibrated by a coordinate measuring machine (CMM) as the ground truth workpiece; the side-view camera was mounted on a precision linear guide rail, which allows it to generate precisely controllable physical translation in the horizontal direction perpendicular to the optical axis to simulate camera position drift in an industrial setting; the experiment set up two test groups: a control group, which used a conventional multi-view geometric calibration method. Before the experiment, this method established a rigid three-dimensional spatial geometric relationship between the front-view camera, the side-view camera, and the rotation center of the robot arm through a calibration process. When generating the trajectory, it also used the front-view image extraction. and However, its The height was calculated by triangulating the side view image using a calibrated 3D geometric model; the experimental group used the same method as described in the specific implementation, namely, acquiring the image through a frontal camera. and And while the workpiece rotates at a constant speed (set to 6 degrees / second), the side-view camera synchronously acquires image time series. (Sampling frame rate 100Hz), the system only extracts... (Set as the horizontal center line of the image) Pixel coordinates are calculated using a conversion factor of 0.015 mm / pixel. And generate an angle-height mapping map. Ultimately passed ( The height was obtained by interpolation (calibrated to 1.5 degrees).

[0037] The experiment was divided into two phases: Phase 1, the initial state, where all cameras were in their precisely calibrated initial positions (drift = 0mm). The control group method and the experimental group method were used to generate the 3D groove trajectory of the standard part (sampling 360 points), and the 3D spatial Euclidean distance error of each point relative to the true value of the CMM ground was calculated. Finally, the root mean square error (RMSE) was calculated. Phase 2, the drift state, involved horizontally shifting the side-view camera by 2.0mm using a linear guide rail to simulate a physical position drift not perceived by the system. No recalibration was performed at this stage, and the control group method and the experimental group method were used again to generate the 3D groove trajectory of the standard part, and the 3D spatial RMSE was calculated again. To simulate image acquisition noise in the production line environment, random image jitter with a peak value of + / - 0.5 pixels was introduced during camera image acquisition in all the above experimental phases. The planar contour point sets extracted by both methods were denoised using the angle gradient method in the specific implementation, with the angle gradient threshold... The angle was set to 8 degrees; the experiment collected the height measurement errors of the two methods under different drift states at different angles, as shown in Table 1; the overall root mean square error (RMSE) of the three-dimensional spatial trajectory of the experiment is shown in Table 2.

[0038] Table 1: Comparison of Height Measurement Errors at Specific Angle Points

[0039]

[0040] Table 2: Comparison of Root Mean Square Error (RMSE) of Overall Three-Dimensional Spatial Trajectory.

[0041]

[0042] Referring to Tables 1 and 2, under the initial state (0mm drift), both the control and experimental groups were able to generate high-precision 3D trajectories, with RMSE values ​​below 0.022mm, indicating that both methods are effective under ideal calibration. However, referring to Table 2, when the side-view camera experienced a 2.0mm physical drift, the control group's generated trajectory deviated significantly due to the failure of its rigid 3D geometric model, with the RMSE increasing dramatically to 1.684mm. (See the data in Table 1.) The measured height (9.91 mm) at the point of origin has deviated significantly from the true value (8.25 mm), which is unacceptable in industrial applications. In contrast, the RMSE of the experimental group at a drift of 2.0 mm (0.022 mm) remained unchanged compared to the initial state (0.019 mm). The results of the experimental group are attributed to its image data processing flow, namely the angle-height mapping. The generation is based on the currently acquired image time series. The reconstruction is dynamic, and the process automatically compensates for changes in the camera's physical position. The core of its data association is the rotation angle. and Height extracted online Based on the temporal relationship between the images, rather than the spatial position of the camera in the global coordinate system, experimental data shows that this method decouples the dependence on the precise spatial calibration of the side-view camera through this temporal reconstruction-based image data processing approach.

[0043] Example 3: This example combines Figures 1 to 3 This paper describes a method for generating 3D spatial groove trajectories based on multi-view images, as follows: Figure 1 As shown, the process begins with two parallel acquisition steps: acquiring a front view image of the product to obtain the workpiece's planar contour information, and simultaneously acquiring a sequence of side view images while the workpiece rotates at a constant speed. For the front view image, the system relies on the gripper center calibration step to determine the polar coordinate origin of the planar contour, and then performs a planar polar coordinate contour extraction step, which includes denoising processing and converting the contour into polar radius and spatial polar angle. For the side view image sequence, the system performs an angle height mapping map generation step, which extracts only the height on a fixed observation line and reconstructs it in sequence. The system also needs to perform a calibration offset determination step to align the frontal polar angle with the side rotation angle. Subsequently, the system obtains the height by querying the mapping map, which relies on the results of generating the angle height mapping map and determining the calibration offset. The corresponding groove height is obtained by querying the spatial polar angle. Finally, the system performs a three-dimensional spatial groove trajectory synthesis step, which synthesizes the final three-dimensional trajectory based on the polar radius and spatial polar angle obtained from the extracted planar polar coordinate contour and the groove height obtained by querying the mapping map. The process then ends.

[0044] like Figure 2 As shown, the X-axis represents the X coordinate (mm), and the Y / Z axis represents the Y / Z coordinate (mm). The graph contains two curves, where black dots connected by a solid line represent frontal contour points, and gray circles connected by a dashed line represent height mapping points. Figure 3 As shown in the figure, the final result is the generation of a high-precision, high-tolerance three-dimensional spatial groove trajectory, which is decomposed into four main aspects: front view image processing, which includes acquiring the product's front view image, contour denoising, and converting it to planar polar coordinates; side view image time-series acquisition, which includes extracting the fixed observation line height, converting it to physical height, and synchronously acquiring it with uniform rotation; mapping and calibration, which includes generating an angle height mapping map, calibrating the gripper center, determining the calibration offset, and mapping and calibration with table lookup and one-dimensional interpolation; and three-dimensional trajectory synthesis, which includes determining the attitude information, fusing the glue outlet position, and synthesizing three-dimensional coordinates.

[0045] Example 4: This comparative example uses the exact same test platform, standard parts, robotic arm, front-view camera, and side-view camera hardware configuration as Example 2. The only difference between this example and the test group in Example 2 is that it processes the time series images acquired by the side-view camera. At that time, the step of extracting only the fixed vertical observation line was omitted. Height data And generate an angle-height mapping map. Instead of following the traditional steps, the system synthesizes a 3D trajectory using a method based on an approximate correlation between the average height and angle of the image region. Specifically, in this comparative example, the system also acquired the front view image of the standard part and extracted the polar coordinate contour point set. Meanwhile, the standard part rotates at a constant speed of 6 deg / s, and a side-view camera simultaneously captures time-series images during the process. The frame rate is 100Hz, where the frame index is... Corresponding rotation angle However, when performing high correlation, the method used in this comparison is as follows: for each frame of side view image Analyze the horizontal center region of the image, for example Coordinates at Within the range, of which Given a preset window half-width of 50 pixels, calculate the average vertical image coordinates of the points within the groove outline of that area. It is then converted to physical height using the same 0.015 mm / pixel conversion factor as in Example 2. Subsequently, the system directly set this height. With spatial polar angle Approximately equal to the current rotation angle ,Right now Ignore or assume Correlate the contour points with zero values ​​to synthesize 3D trajectory points. Using this comparative method, under the same initial state as Example 2 (0mm drift) and drift state (2.0mm drift), a three-dimensional groove trajectory was generated for the standard part, and its three-dimensional spatial RMSE was calculated; the image acquisition noise and contour denoising parameters used in the experiment, i.e. The degrees are consistent with those in Example 2; the height measurements at specific angle points and the overall three-dimensional spatial RMSE results are shown in Tables 3 and 4.

[0046] Table 3: Comparison of height measurement errors at specific angle points in Comparative Example 1.

[0047]

[0048] Table 4: Comparison of Root Mean Square Error (RMSE) of Overall Three-Dimensional Spatial Trajectory in Comparative Example 1

[0049]

[0050] Referring to Tables 3 and 4, using the method of Comparative Example 1, the RMSE of the generated 3D trajectory in the initial state was 0.185 mm, which is higher than the result of 0.019 mm in the experimental group of Example 2. Its height measurement error at a specific angle point, for example... The error is mm, which is also greater than the error at this point in the experimental group of Example 2. mm; when the side-view camera drifted by 2.0 mm, the RMSE of Comparative Example 1 increased to 0.213 mm; Comparative Example 1 used the method of extracting the average height from a region of the side-view image. and directly with The correlation method failed to capture a specific spatial polar angle. The specific observation line of the side-view camera directly opposite this angle point. The height of time To achieve precise correlation, in the side view image The region's contour information and angular approximation are the sources of error.

[0051] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for generating a three-dimensional groove track based on multi-view images, applied to a system comprising a mechanical arm, a front-view camera and a side-view camera, the method comprising controlling the mechanical arm to move to a working position of the front-view camera and grasp a product, and collecting a front-view image of the product by the front-view camera, processing the front-view image to extract a set of groove contour points, and converting the set of groove contour points into a set of polar radii with the center of the gripper as the origin and a set of spatial polar angles , characterized in that It also includes the following steps: Step a, in the process of rotating the mechanical arm around the center of the gripper at a uniform speed, the side view image of the product is synchronously collected by the side view camera to obtain an image time sequence wherein, is a frame index number, is a horizontal image coordinate, is a vertical image coordinate, frame index number has a linear correspondence relationship between the mechanical arm rotation angle ; Step b. extracting only the groove vertical image coordinates on a preset fixed vertical observation line in each frame image in the image time series the horizontal center vertical line of each frame image collected by the side-view camera​​​​​​ Step c, based on the frame index sequentially arranged physical height , generate a one-dimensional angle-height mapping diagram , the angle-height mapping diagram establishes the mapping relationship between the rotation angle of the mechanical arm corresponding to the rotation angle and the physical height ; Step d, for any spatial polar angle in the set of spatial polar angles , determine the corresponding groove height for that spatial polar angle by querying the angle-height map . Step e, based on polar radius , spatial polar angle , and groove height , synthesize three-dimensional spatial groove trajectories.

2. The method for generating a three-dimensional spatial groove trajectory based on multi-view images according to claim 1, characterized in that, In step d, the height of the groove identified as ,in The function represented by the angle-height mapping. Polar angle With rotation angle The calibration offset between; and, when the query angle Inaccurate Angle-Height Map When using discrete indexing, one-dimensional interpolation is used to calculate the groove height. .

3. The method for generating a three-dimensional spatial groove trajectory based on multi-view images according to claim 1, characterized in that, In step b, the vertical image coordinates of the groove are... Convert to physical height The process is achieved through a pre-calibrated millimeter / pixel conversion factor.

4. The method for generating a three-dimensional spatial groove trajectory based on multi-view images according to claim 1, characterized in that, The groove profile point set is converted into a set of polar radii with the center of the gripper as the origin. and spatial polar angle The steps include: smoothing the groove contour point set and equidistant interpolation.

5. The method for generating a three-dimensional spatial groove trajectory based on multi-view images according to claim 1, characterized in that, The method also includes: performing a gripper center calibration step before the robotic arm grasps the product and moves to the working position of the front-view camera; the gripper center calibration step includes: controlling the robotic arm to grasp the calibration object and move to the working position of the front-view camera, acquiring calibration images when the sixth axis of the robotic arm is at 0 degrees and 180 degrees respectively, and analyzing and calculating the position of the gripper center in the coordinate system of the front-view camera.

6. The method for generating a three-dimensional spatial groove trajectory based on multi-view images according to claim 1, characterized in that, Before processing the front view image to extract the groove contour point set, the method also includes a step of denoising the front view image of the product. The noise reduction process includes: calculating the angular gradient of each point in the groove contour point set. ; Loop to check angle gradient Is the maximum value less than the preset angle gradient threshold? If not, delete the contour point corresponding to the maximum value and recalculate the angle gradient until the angle gradient of all remaining contour points is less than the angle gradient threshold. Among them, the angle gradient , For contour points The coordinates.

7. The method for generating a three-dimensional spatial groove trajectory based on multi-view images according to claim 2, characterized in that, The method also includes determining the calibration offset. Steps: Determine the calibration offset The steps include: operating the robotic arm to rotate the product so that a known spatial polar angle is found in the product's front view image. The feature point at the location moves to the preset fixed vertical observation line of the side view image. Record the rotation angle of the robotic arm at this point. ; calibrate offset Determined as .

8. The method for generating a three-dimensional spatial groove trajectory based on multi-view images according to claim 1, characterized in that, The method is also applied to systems including a dispensing nozzle; the method further includes: acquiring an image of the dispensing nozzle and identifying and calculating the location of the dispensing nozzle. In step e, the three-dimensional spatial groove trajectory is based on the polar radius. Polar angle Groove height and location synthesis.

Images