A 3D vision active time synchronization method and system for robot dynamic tracking

Abstract

The application relates to the technical field of industrial robot vision, and discloses a 3D vision active time synchronization method and system for robot dynamic tracking, wherein a robot controller and a 3D vision controller exchange time stamps, and a fixed time stamp difference value is calculated; during tracking operation, the robot controller periodically sends a hard trigger IO signal and time stamp data, and a conveyor belt encoder triggers 3D line scanning camera to collect line scanning data; the 3D vision controller maps the robot system time stamp into a camera reference time stamp by using the fixed time stamp difference value, positions an image line number after time stamp matching, splices a complete 3D image by combining an image cache area, and sends an object coordinate and a conveyor belt displacement compensation value to the robot controller after the object coordinate and the conveyor belt displacement compensation value are solved; the application solves the problems of image distortion and grabbing failure caused by time synchronization error accumulation by means of a dynamic drift compensation and an abnormal automatic calibration mechanism, and improves time synchronization precision, 3D image splicing accuracy and grabbing success rate.

Description

Technical Field

[0001] This invention relates to the field of industrial robot vision technology, specifically a 3D vision active time synchronization method and system for robot dynamic tracking. Background Technology

[0002] In industrial automated production lines, parallel robots working in conjunction with 3D vision systems to dynamically identify and grasp objects on high-speed conveyor belts is a common application scenario. Among these, 3D line scan cameras are widely used due to their fast imaging speed and high accuracy. To achieve precise grasping, the vision system needs to acquire the 3D contour information of the object and send the object's coordinates to the robot controller in real time. This requires precise time synchronization between the robot controller and the 3D vision controller.

[0003] In practical applications, robot controllers and vision controllers typically operate as two independent systems, each with its own system clock. Traditional synchronization methods often use network time protocols or hardware-triggered I / O signals for clock alignment. Due to slight differences in the crystal oscillators of the two systems, timestamp differences can drift during long-term system operation, causing time synchronization errors to accumulate over time.

[0004] 3D line scan cameras generate images by scanning line by line. The line scan data needs to be aligned with the robot's trigger time to reconstruct a complete 3D image. During high-speed data acquisition, the vision controller needs to simultaneously write the line scan data to the image buffer and provide it for the algorithm to read, placing high demands on the synchronous control of data writing and reading.

[0005] Furthermore, in complex industrial environments, communication interference or hardware fluctuations can cause anomalies in timestamp data transmission. When the vision system fails to complete timestamp matching within the expected timeframe, subsequent processing flows may be affected. Summary of the Invention

[0006] The purpose of this invention is to overcome the problems of the prior art and provide a 3D vision active time synchronization method and system for robot dynamic tracking, which has the advantages of improving time synchronization accuracy, ensuring the accuracy of 3D image stitching and improving the success rate of grasping.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a 3D vision active time synchronization method for robot dynamic tracking, comprising the following steps: S1. The robot controller and the 3D vision controller exchange timestamps. The 3D vision controller calculates a fixed timestamp difference between the robot system timestamp and the camera system timestamp. S2. During the tracking operation, the robot controller periodically sends hard trigger IO signals, robot system timestamps and conveyor belt encoder values ​​to the 3D vision controller, and the conveyor belt encoder triggers the 3D line scan camera to collect line scan data. S3. The 3D vision controller uses the fixed timestamp difference to map the robot system timestamp to the camera reference timestamp, performs timestamp matching, and locates the image row number; it binds the starting row of the image frame to the hard trigger IO signal of the robot controller to realize the synchronization between the vision row starting timestamp and the robot system timestamp; and combines the image buffer to stitch the located row scan data into a complete 3D image. S4. The 3D vision controller calculates the object coordinates from the 3D image, converts them to the robot coordinate system, binds the conveyor belt displacement compensation value, and sends them to the robot controller for dynamic trajectory planning and grasping.

[0008] Furthermore, this application also proposes that the calculation of the fixed timestamp difference between the two in step S1 includes: Multiple sets of data corresponding to the robot system timestamps and camera system timestamps were collected. Unstable trigger intervals were removed, and the timestamp differences of the remaining valid data were averaged to obtain a fixed timestamp difference. ; The fixed timestamp difference Calculated using formula (1), (1) in, n This refers to the total number of times the robot controller sends hard-triggered I / O signals in step S1. d For the number of unstable trigger intervals, and n>d , nd The number of valid data sets; For the first i The robot system timestamp value at the time of the next trigger. For the first i The camera system timestamp value at the time of the next trigger. i The index value that triggers the event.

[0009] Furthermore, this application also proposes to include a dynamic drift compensation step: During system operation, the drift of the fixed timestamp difference is monitored. When the change of the fixed timestamp difference exceeds a set range, the fixed timestamp difference is corrected in real time based on historical difference data to obtain a real-time corrected timestamp difference. ; The timestamp difference of the real-time repair and correction Calculate using formula (2), (2) in, The time since the last refresh The time interval, for The phase difference timestamp difference when it equals 0. The slope of the drift. The slope of the drift By recording multiple sets of different time periods The result was obtained by linear fitting using the least squares method. In the dynamic drift compensation step, a difference threshold and a maximum number of out-of-tolerance values ​​are set. The difference threshold is a threshold for determining whether the fixed timestamp difference has drifted. The maximum number of out-of-tolerance values ​​is the cumulative number of times the fixed timestamp difference exceeds the difference threshold. When the change of the fixed timestamp difference exceeds the difference threshold, the number of out-of-tolerance values ​​is counted. During the execution of step S3, the number of consecutive timestamp matching failures is counted, and a maximum number of consecutive matching failures is set. When the number of out-of-tolerance errors reaches the maximum number of out-of-tolerance errors, or the number of consecutive timestamp matching failures reaches the maximum number of consecutive matching failures, step S1 is re-executed.

[0010] Furthermore, this application also proposes that the parameters in the dynamic drift compensation step be set as follows: The difference threshold is 3-10 milliseconds, preferably 5 milliseconds; The maximum number of out-of-tolerance errors is 5-20 times, preferably 10 times; The maximum number of consecutive matching failures is 10-50 times, preferably 25 times.

[0011] Furthermore, this application also proposes that step S3, which involves stitching together the located line scan data into a complete 3D image using an image buffer, includes: Based on the physical index value of the image cache Locate the starting position of the image row and associate it with the camera system timestamp at that position to align the row scan data with the robot system timestamp. The physical index value of the image cache Calculated using formula (3), (3) in, This represents the current loop write count for the image buffer. For the first This completes the number of times the conveyor belt encoder triggers the camera to perform line scan acquisition in the loop. For each image row height triggered by the conveyor encoder, For the first The row offset within the current loop. The total row height of the image physical array. This represents the modulo operation; The row offset The range of values ​​of satisfies formula (4): ( (4) in, For the first This is the number of times the conveyor encoder has triggered the camera to perform line scan acquisition in the loop.

[0012] Furthermore, this application also proposes that in step S3, during the process of writing line scan data into the image buffer to generate a brightness map and a depth map, a thread lock mechanism is used for control to ensure that reading and writing data do not run in parallel, thereby ensuring the position and data consistency of the brightness map and the depth map.

[0013] Furthermore, this application also proposes that, prior to step S1, a clock synchronization calibration triggering step is included: The system monitors the operating status. When the system startup or communication interruption reconnection time exceeds the set time threshold, the robot controller initiates the time synchronization process and enters the synchronization calibration state. The time synchronization process includes the following signal interaction steps: The robot controller sends a time synchronization start signal to the 3D vision controller; After receiving the time synchronization start signal, the 3D vision controller sends a trigger response signal to the robot controller; After receiving the trigger response signal, the robot controller sends hard trigger IO signals and timestamp data containing the robot system timestamp and conveyor encoder value to the 3D vision controller at set intervals, and starts the response timeout timer. The 3D vision controller records the camera system timestamp when it receives the hard-triggered IO signal, triggers the execution of step S1, and sends a time synchronization completion signal to the robot controller after completing the calculation. After receiving the time synchronization completion signal, the robot controller sends an end signal to the 3D vision controller to complete the time synchronization process. When the response times out or the calculated result of the fixed timestamp difference exceeds the time synchronization accuracy requirement, the synchronization error state is entered and the time synchronization process is re-initiated. When the fixed timestamp difference is calculated and is within the required time synchronization accuracy range, the synchronization is completed. The clock synchronization calibration triggering step sets the number of valid data groups and the required range of time synchronization accuracy to determine whether time synchronization is successful.

[0014] Furthermore, this application also proposes that the parameters in the clock synchronization calibration triggering step be set as follows: The set time threshold is 3-10 minutes, preferably 5 minutes; The response timeout timer is set to 1000-5000 milliseconds, preferably 2000 milliseconds; The set interval for sending hard-triggered IO signals in the time synchronization process is 200-500 milliseconds, preferably 300 milliseconds; The number of valid data sets is 5-20 sets, preferably 10 sets; The required time synchronization accuracy is within the range of ±3-10 milliseconds, preferably ±5 milliseconds.

[0015] Furthermore, this application also proposes a 3D contour vision active synchronization clock system for real-time tracking of parallel robots, including a robot controller, a 3D vision controller, a 3D line scan camera, and a conveyor belt encoder. The robot controller, 3D vision controller, 3D line scan camera and conveyor encoder are interconnected. The robot controller is used to send hard-triggered I / O signals and timestamp data, and to perform trajectory planning based on coordinate information; The 3D vision controller is used to perform image stitching and coordinate calculation based on a fixed timestamp difference. The 3D line scan camera is used to collect line scan data; The conveyor encoder is used to provide hard-trigger I / O signals and position values; The 3D vision controller is used to execute the steps in the above-described 3D contour vision active synchronization clock method for real-time tracking of parallel robots.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention eliminates the problem of accumulated time synchronization errors caused by hardware crystal oscillator differences by exchanging timestamps and calculating fixed timestamp differences between the robot controller and the 3D vision controller, significantly improving time synchronization accuracy. Through a dynamic drift compensation step, the drift of the fixed timestamp difference is monitored in real time. Based on historical difference data, the drift slope is fitted using the least squares method for real-time correction, ensuring the system's synchronization stability over long-term operation. By calculating the physical index value of the image buffer and using a thread lock mechanism, precise alignment between the line scan data and the robot system's timestamps is guaranteed, avoiding line misalignment caused by data read / write conflicts and ensuring the position and data consistency of the brightness and depth maps. Furthermore, by setting difference thresholds, maximum number of out-of-tolerance errors, and maximum number of consecutive matching failures, automatic recalibration under abnormal conditions is achieved, improving the system's robustness and grasping success rate, making it suitable for real-time tracking and precise grasping of objects on high-speed conveyor belts. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the 3D contour vision active synchronization clock method for real-time tracking of parallel robots according to the present invention. Figure 2 This is a flowchart of the time synchronization calibration triggering process in this invention; Figure 3 This is a schematic diagram of the system composition of the present invention; Figure 4 This is a 3D scanned image of the present invention. Detailed Implementation

[0019] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0020] In traditional industrial automated production lines, robot controllers and 3D vision controllers operate as independent systems. Slight differences in their respective system clocks cause timestamp discrepancies to accumulate and drift over time. This drift prevents precise alignment between line scan data and robot trigger times during line-by-line scanning by the 3D line scanner, affecting the accuracy of 3D image reconstruction. Simultaneously, high-speed data acquisition places stringent demands on the synchronous control of image buffer read / write operations. Communication interference or hardware fluctuations can cause abnormal timestamp data transmission, interrupting system processing. The accumulated timestamp discrepancy directly leads to inaccurate mapping between line scan data and trigger times, resulting in incorrect image line number positioning and deviations in object coordinate calculations during 3D image stitching.

[0021] For example, in a logistics sorting system, a conveyor belt transports packages at a constant speed, and a 3D line scan camera is used to collect the 3D contour information of the moving packages. When the robot controller sends a hard trigger I / O signal, clock drift causes a significant deviation between the robot system timestamp received by the vision controller and the camera system timestamp, resulting in incorrect positioning of the corresponding image row number in the line scan data stream. Furthermore, during the writing of line scan data, the image buffer cannot align with the robot trigger time due to timestamp matching failure, leading to misalignment in the stitched 3D image. Consequently, the deviation in object coordinate calculation results is transmitted to the robot controller, interfering with dynamic trajectory planning and causing a shift in the grasping position. In addition, when communication interference occurs, timestamp data is not transmitted in time, the vision controller cannot complete timestamp matching, the system waits for a timeout, the triggering process is halted, and the sorting operation is interrupted.

[0022] If the above problems are not addressed, time synchronization errors will continue to accumulate, reducing the accuracy of 3D image reconstruction and making object coordinate calculation results unreliable. This will lead to errors in robot dynamic trajectory planning, a reduced grasping success rate, and compromised system stability. During long-term operation, the anomaly handling mechanism will be frequently triggered, increasing the risk of system downtime and affecting the continuous operation capability of the production line. In particular, matching failures caused by timestamp transmission anomalies will cause processing to stall, and coordinate calculation deviations will directly lead to grasping action failure, thus affecting the overall system reliability.

[0023] Example 1 In this regard, such as Figure 1 As shown, this application proposes a 3D vision active time synchronization method for robot dynamic tracking, including the following steps: S1. The robot controller and the 3D vision controller exchange timestamps. The 3D vision controller calculates the fixed timestamp difference between the robot system timestamp and the camera system timestamp based on the exchanged timestamps. S2. During the tracking operation, the robot controller periodically sends hard-triggered IO signals, the current robot system timestamp, and the conveyor belt encoder value to the 3D vision controller. At the same time, the conveyor belt encoder triggers the 3D line scan camera to perform line scan data acquisition. S3. The 3D vision controller uses the fixed timestamp difference to map the received robot system timestamp to the camera reference timestamp for timestamp matching, and locates the corresponding image row number in the line scan data stream output by the 3D line scan camera. At the same time, the 3D vision controller binds the starting row of the image frame to the hard trigger IO signal of the robot controller to ensure that the vision row start timestamp is accurately synchronized with the robot system timestamp. Finally, the located line scan data is stitched together into a complete 3D image by combining the image buffer. S4. The 3D vision controller calculates the object coordinates from the complete 3D image, transforms the object coordinates to the robot coordinate system, binds the conveyor belt displacement compensation value, and sends it to the robot controller so that the robot controller can perform dynamic trajectory planning and execute grasping based on the coordinate values ​​and the real-time displacement of the conveyor belt.

[0024] The method provided in this embodiment forms a complete and efficient technical system through active timestamp exchange, precise calculation of fixed timestamp differences, periodic multi-information triggering, mapping-based timestamp matching, and dynamic trajectory planning combined with conveyor belt displacement compensation. This system ensures high-precision time synchronization between the robot and the vision system while enhancing the ability of parallel robots to track and accurately grasp moving objects on high-speed conveyor belts in real time, overcoming technical problems such as time drift, data alignment difficulties, and insufficient dynamic grasping accuracy in existing technologies.

[0025] For ease of understanding, the following explains some key terms in this embodiment: The robot controller is the core control unit of a parallel robot, responsible for receiving instructions, planning motion trajectories, controlling the movement of robot joints, and communicating with other systems.

[0026] The 3D vision controller is the core processing unit of the 3D contour vision system. It is responsible for receiving camera data, performing image processing, calculating the three-dimensional information of the object, and interacting with the robot controller.

[0027] A camera system timestamp is a time stamp recorded by the internal clock of a 3D line scan camera, indicating the time when the camera acquires data or when an event occurs.

[0028] A robot system timestamp is a time stamp recorded by the internal clock of the robot controller, indicating the time when the robot performs an action or sends a signal.

[0029] A fixed timestamp difference refers to the constant time offset between the timestamps of a robot system and a camera system under ideal synchronization conditions. Once determined, this difference can be used to map the timestamps of one system to the timestamps of another system.

[0030] A conveyor belt encoder is a sensor used to measure the distance and speed of a conveyor belt. It typically generates pulse signals by detecting the rotational or linear displacement of the conveyor belt, and these signals can be used to calculate the real-time position of the conveyor belt.

[0031] A 3D line scan camera is a camera that acquires three-dimensional information about the surface of an object by scanning line by line. It is usually equipped with a laser line projector and uses the principle of triangulation to obtain the depth data of each scan line, which is then stitched together to form a complete 3D image.

[0032] Line scan data acquisition refers to the process by which a 3D line scan camera, triggered by a conveyor belt encoder, acquires images and depth data of an object's surface line by line.

[0033] The camera reference timestamp is a camera-side time stamp calculated by the 3D vision controller based on the difference between the robot system timestamp and a fixed timestamp, corresponding to the robot system timestamp. This timestamp is used to locate matching image data in the camera data stream.

[0034] Timestamp matching refers to comparing the timestamp of the robot system or its mapped camera reference timestamp with the timestamp of the line scan data acquired by the camera to determine which line scan data are associated with specific robot events.

[0035] Image line numbers are unique identifiers for each line in the line scan data stream output by a 3D line scan camera. Specific line scan data can be precisely located and extracted using image line numbers.

[0036] The image buffer is a memory area in the 3D vision controller used to temporarily store line scan data acquired by the 3D line scan camera.

[0037] The conveyor belt displacement compensation value is a numerical value calculated based on real-time displacement information provided by the conveyor belt encoder. It is used to correct positional deviations caused by the movement of objects on the conveyor belt. This value ensures that the robot can accurately grasp moving objects.

[0038] In some of the embodiments described above in this application, a robot controller and a 3D vision controller are proposed to exchange timestamps and calculate a fixed timestamp difference between them. However, in the implementation process, if only a simple single or small number of timestamp difference calculations are performed, the calculation results may be inaccurate due to system noise, transient response or occasional data anomalies, which will affect the accuracy of subsequent timestamp matching and the synchronization stability of the overall system.

[0039] In this regard, this application also proposes that the calculation of the fixed timestamp difference between the two in step S1 includes: Multiple sets of data corresponding to the robot system timestamps and camera system timestamps were collected. Unstable trigger intervals were removed, and the timestamp differences of the remaining valid data were averaged to obtain a fixed timestamp difference. ; The fixed timestamp difference Calculated using formula (1), (1) in, n This represents the total number of times the robot controller sends hard-triggered I / O signals in step S1. These hard-triggered I / O signals are sent through the robot controller's GPIO interface. Simultaneously, a timestamp string is sent via the robot's proprietary communication protocol to mark precise synchronization moments. d The number of unstable trigger intervals refers to the number of data sets with large fluctuations in timestamp differences during the initial system startup, when the communication link is newly established, or when it is subjected to strong interference. These data sets are discarded to ensure calculation accuracy. n>d , nd The number of valid data sets; For the first i The robot system timestamp value at the time of the next trigger is generated by the high-precision clock inside the robot controller. For the first i The timestamp value of the camera system at the time of the next trigger is recorded by the internal clock of the 3D line scan camera or 3D vision controller. i The trigger index value is used to identify the order in which each trigger occurs.

[0040] Specifically, collecting multiple sets of corresponding timestamps from the robot system and the camera system aims to obtain sufficient data samples to evaluate the temporal relationship between the two systems. Through multiple triggers and recordings, the distribution and stability of the timestamp differences can be observed, providing a foundation for subsequent accurate calculations. Eliminating unstable trigger intervals aims to identify and exclude timestamp data caused by system startup, communication jitter, hardware initialization, or other transient anomalies. This unstable data significantly interferes with the accurate calculation of fixed timestamp differences, reducing synchronization accuracy. Averaging the timestamp differences of the remaining valid data after eliminating unstable data effectively smooths out random noise in the data, improving the robustness and accuracy of the calculation results. After the above processing, a stable and reliable offset between the robot system timestamp and the camera system timestamp is obtained. This value will serve as the basis for subsequent timestamp matching, ensuring precise temporal alignment between robot actions and visual data acquisition.

[0041] In practical industrial applications, due to various factors such as changes in ambient temperature, aging of electronic components, and fluctuations in system load, the timestamp difference between the robot controller and the 3D vision controller may not be absolutely fixed, but may drift slightly over time. This may lead to a decrease in the accuracy of timestamp matching, which in turn affects the accuracy of 3D image stitching and the reliability of object coordinate calculation, ultimately reducing the accuracy and stability of robot tracking and grasping.

[0042] Furthermore, this application also proposes a dynamic drift compensation step: During system operation, the drift of the fixed timestamp difference is monitored. When the change in the fixed timestamp difference exceeds a set difference threshold, the fixed timestamp difference is corrected in real time based on historical difference data to obtain a real-time corrected timestamp difference. ; The timestamp difference of the real-time repair and correction Calculate using formula (2), (2) in, The time since the last refresh The time interval is calculated in real time by the system clock of the 3D vision controller. for The phase difference timestamp difference when it equals 0 is the reference difference value calculated initially at the time point or each time step S1 is re-executed. The slope of the drift represents the change in the timestamp difference per unit time, and its physical meaning is the rate of clock drift between the robot controller and the 3D vision controller. The slope of the drift By recording multiple sets of different time periods The result was obtained by linear fitting using the least squares method. To effectively manage and respond to drift, the dynamic drift compensation step sets a difference threshold and a maximum number of out-of-tolerance values. The difference threshold is a threshold for determining whether the fixed timestamp difference has drifted, and the maximum number of out-of-tolerance values ​​is the cumulative number of times the fixed timestamp difference exceeds the difference threshold. When the change in the fixed timestamp difference exceeds the difference threshold, the number of out-of-tolerance values ​​is counted. During the execution of step S3, the number of consecutive timestamp matching failures is counted, and a maximum number of consecutive matching failures is set. When the number of out-of-tolerance errors reaches the maximum number of out-of-tolerance errors, or the number of consecutive timestamp matching failures reaches the maximum number of consecutive matching failures, step S1 is re-executed.

[0043] Specifically, the difference threshold, acting as the first line of defense in drift detection, ensures that the system, while requiring high synchronization accuracy, will not frequently misjudge due to minor, harmless fluctuations. When the timestamp difference exceeds this threshold, the system does not immediately trigger resynchronization but instead uses the maximum number of out-of-range errors for further confirmation. This mechanism effectively filters out instantaneous noise or occasional deviations; only when drift persists and accumulates to a certain extent is it confirmed as a true drift, thus avoiding unnecessary system interruptions and resource consumption. Furthermore, the maximum number of consecutive matching failures, as an emergency response mechanism, triggers resynchronization immediately once the number of consecutive matching failures reaches a set value during timestamp matching, even if the cumulative out-of-range error has not yet been reached. This ensures that the system can quickly recover when severe time synchronization deviations lead to ineffective data matching, avoiding data loss or processing errors caused by prolonged matching failures.

[0044] This application's solution introduces a dynamic drift compensation mechanism to continuously monitor and correct the drift of fixed timestamp differences during system operation. When drift is detected to exceed a set range, the system can predict and adjust the timestamp difference in real time based on historical data, thereby maintaining the accuracy of time synchronization. This real-time correction avoids matching errors caused by timestamp drift, ensuring that the line scan data acquired by the 3D line scan camera can be accurately aligned with the robot's trigger time, thus guaranteeing the accuracy of complete 3D image stitching. Furthermore, by setting a difference threshold, a maximum number of out-of-tolerance errors, and a maximum number of consecutive matching failures, this application's solution can intelligently determine the health of the system's synchronization status. Once drift accumulation or matching failures reach a preset threshold, the system will automatically trigger a re-execution of step S1, i.e., re-exchanging timestamps and recalculating the fixed timestamp difference, thereby achieving adaptive synchronization calibration. This mechanism enables the system to resist the effects of environmental changes and component aging during long-term operation, continuously providing high-precision 3D vision data and a reliable time synchronization foundation for robot dynamic trajectory planning and grasping operations.

[0045] This application also proposes that the parameters in the dynamic drift compensation step be set as follows: The difference threshold is 3-10 milliseconds; The maximum number of out-of-tolerance instances is 5-20. The maximum number of consecutive matching failures is 10-50.

[0046] In this embodiment, the threshold for dynamic compensation difference (referring to the accuracy of time synchronization) is set to a default value of 5ms, and the maximum number of out-of-range errors for dynamic compensation is set to a default value of 10. If the number of out-of-range errors exceeds the set default value threshold for difference (5 milliseconds) for a continuous period of time, the slope will be continuously updated.

[0047] The conditions for resynchronizing time are as follows: During operation, the maximum number of consecutive matching failures (the number of consecutive times exceeding the time synchronization accuracy) is 25 by default. At this time, time synchronization should be resynchronized to prevent frame misalignment compensation.

[0048] During continuous high-speed line scanning, image data is constantly written to the image buffer, which typically employs a cyclic writing mechanism. This makes accurately locating the starting position of a specific line scan data and ensuring its precise alignment with the robot's trigger time a challenge. Inaccurate positioning will lead to distortion of the stitched 3D image, thereby affecting the accuracy of object coordinate calculation. To address this, this application proposes that step S3, which involves stitching the located line scan data into a complete 3D image using the image buffer, includes: The starting position of the image row is located based on the physical index value A of the image buffer, and the camera system timestamp at that position is associated to align the row scan data with the robot system timestamp. The physical index value of the image cache Calculated using formula (3), (3) in, This represents the current loop write count for the image buffer, indicating how many loops of data have been completely written to the buffer. For the first This represents the number of times the conveyor belt encoder triggers the camera to perform line scan acquisition in a complete cycle, i.e., the number of lines of line scan data that the buffer can hold in each revolution. The conveyor encoder is triggered once for the corresponding image row height; for example, one row of images is acquired each time it is triggered. For the first The row offset within the current loop indicates which row is currently being written. This is the total row height of the image's physical array, i.e., the size of the buffer. This represents the modulo operation, used for address calculation during loop coverage; It should be noted that the size of the final stitched complete 3D image generated in this embodiment can be set according to the actual size of the object being grasped, but the image size must be less than or equal to the total capacity of the image buffer. M If the set image size exceeds the total capacity, some line scan data will not be stored in the buffer or will be overwritten, thus affecting the integrity of the image stitching. The physical index value A calculated by formula (3) ensures that the required image data can be accurately extracted within the buffer capacity limit.

[0049] The row offset The range of values ​​of satisfies formula (4): ( (4) in, For the first This is the number of times the conveyor encoder has triggered the camera to perform line scan acquisition in the loop.

[0050] Figure 4 The invention demonstrates 3D scanned images obtained during actual operation. Through the above technical solution, this application can accurately locate line scan data in the image buffer and achieve precise alignment between the line scan data and the robot trigger time. In an image buffer environment with continuous high-speed line scanning and cyclic writing, traditional simple indexing methods are prone to data misalignment or splicing errors. However, this solution introduces physical index values ​​based on the number of cyclic writes, encoder trigger counts, and line offsets. A The calculation can accurately determine the actual physical location of each line of scanned data in the cache, ensuring the accuracy of the location even if the data is overwritten in a loop.

[0051] Furthermore, this application also proposes that in step S3, during the process of writing line scan data into the image buffer to generate luminance and depth maps, a thread lock mechanism is used for control to ensure that reading and writing data do not overlap. The thread lock ensures that only one thread accesses a specific line in the buffer at any given time: when the data acquisition thread writes to a certain line, it acquires the write lock for that line, and the image processing thread must wait for the write lock to be released before it can read; and vice versa, to ensure the position and data consistency of the luminance and depth maps.

[0052] Furthermore, this application also proposes that, prior to step S1, a clock synchronization calibration triggering step is included: The system monitors the operational status. When the system startup or communication interruption reconnection time exceeds a set time threshold, the robot controller initiates a time synchronization process, entering a synchronization calibration state. This step aims to proactively initiate and perform a comprehensive time synchronization calibration before system operation or under specific unstable conditions, ensuring a high degree of consistency between the time references of the robot controller and the 3D vision controller. Its purpose is to provide a reliable and accurate time synchronization starting point for subsequent real-time tracking, avoiding performance degradation due to initial synchronization errors. This step can be part of the system initialization process or a key component of the system self-check and fault recovery mechanism.

[0053] like Figure 2 As shown, the time synchronization process includes the following signal interaction steps: The robot controller sends a time synchronization start signal to the 3D vision controller; After receiving the time synchronization start signal, the 3D vision controller sends a trigger response signal to the robot controller; After receiving the trigger response signal, the robot controller sends hard trigger IO signals and timestamp data containing the robot system timestamp and conveyor encoder value to the 3D vision controller at set intervals, and starts the response timeout timer. The periodically sent data is encapsulated into data frames through a preset communication protocol. The data frame contains at least a timestamp field, a separator, a conveyor position value field, and an end identifier. The separator is used to distinguish different fields and can be a specific character (such as ";"). The end identifier is used to mark the end of the data frame for easy parsing by the receiving end. The 3D vision controller records the camera system timestamp when it receives the hard-triggered IO signal, triggers the execution of step S1, and sends a time synchronization completion signal to the robot controller after completing the calculation. After receiving the time synchronization completion signal, the robot controller sends an end signal to the 3D vision controller to complete the time synchronization process. When the response times out or the calculated result of the fixed timestamp difference exceeds the time synchronization accuracy requirement, the synchronization error state is entered and the time synchronization process is re-initiated. When the fixed timestamp difference is calculated and is within the required time synchronization accuracy range, the synchronization is completed. The clock synchronization calibration triggering step sets the number of valid data groups and the required range of time synchronization accuracy to determine whether time synchronization is successful.

[0054] Furthermore, this application also proposes that the parameters in the clock synchronization calibration triggering step be set as follows: The set time threshold is 3-10 minutes, preferably 5 minutes; The response timeout timer is set to 1000-5000 milliseconds, preferably 2000 milliseconds; The set interval for sending hard-triggered IO signals in the time synchronization process is 200-500 milliseconds, preferably 300 milliseconds; The number of valid data sets is 5-20 sets, preferably 10 sets; The required time synchronization accuracy is within the range of ±5 milliseconds.

[0055] In this embodiment, the set time threshold is 5 minutes; The response timeout timer is set to 2000 milliseconds; The set interval for sending hard-triggered I / O signals in the time synchronization process is 300 milliseconds; The number of valid data sets is 10. The required time synchronization accuracy is within the range of ±5 milliseconds.

[0056] Table 1. Timestamp log record table of key events during the time synchronization process of the system of the present invention

[0057] Table 1 shows the timestamp log records of key events during the system's time synchronization process. Among them, "System Time" records the precise moment the event occurred, "Log Level" indicates the importance of the event, and "Event Content" records in detail the signal interaction between the robot controller and the 3D vision controller and the calculation process of the timestamp difference.

[0058] Table 2. Test data table of timestamp difference of continuous valid data in this invention

[0059] Table 2 shows the timestamp test data of 7 consecutive sets of valid data after removing unstable trigger intervals in the clock synchronization calibration triggering step of the present invention, which is used to verify the calculation stability of the fixed timestamp difference.

[0060] Example 2 like Figure 3 As shown, this application also proposes a 3D contour vision active synchronization clock system for real-time tracking of parallel robots, including a robot controller, a 3D vision controller, a 3D line scan camera and a conveyor belt encoder. The robot controller, 3D vision controller, 3D line scan camera, and conveyor belt encoder are interconnected. The robot controller is preferably a parallel robot-specific controller with multi-axis motion control and an Ethernet communication interface. The 3D vision controller is preferably an embedded industrial computer or a high-performance vision processing unit with an image acquisition card or gigabit Ethernet port. The 3D line scan camera is preferably a laser triangulation principle line scan camera with a hardware trigger input interface, whose trigger input terminal is directly connected to the pulse output terminal of the conveyor belt encoder. The conveyor belt encoder is preferably an incremental rotary encoder or a linear encoder, installed on the conveyor belt drive shaft or side, used to detect conveyor belt displacement in real time and output trigger pulses. The robot controller and the 3D vision controller are connected via an industrial Ethernet or serial communication interface for exchanging timestamp data and control commands.

[0061] The robot controller is used to send hard-triggered I / O signals and timestamp data, and to perform trajectory planning based on coordinate information; The 3D vision controller is used to perform image stitching and coordinate calculation based on a fixed timestamp difference. The 3D line scan camera is used to collect line scan data; The conveyor encoder is used to provide hard-trigger I / O signals and position values; The 3D vision controller is used to execute the steps in the above-described 3D contour vision active synchronization clock method for real-time tracking of parallel robots.

[0062] By combining the mechanism of time mapping between the robot controller and the 3D vision controller using a fixed timestamp difference with the line scan data acquisition triggered by the conveyor belt encoder, the system effectively eliminates the problem of accumulated time synchronization errors caused by hardware crystal oscillator differences, achieving real-time and accurate 3D contour tracking and grasping of objects on a high-speed conveyor belt. Specifically, during the initialization phase, the system calculates a fixed timestamp difference through timestamp exchange as a reference for subsequent time synchronization. During tracking operation, the robot controller periodically sends hard-triggered IO signals, the robot system timestamp, and the conveyor belt encoder value. Simultaneously, the conveyor belt encoder triggers the 3D line scan camera to perform line scan data acquisition, ensuring that data acquisition is closely related to the movement of the conveyor belt. The 3D vision controller uses the fixed timestamp difference to map the robot system timestamp to the camera reference timestamp, accurately locates the image line number in the line scan data stream through timestamp matching, and stitches together a complete 3D image using the image buffer. Finally, the object coordinates are calculated and transformed to the robot coordinate system, bound with the conveyor belt displacement compensation value, and sent to the robot controller, supporting dynamic trajectory planning and grasping operations.

[0063] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A 3D vision active time synchronization method for robot dynamic tracking, characterized in that, Includes the following steps: S1. The robot controller and the 3D vision controller exchange timestamps. The robot controller sends a hard-triggered IO signal and the robot system timestamp to the 3D vision controller. The 3D vision controller calculates the fixed timestamp difference between the robot system timestamp and the camera system timestamp. S2. During the tracking operation, the robot controller periodically sends hard trigger IO signals, robot system timestamps and conveyor belt encoder values ​​to the 3D vision controller, and the conveyor belt encoder triggers the 3D line scan camera to collect line scan data. S3. The 3D vision controller uses the fixed timestamp difference to map the robot system timestamp to the camera reference timestamp, performs timestamp matching, and locates the image row number; it binds the starting row of the image frame to the hard trigger IO signal of the robot controller to realize the synchronization between the vision row starting timestamp and the robot system timestamp; and combines the image buffer to stitch the located row scan data into a complete 3D image. S4. The 3D vision controller calculates the object coordinates from the 3D image, converts them to the robot coordinate system, binds the conveyor belt displacement compensation value, and sends them to the robot controller for dynamic trajectory planning and grasping.

2. The 3D vision active time synchronization method for robot dynamic tracking according to claim 1, characterized in that, The step S1 of calculating the fixed timestamp difference between the two includes: Multiple sets of data corresponding to the robot system timestamps and camera system timestamps were collected. Unstable trigger intervals were removed, and the timestamp differences of the remaining valid data were averaged to obtain a fixed timestamp difference. ; The fixed timestamp difference Calculated using formula (1), （1） in, n This refers to the total number of times the robot controller sends hard-triggered I / O signals in step S1. d For the number of unstable trigger intervals, and n>d , nd The number of valid data sets; For the first i The robot system timestamp value at the time of the next trigger. For the first i The camera system timestamp value at the time of the next trigger. i The index value that triggers the event.

3. The 3D vision active time synchronization method for robot dynamic tracking according to claim 2, characterized in that, It also includes a dynamic drift compensation step: During system operation, the drift of the fixed timestamp difference is monitored. When the change of the fixed timestamp difference exceeds a set range, the fixed timestamp difference is corrected in real time based on historical difference data to obtain a real-time corrected timestamp difference. ; The timestamp difference of the real-time repair and correction Calculate using formula (2), （2） in, The time since the last refresh The time interval, for The phase difference timestamp difference when it equals 0. The slope of the drift. The slope of the drift By recording multiple sets of different time periods The result was obtained by linear fitting using the least squares method. In the dynamic drift compensation step, a difference threshold and a maximum number of out-of-tolerance values ​​are set. The difference threshold is a threshold for determining whether the fixed timestamp difference has drifted. The maximum number of out-of-tolerance values ​​is the cumulative number of times the fixed timestamp difference exceeds the difference threshold. When the change of the fixed timestamp difference exceeds the difference threshold, the number of out-of-tolerance values ​​is counted. During the execution of step S3, the number of consecutive timestamp matching failures is counted, and a maximum number of consecutive matching failures is set. When the number of out-of-tolerance errors reaches the maximum number of out-of-tolerance errors, or the number of consecutive timestamp matching failures reaches the maximum number of consecutive matching failures, step S1 is re-executed.

4. The 3D vision active time synchronization method for robot dynamic tracking according to claim 3, characterized in that, The parameters for the dynamic drift compensation step are set as follows: The difference threshold is 3-10 milliseconds; The maximum number of out-of-tolerance errors is 5-20. The maximum number of consecutive matching failures is 10-50.

5. The 3D vision active time synchronization method for robot dynamic tracking according to claim 1, characterized in that, In step S3, the located line scan data is stitched together into a complete 3D image using the image buffer, including: Based on the physical index value of the image cache Locate the starting position of the image row and associate it with the camera system timestamp at that position to align the row scan data with the robot system timestamp. The physical index value of the image cache Calculated using formula (3), （3） in, This represents the current loop write count for the image buffer. For the first This completes the number of times the conveyor belt encoder triggers the camera to perform line scan acquisition in the loop. For each image row height triggered by the conveyor encoder, For the first The row offset within the current loop. The total row height of the image physical array. This represents the modulo operation; The row offset The range of values ​​of satisfies formula (4): ( )（4） in, For the first This is the number of times the conveyor encoder has triggered the camera to perform line scan acquisition in the loop.

6. The 3D vision active time synchronization method for robot dynamic tracking according to claim 1, characterized in that, In step S3, during the process of writing line scan data into the image buffer to generate brightness and depth maps, a thread lock mechanism is used for control to ensure that reading and writing data do not overlap, thereby ensuring the position and data consistency of the brightness and depth maps.

7. The 3D vision active time synchronization method for robot dynamic tracking according to claim 1, characterized in that, Before step S1, a clock synchronization calibration triggering step is also included: The system monitors the operating status. When the system startup or communication interruption reconnection time exceeds the set time threshold, the robot controller initiates the time synchronization process and enters the synchronization calibration state. The time synchronization process includes the following signal interaction steps: The robot controller sends a time synchronization start signal to the 3D vision controller; After receiving the time synchronization start signal, the 3D vision controller sends a trigger response signal to the robot controller; After receiving the trigger response signal, the robot controller sends hard trigger IO signals and timestamp data containing the robot system timestamp and conveyor encoder value to the 3D vision controller at set intervals, and starts the response timeout timer. The 3D vision controller records the camera system timestamp when it receives the hard-triggered IO signal, triggers the execution of step S1, and sends a time synchronization completion signal to the robot controller after completing the calculation. After receiving the time synchronization completion signal, the robot controller sends an end signal to the 3D vision controller to complete the time synchronization process. When the response times out or the calculated result of the fixed timestamp difference exceeds the time synchronization accuracy requirement, the synchronization error state is entered and the time synchronization process is re-initiated. When the fixed timestamp difference is calculated and is within the required time synchronization accuracy range, the synchronization is completed. The clock synchronization calibration triggering step sets the number of valid data groups and the required range of time synchronization accuracy to determine whether time synchronization is successful.

8. A 3D vision active time synchronization method for robot dynamic tracking according to claim 7, characterized in that, The parameters for the clock synchronization calibration triggering step are set as follows: The set time threshold is 3-10 minutes; The response timeout timer is set to 1000-5000 milliseconds; The set interval for sending hard-triggered IO signals in the time synchronization process is 200-500 milliseconds; The number of valid data sets is 5-20 sets; The required time synchronization accuracy is within the range of ±3-10 milliseconds.

9. A 3D contour vision active synchronization clock system for real-time tracking of parallel robots, characterized in that, Includes robot controller, 3D vision controller, 3D line scan camera and conveyor belt encoder; The robot controller, 3D vision controller, 3D line scan camera and conveyor encoder are interconnected. The robot controller is used to send hard-triggered I / O signals and timestamp data, and to perform trajectory planning based on coordinate information; The 3D vision controller is used to perform image stitching and coordinate calculation based on a fixed timestamp difference. The 3D line scan camera is used to collect line scan data; The conveyor encoder is used to provide hard-trigger I / O signals and position values; The 3D vision controller is used to perform the steps in the 3D contour vision active synchronization clock method for real-time tracking of parallel robots as described in any one of claims 1 to 8.

Images