A dual-vision data collaborative communication method based on path triggering and coordinate mapping

Abstract

The application discloses a kind of dual vision data collaborative communication methods based on path triggering and coordinate mapping, belong to key assembly machine dual vision positioning technical field, including path triggering;Data synchronization;Visual pose solution;Coordinate mapping and error prediction;Motion compensation;Final assembly.The application is through the data synchronous acquisition mechanism based on path triggering and the dynamic mapping method across coordinate system, effectively solve the information island problem existing in the existing dual vision positioning system, by the data collected by the second vision component in the middle of motion path, which is considered as invalid, with the motion data of robot arm is collaboratively fused, can real-time perception and compensate the path deviation from picking position to assembly position whole process, thereby significantly improve the positioning accuracy and reliability of final assembly link.

Description

Technical Field

[0001] This invention relates to the field of dual-vision positioning technology for button assembly machines, specifically to a dual-vision data collaborative communication method based on path triggering and coordinate mapping. Background Technology

[0002] With the rapid development of the electronics and information industry, smartphones, tablets, and other electronic products have become an important part of people's daily lives. The manufacturing processes of these devices are becoming increasingly precise and complex. Among them, the assembly of the side buttons on the mid-frame is a key step, and its assembly accuracy and efficiency directly affect the overall aesthetics, feel, and long-term reliability of the device.

[0003] Currently, mainstream fully automatic button assembly equipment in the industry typically adopts a multi-station design and vision-assisted system. For example, a vision component is set up at the loading station to accurately locate the button to be picked up, and another vision component is set up at the assembly station to locate the assembly groove on the middle frame. However, such designs inherently lack system flexibility. Existing equipment is mostly dedicated to specific products, and its vision system and the control logic of the actuator are tightly coupled to the specific product model. When the product is updated and the shape of the middle frame or the layout of the side buttons changes, it is often necessary to make significant modifications or readjustments to the hardware, resulting in long production line adjustment cycles and high costs.

[0004] Furthermore, information silos have formed between its vision systems. Each vision component only serves its local workstation, with functions isolated from each other and data fragmented. The vision system that performs the final assembly task can only perform calculations based on the instantaneous static image of the mid-frame, and is completely unable to perceive and compensate for path deviations caused by vibration, transmission errors, or structural deformation during the movement of the mechanical gripping mechanism from the material picking position to the assembly position. This has become a bottleneck restricting further improvement in assembly accuracy.

[0005] Based on this, the present invention designs a dual-vision data collaborative communication method based on path triggering and coordinate mapping to solve the above problems. Summary of the Invention

[0006] To address the aforementioned shortcomings of existing technologies, this invention provides a dual-vision data collaborative communication method based on path triggering and coordinate mapping.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A dual-vision data collaborative communication method based on path triggering and coordinate mapping includes:

[0009] S1: Based on the motion trajectory of the robotic arm end effector in a unified spatial reference coordinate system, when it enters the preset trigger area, a trigger signal is generated, the first vision unit is simultaneously activated to acquire images, and the motion data of the robotic arm is simultaneously recorded.

[0010] S2: Perform time synchronization processing on the image acquisition and motion data acquisition to form a visual-motion dataset;

[0011] S3: Process the images in the visual-motion dataset, extract the features of the target object, and calculate its relative pose parameters in the first visual unit coordinate system;

[0012] S4: Establish the mapping relationship between the coordinate systems of each visual unit and the unified spatial reference coordinate system, construct a dynamic mapping model based on the motion data and relative pose parameters, and predict the pose error of the robotic arm end effector.

[0013] S5: Convert the pose error into a joint space correction amount, and issue the correction amount before moving to the target assembly point;

[0014] S6: The second vision unit is used to accurately position the target object and the assembly datum to achieve final assembly.

[0015] Furthermore, in S1, a state machine-based triggering logic is adopted, which includes an idle state, a potential triggering state, and a triggering activation state, and the state transition is verified by a debouncing timer.

[0016] Furthermore, S2 specifically involves: performing clock offset compensation to unify the time reference of the visual acquisition and motion control system; pairing the time-synchronized visual data with the motion data and calculating the standard deviation of the time difference of all paired data; when the standard deviation exceeds a preset threshold, determining that the synchronization has failed and discarding the data.

[0017] Furthermore, S3 specifically involves: performing grayscale conversion and Gaussian filtering preprocessing on the image sequentially; extracting the corner pixel coordinates of the target object using a corner detection algorithm; performing distortion correction and coordinate transformation on the corner pixel coordinates using camera intrinsic parameters to obtain their physical coordinates in the first visual unit coordinate system; and calculating the horizontal offset, vertical offset, and rotation angle of the target object based on the physical coordinates.

[0018] Furthermore, in S4, the mapping relationship is established using the checkerboard calibration method, and the transformation matrices from the first visual unit coordinate system and the second visual unit coordinate system to the unified spatial reference coordinate system are obtained respectively.

[0019] Furthermore, the construction of the dynamic mapping model in S4 includes: constructing a high-dimensional feature vector containing error accumulation, fluctuation range, and dynamic characteristics; using the high-dimensional feature vector as input and the actual deviation observed in the second visual unit coordinate system as output, training the mapping model using a multiple linear regression method.

[0020] Furthermore, S5 uses the damped least squares method to convert pose error into joint correction, and introduces a dynamic damping coefficient to adapt to different singularities of the robotic arm.

[0021] Furthermore, the timing for issuing corrections in S5 is based on the pre-compensation distance: obtaining the instantaneous motion speed of the robotic arm's end effector;

[0022] Calculate the minimum time step required for displacement compensation based on the control cycle;

[0023] The pre-compensation distance is the product of instantaneous velocity, time step, and control cycle;

[0024] When the remaining distance from the end of the robotic arm to the target assembly point is less than or equal to the pre-compensation distance, a correction amount is issued.

[0025] Furthermore, in S6, the poses of the target object and the assembly reference are simultaneously acquired through the second vision unit, the relative pose difference is calculated, and a proportional control algorithm is used to generate fine-tuning speed commands to drive the robotic arm to complete the final fitting.

[0026] Furthermore, the robotic arm is a six-axis serial robotic arm, with a gripping head based on the principle of negative pressure adsorption installed at its end;

[0027] The first vision unit is a camera assembly fixed to the side wall of the platform. Its optical axis forms an angle of 45-90° with the movement trajectory of the robotic arm, and its field of view covers the middle area of ​​the path of the robotic arm from the picking position to the assembly position.

[0028] The second vision unit is a camera assembly aligned with the assembly station. Its installation height is lower than the plane of the robotic arm's movement trajectory, and its optical axis forms a 10-30° depression angle with the normal to the top surface of the platform.

[0029] The origin of the unified spatial reference coordinate system is located at the center of the first bolt hole on the top surface of the test bench. The X-axis is parallel to the conveying direction of the middle frame, and the Z-axis is perpendicular to the top surface of the test bench and pointing upwards.

[0030] Compared with the prior art, the beneficial effects of this invention are as follows:

[0031] 1. This invention effectively solves the information silo problem in existing dual-vision positioning systems by using a path-triggered data synchronization acquisition mechanism and a cross-coordinate system dynamic mapping method. By integrating the data collected by the second vision component in the middle of the motion path, which is considered invalid, with the robotic arm motion data, the system can perceive and compensate for the path deviation from the material picking position to the assembly position in real time, thereby significantly improving the positioning accuracy and reliability of the final assembly stage.

[0032] 2. This invention avoids false triggering caused by mechanical vibration or measurement noise by using state machine triggering and debouncing, ensuring the accuracy and stability of the data acquisition starting point. The data synchronization method based on clock offset compensation ensures the consistency of visual data and motion data in the time dimension, laying a reliable foundation for subsequent accurate error prediction and compensation. The constructed dynamic mapping model can adapt to the dynamic errors of the robotic arm in different motion states, realizing high-precision prediction of the end-point pose error.

[0033] 3. This invention overcomes response delay by converting the predicted Cartesian space error into a pre-compensation amount in the joint space and issuing instructions in advance based on the pre-compensation distance. This allows the compensation action to be completed smoothly before reaching the target point, further ensuring the assembly success rate. In addition, this method uses a unified spatial reference coordinate system for coordinate mapping, which enhances the flexibility of the system. When facing changes in product models, it can adapt mainly through software calibration and model updates, reducing hardware modification costs and production line adjustment cycles. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0035] It should be noted that the mechanical parts, vision components, etc. described below use the structural names in our company's patent application No. 202511320778.5 "A Fully Automatic Modular Button Assembly Machine", and the structural spatial layout also follows that.

[0036] Among them, the robotic arm corresponds to the third gripping component, and its end gripping head adopts negative pressure adsorption; the first vision unit is the second vision positioning component, which is installed on the side wall of the frame and the optical axis forms an angle of 45-90° with the path of the robotic arm to ensure that the field covers the middle area of ​​the path of the robotic arm from the material picking position to the assembly position; the second vision unit is the third vision positioning component, which faces the assembly position and the optical axis has a downward angle of 10-30°. The purpose is that the surface of the middle frame is usually a highly reflective material (such as metal or glass), and vertical shooting is prone to specular reflection interference. By using a downward angle design, the optical axis is deviated from the direction of the reflection angle, avoiding strong reflections from obscuring the visual features of the assembly reference. The end effector of the robotic arm (gripping head 2 311 in 202511320778.5) will block the vertical field of view during assembly. A downward angle of 10-30° allows the camera's field of view to cut in from the lower side of the robotic arm, fully capturing the engagement area between the side groove of the middle frame and the button (as shown in Figures 7 and 10 in 202511320778.5). When the ring light source is projected from the downward angle, the shadow generated by the robotic arm is pushed to the outside of the field of view, avoiding obscuring the key alignment area (groove edge positioning point).

[0037] The center point of the first bolt hole of the drive end mounting seat of the first conveying transverse screw assembly for installing the first conveying component is selected on the top surface of the platform and used as the base coordinate system of the robotic arm. The origin of the bolt hole is located by using a metrology-grade spherical probe (whose head diameter is slightly smaller than the nominal diameter of the bolt hole) to make the probe contact the inner wall of the bolt hole at at least four different angles. The three-dimensional coordinates of the precise center point of the bolt hole are calculated by fitting these contact points using the least squares method. The coordinates of this center point are set as the origin (0,0,0).

[0038] Specifically, the method of fitting these contact points using the least squares method to calculate the three-dimensional coordinates of the precise center point of the bolt hole involves using a metrology-grade spherical probe to contact the inner wall of the bolt hole at at least four different angles, collecting the three-dimensional coordinates of multiple (≥4) points. Then, using a standard least squares spatial circle fitting routine, the precise center coordinates of the bolt hole are calculated based on these collected points. This calculation routine can be directly called from general metrology software or mathematical libraries and is a well-known method used by those skilled in the art to extract geometric features from discrete point clouds. The calculated center coordinates are defined as the origin of the robot arm's base coordinate system.

[0039] On the top surface of the platform, locate the corresponding bolt hole of the mounting base furthest from the origin for installing the first conveying assembly. Use the same probe measurement method to determine the center coordinates of this bolt hole. Connect the origin and this point to form a spatial vector. Define the projection direction of this straight vector in the horizontal plane as... The positive direction of the axis, which is consistent with the conveying direction of the middle frame;

[0040] The top surface of the mounting platform serves as the mounting reference surface. Using a high-precision electronic level, the levelness of this top surface is measured at at least four corner points. By adjusting the platform's support feet, the levelness error of the top surface is ensured to be less than 0.02 mm / m. The direction perpendicular to this top surface and upwards is defined as... The positive direction of the axis;

[0041] Determined through physical measurements shaft and The axis is calculated according to the right-hand rule, with the right index finger pointing towards... Positive axis direction, middle finger pointing If the axis is positive, then the direction the thumb points is... The positive direction of the axis.

[0042] Based on the aforementioned unified spatial reference coordinate system, this embodiment discloses a dual-vision data collaborative communication method based on path triggering and coordinate mapping, specifically including the following steps:

[0043] S101: The second visual positioning component is fixed to the platform by its bracket, and its mounting base is measured. Using the known coordinates and the dimensions of the bracket and camera's mechanical drawings, calculate the optical center of the camera lens. coordinates below ( , , Then, a standard checkerboard calibration board (selecting a 10mm corner spacing specification, which is compatible with the camera's field of view width (135mm) and ensures that the calibration board can fully display at least 10×8 corner points within the field of view, balancing calibration accuracy and ease of operation. Too small a spacing (such as 5mm) is prone to misidentification of corner points due to lens distortion, while too large a spacing (such as 20mm) requires a larger calibration space) is used to finely correct the coordinates and obtain the measured coordinates of the optical center.

[0044] The field of view size is the trigger area , The core basis for the axis boundary calculation requires combining the camera image sensor, lens parameters, and actual measurement distance. The physical size of the field of view is obtained by consulting the camera image sensor datasheet to obtain the horizontal pixel count (e.g., ...). ), vertical pixel count (e.g.) ), pixel size (e.g.) ), consult the lens datasheet to obtain the focal length (e.g. ), measure the physical distance from the front of the lens to the surface of the button being grasped (e.g. Combining a 25mm focal length with a 300mm distance, a button (typically 10mm in size) can occupy approximately 300-400 pixels in the image. This ensures accurate feature extraction without the field of view being too small (e.g., 200mm) to fully capture the button, or too large (e.g., 400mm) resulting in an excessively low pixel ratio for the button, affecting corner detection stability. Based on the optical field of view calculation formula, substituting the parameters yields:

[0045] Field of view width (corresponding) (axis range) ;

[0046] Field of view height (corresponding) (axis range) ;

[0047] Triggering area thickness corresponds The axis boundary range is determined based on depth of field (DoF) calculations, with reliability redundancy reserved. Depth of field related parameters are defined as, for example, aperture coefficients. Acceptable fuzzy circle diameter (That is, twice the pixel size to ensure no obvious blur in the image), calculate the basic depth of field using the approximate depth-of-field formula:

[0048]

[0049] Reserve reliability redundancy and determine the thickness of the trigger area:

[0050] ;

[0051] Center of the button target coordinates ( , , Based on the above calculations, (Field of view width) (Field of vision height) (Trigger thickness), defines the bounding box boundary:

[0052] Axis boundaries (matching field of view width):

[0053]

[0054] Axis boundary (matching view height):

[0055] ;

[0056] Axis boundary (matching trigger thickness):

[0057] ;

[0058] Ensure that the target button is completely within the effective detection range in both the horizontal and vertical directions. The axis boundary is designed with depth redundancy to avoid trigger failure or false triggering caused by slight displacement of the button surface;

[0059] It should be noted that among the depth-of-field related parameters mentioned above, the aperture coefficient is selected as 8 (not a limited example), which directly addresses the minute surface displacement of ±0.5mm during the button assembly process. If the aperture coefficient is too small (such as 5.6), the depth-of-field range will be compressed to within ±5mm, and the button may fall out of the clear imaging range with a slight offset, resulting in trigger failure.

[0060] If the aperture factor is too large (such as 11), although the depth of field increases to ±8mm or more, the reduced amount of light will lead to increased image noise. This may cause the blurred image of the adjacent button to be misjudged as the target trigger signal, resulting in false triggering. The value of 8 can strike a balance between covering small displacements and suppressing noise interference.

[0061] Setting the permissible circle of confusion diameter to twice the image sensor pixel size (e.g., 11.72μm when the pixel size is 5.86μm) further adapts to the imaging clarity requirements under minute displacements. This value references the conventional specification in the machine vision field that the circle of confusion must match the sensor's minimum resolution capability. This avoids oversensitivity to minute displacements at 1 times the pixel size (easily misjudged as out of focus due to pixel-level shaking) and prevents feature blurring at sizes greater than 3 times (inability to distinguish subtle positioning marks on the button surface), ensuring that visual data within the range of minute displacements can still be used for accurate pose calculation.

[0062] Based on the redundancy of 0.8 times the base depth of field, it additionally covers the vibration error of ±0.1mm during the movement of the robotic arm. After this error is superimposed on the small displacement of the button itself, the total offset can still be completely contained by the trigger area thickness threshold. At the same time, the 0.8 times coefficient controls the upper limit of the threshold, avoiding the inclusion of non-target areas of adjacent buttons in the detection range due to the trigger area being too large. This fundamentally eliminates the risk of trigger failure or false triggering caused by the combination of small displacement and vibration error.

[0063] In practical applications, parameters can be dynamically adjusted based on button size (e.g., from 10mm to 5mm) or movement speed (e.g., from 50mm / s to 100mm / s). The smaller the button size, the allowable dispersion circle diameter can be reduced to 1.5 times the pixel size to improve positioning accuracy. The higher the movement speed, the redundancy coefficient can be fine-tuned to 0.85 times to cope with greater inertial vibration. The adjustment logic is all based on adapting to the actual small displacement amplitude, and those skilled in the art can directly deduce and implement it without experimentation.

[0064] S102: To achieve accurate and reliable triggering of the second vision component and avoid false or missed triggers caused by robotic arm movement jitter or measurement noise, a state machine-based triggering logic and de-jittering are adopted. This state machine uses the end effector in... Real-time coordinates in coordinate system ( , , Using this as input, the state machine performs state transitions based on its spatial relationship with the trigger area, and combines this with fixed-duration debouncing verification to ensure the stability and synchronization accuracy of the trigger signal. The state machine includes the following three states:

[0065] In the idle state, the end effector has not yet entered the trigger area, or has completely left the trigger area and completed the reset. At this time, no trigger signal is sent to the vision component, and it waits for the entry condition to be met.

[0066] Potential trigger state: The end effector coordinates have entered the trigger region for the first time, but have not yet passed the debouncing verification. This state is used to filter out momentary out-of-bounds or jitter interference.

[0067] Once the activation state is triggered, the de-jitter verification is passed, the trigger signal has been issued, and the vision component enters the working state. This state continues until the end effector completely leaves the trigger area and meets the de-jitter departure condition.

[0068] The initial state is idle;

[0069] The state machine uses the real-time coordinates of the end effector ( , , The spatial relationship between the trigger area and the debouncing timer is the core driver. To achieve reliable state switching, the transition rules are as follows:

[0070] Idle state potential trigger state, until ( , , When all boundary conditions of the trigger region are met simultaneously for the first time, the state changes from idle to potential trigger state, and the entry time is recorded. and initiate a duration of The timer is 5 times the motion control cycle of the robotic arm (1ms). The jitter of the robotic arm is essentially an instantaneous position fluctuation caused by joint inertia and transmission clearance. This type of fluctuation has a short time effect. The servo system will correct the position in real time with a 1ms control cycle. Within 3 cycles, most of the instantaneous fluctuations can be initially corrected. The 5 times cycle reserves redundancy on this basis, which can cover the brief strong jitter in extreme cases (such as the instantaneous offset caused by sudden load changes). At the same time, the 5 times cycle (5ms) will not cause the trigger signal delay due to the long waiting time, as it does with the 10 times cycle, thus missing the opportunity for the first vision unit to collect key data in the middle area of ​​the path. It achieves a balance between filtering effective jitter and ensuring real-time acquisition, and is used to filter jitter interference near the boundary.

[0071] If within the potential triggering state If any coordinate component leaves the trigger area before the timeout, it will immediately reset to the idle state. If the timeout occurs and the coordinates are still completely within the trigger area, it is determined to be a stable entry, and the state transitions to the trigger activation state.

[0072] Triggering the transition from active to idle state occurs when the end effector coordinates completely leave the trigger area and the duration reaches [a certain value]. When it is confirmed that the departure has been stable, the state machine is reset to the idle state and the trigger signal output is stopped. This departure debounce mechanism can avoid the trigger signal being interrupted too early due to instantaneous jitter, ensuring that the vision component has enough time to complete image acquisition.

[0073] S103: At the instant the state machine enters the trigger activation state, the control unit sends a continuous high-level trigger signal to the second vision component, and simultaneously initiates the bandgap synchronous acquisition process of visual data and motion data. The camera of the second vision component is configured in hardware trigger mode. After receiving the continuous high-level trigger signal from the control unit, the camera immediately begins exposure and image acquisition at a fixed frame rate of 200fps, continuously acquiring data. The frame image (the time for the robotic arm to pass through the trigger area is approximately 0.1s (calculated based on a trigger area thickness of 4.32mm and an end effector speed of 50mm / s). At a frame rate of 200fps, 20 frames can be captured in 0.1s, which can fully cover the motion details of the robotic arm within the trigger area without increasing the data transmission and processing burden due to too many frames (such as 30 frames). The frame rate of 200fps is based on the robotic arm servo cycle (0.5ms) and follows the Shannon sampling theorem to ensure that motion details are fully sampled without loss. The camera binds a microsecond-level timestamp provided by the internal crystal oscillator to each frame image. Used for subsequent time alignment;

[0074] Simultaneously with visual acquisition, the control unit, upon entering the trigger activation state, establishes a circular buffer and records the motion data in real time at a frequency of 1kHz (corresponding to a control cycle of 1 ms, ensuring that a set of motion data is acquired within each servo control cycle, avoiding loss of motion details due to excessively low frequency (e.g., 500Hz) or data redundancy due to excessively high frequency (e.g., 2kHz), thus reducing the computational burden on the control unit). Six-DOF pose in coordinate system:

[0075] ( , )

[0076] Each pose data packet is bound to the system timestamp of the control unit. This ensures comparability with visual data on the same time scale.

[0077] S104: To ensure consistency between visual and motion data on the time base, this step achieves clock offset compensation through bidirectional delay measurement. Based on this, precise data alignment and quality verification are performed. Clock synchronization calibration is performed once during initialization and automatically repeated every 30 minutes in subsequent runs (the drift of a conventional crystal oscillator within 30 minutes can be controlled within 10μs, avoiding resource consumption from frequent calibrations (e.g., 15 minutes) and preventing long-term drift (e.g., 60 minutes) from causing time synchronization errors to exceed the allowable range). This dynamically corrects the time deviation caused by crystal oscillator drift. Specifically:

[0078] The control unit sends a message with a transmission timestamp to the camera of the second vision component. The camera receives a synchronization pulse signal; upon receiving the synchronization pulse, it immediately returns a signal with a receiving timestamp. The response signal; the control unit records the time the response is received. Then calculate the one-way transmission delay:

[0079]

[0080] Single clock offset:

[0081]

[0082] Repeat the above process 100 times (100 measurements can reduce random errors (such as offset fluctuations caused by transient electromagnetic interference) to less than 1 / 10 of the error of a single measurement, ensuring the reliability of the average value; 50 measurements will leave more random errors; and 200 measurements will increase the time spent on initialization and calibration), and calculate the average value of the offset. This average value represents the current clock offset of the camera relative to the control unit.

[0083] After clock offset compensation is completed, the control unit will input the camera timestamp. Transformed into a unified control unit time scale Subsequently, for each corrected visual timestamp Find the data packets with the closest timestamps in the motion data sequence to form a synchronized dataset:

[0084]

[0085] Calculate the standard deviation of the time difference for all paired data:

[0086]

[0087] in ;

[0088] like This indicates that the synchronization quality is acceptable, and the data can be used for subsequent processing.

[0089] like If the synchronization fails, the data is discarded to avoid affecting the accuracy of subsequent algorithms.

[0090] threshold Based on the system sampling period and allowable spatial error, it is derived that, at the maximum movement speed of the robotic arm of 0.5 m / s, to ensure that the spatial error caused by the time synchronization error does not exceed 0.1 mm, the synchronization time error needs to be controlled within [specific parameters]. Within.

[0091] S201: After clock skew compensation and data alignment in S104, a time-synchronized visual-motion dataset has been obtained. To achieve accurate key pose calculation in subsequent steps, image preprocessing is required to suppress noise and extract stable feature points. For each frame of the image, grayscale conversion and Gaussian filtering are performed sequentially. First, the color image is converted to grayscale to reduce the amount of data and simplify subsequent calculations. The conversion formula uses weights perceived by the human eye:

[0092]

[0093] Using a 3×3 convolution kernel, standard deviation Gaussian blur is applied to smooth high-frequency noise and preserve edge information. In this invention, using a camera pixel size of 5.86 μm as an example, the 3×3 convolution kernel corresponds to a physical range of approximately 17.6 μm. It allows the filtering and smoothing range to be concentrated within the neighborhood of a single pixel. This avoids noise residue due to a too-small smoothing range (σ < 0.3) or blurring of button edges due to a too-large range (σ > 0.8). Even with minor lens distortion, this smoothing level can cover the local noise caused by the distortion. It operates at a frame rate of 200fps, with 3×3 cores+ The computational load can be run in real time on a conventional control unit without slowing down the overall process. However, if a larger convolutional kernel (such as 5×5) is used, the computation time will double, which cannot meet the requirements of high frame rates. In button detection scenarios, slight changes in lighting can introduce high-frequency noise. The smoothing intensity is just right to filter out this type of noise while preserving the sharpness of the corners. Even if the lighting is 10% stronger / weaker than expected, this parameter works stably, hence its selection. ;

[0094] The Shi-Tomasi corner detection algorithm is used to extract the pixel coordinates of the four corner points of the button. The algorithm uses a 3×3 local window and calculates the autocorrelation matrix of the local window of the image. :

[0095]

[0096] Take the smallest eigenvalue ,like If the value exceeds a threshold, the point is considered a corner. To avoid instability caused by manually setting the threshold, a set of (e.g., 50 frames; this number balances statistical significance and acquisition efficiency. 50 frames can cover different light reflection angles on the button surface, ensuring the representativeness of the statistical feature value distribution. Too few (e.g., 30 frames) may lead to threshold deviation due to insufficient samples, while too many (e.g., 100 frames) will increase preprocessing time) of button images in a static state are collected. For each frame, the minimum feature value R of all candidate corner regions is calculated, and the distribution of these R values ​​is statistically analyzed. The 95th percentile is taken as the detection threshold. The core of taking the 95th percentile is to adapt to the texture characteristics of the button scene. The button surface is mostly a regular plane, and only the four corners are high-response feature areas. The eigenvalues ​​of the key plane are generally low. Taking the 95th percentile can filter out 5% of the low-response areas (mostly background noise and minor surface scratches) while retaining all true corner points. Even if there are tiny stains on the key surface, the low-response area will not be mistaken for a corner point. In industrial inspection scenarios, taking a high percentile (90%-95%) to filter out low-response interference is a common practice. This can avoid the failure of fixed thresholds (such as manually setting a value) when the lighting changes, and can also cope with most scene variables (such as local low response caused by slight dust on the lens). Even if the same model of camera is replaced or the lens focal length is adjusted later, as long as the key size and detection distance remain unchanged, the difference in eigenvalues ​​between corner points and the background will remain unchanged, and the statistical logic of the 95th percentile will still be applicable, without the need to readjust the threshold.

[0097] The above uses the Shi-Tomasi corner detection algorithm to extract the pixel coordinates of the four corners of the button. The core reason for choosing this algorithm is that, compared with Harris corner detection, the Shi-Tomasi algorithm has higher stability in feature point detection in low-contrast areas on the button surface (such as slightly worn edges of the button), and is more robust to changes in lighting. It can adapt to local reflections or stains that may occur in button assembly scenarios, ensuring the consistency of corner point extraction.

[0098] S202: In S201, the pixel coordinates of the four corner points of the button have been extracted from each frame of the image. This step utilizes camera intrinsics to transform pixel coordinates to a second visual coordinate system. The physical coordinates are used to calculate the relative pose parameters (horizontal offset, vertical offset, rotation angle) of the buttons. Using the intrinsic parameter matrix and distortion coefficients obtained from camera calibration, distortion correction is applied to the pixel coordinates of the four corner points. The distortion-corrected pixel coordinates are then projected onto the second visual coordinate system through the intrinsic parameter matrix. Below, we obtain the physical coordinates ( (Unit: mm) It should be noted that, The origin of the coordinate system is located at the foot of the perpendicular from the camera's optical center to the imaging plane. The x-axis is to the right, the y-axis is downward, and the z-axis points along the optical axis toward the object being photographed.

[0099] Assuming the four corner points of the button are at physical coordinates below Arranged in clockwise or counterclockwise order, the horizontal offset reflects the translation of the button center in the x-direction:

[0100]

[0101] The above formula is to take the x-coordinate of the midpoint of the two upper corners and subtract the x-coordinate of the midpoint of the two lower corners to obtain the overall horizontal offset of the button. This calculation method is based on the rectangular symmetrical structure of the button, with the upper and lower sides of the button parallel to the direction of movement of the robotic arm. The difference between the midpoints of the upper and lower sides can directly reflect the translation error of the button along the direction of movement. Compared with the difference between the midpoints of the diagonals (which is easily affected by rotational errors) or the deviation of a single center point (which is easily affected by local deformation), this method can more accurately isolate translational errors and rotational errors, ensuring the targeted nature of subsequent compensation.

[0102] Vertical offset reflects the translation of the button center in the y-direction:

[0103]

[0104] The above formula is to take the y-coordinate of the midpoint of the two left corner points and subtract the y-coordinate of the midpoint of the two right corner points to obtain the overall vertical offset of the button. The calculation logic of vertical offset is the same as that of horizontal offset. The left and right sides of the button are parallel to the vertical movement direction. The difference between the midpoints of the left and right sides can accurately isolate the translation error in the vertical direction, avoid interference from rotation error or local deformation, and form a synergy with the horizontal compensation.

[0105] The rotation angle reflects the rotation of the button around the z-axis:

[0106]

[0107] The above formula is derived from vectors. The direction is calculated to determine the rotation angle of the key, where It is the y-axis component. It is the x-axis component. Able to correctly handle angle values ​​(range) in all quadrants );

[0108] Horizontal and vertical offsets reflect the translational error of the button in the image plane, while the rotation angle reflects the posture deflection of the button. These three parameters can be directly used for subsequent alignment control or hand-eye calibration to achieve precise guidance for button grasping.

[0109] S203: Utilizing a time-synchronized visual-motion dataset Synchronously recorded robotic arm joint angle data The end effector was calculated using a forward kinematics model. Theoretical pose in coordinate system The forward kinematics model is defined as follows: , The forward kinematics calculation function is represented by the formula, which is obtained by multiplying the homogeneous transformation matrices of the links sequentially.

[0110]

[0111] Transformation matrix of each link Includes rotational and translational components, describing the pose of the link relative to the previous link; the end effector is in... The pose in the coordinate system, the transformation matrix from the base coordinate system to the end point and It is obtained by a fixed transformation to the base coordinate system, and physical constraints are provided by a positive kinematic model to avoid drift of purely visual data;

[0112] Convert the displacement increment observed visually to Calculate the displacement increment predicted by the kinematic model in the coordinate system. The residual is calculated as follows:

[0113]

[0114] The residual threshold is determined jointly by the accuracy of the robotic arm and the vision system.

[0115]

[0116] in To improve the repeatability and positioning accuracy of the robotic arm, For visual positioning accuracy, if The frame of data is marked as invalid. The residual threshold combines the accuracy of the two sensors to achieve adaptive anomaly rejection. The coefficient 2 in the formula is based on the redundancy of the dual accuracy error of the robotic arm and the vision system, avoiding misjudging valid data due to the random error of a single sensor. The square root of the accuracy square ensures that the residual threshold can filter out real anomalies without being too strict and causing the loss of valid data.

[0117] For invalid data points, the expected pose is calculated by forward kinematics using the joint angles of the robotic arm before and after the valid points to repair the invalid data points. A moving average filter with a window size of 5 is applied to all data points. This size can filter out high-frequency noise caused by robotic arm vibration (such as small fluctuations caused by joint gaps) while avoiding trajectory lag caused by an excessively large window (such as 7). It also matches the 200fps visual acquisition frame rate to ensure that the filtered data can be synchronized with the visual-motion dataset in real time.

[0118] S301: After trajectory physical constraint verification and fusion in S203, the data in the robot's base coordinate system has been obtained. To achieve precise mapping between vision guidance and robot motion, high-precision end effector trajector trajector trajector trajectories are required, necessitating the establishment of various vision coordinate systems to... The transformation relationship, i.e., obtaining:

[0119] Second visual coordinate system arrive Transformation matrix ;

[0120] Third visual coordinate system arrive Transformation matrix ;

[0121] The transformation matrix above is solved using a checkerboard calibration board and Zhang's calibration method. A high-precision checkerboard calibration board is used, with a known corner spacing (e.g., 10mm). The calibration board is fixed in the robot's workspace, and the robot's end effector is controlled to hold the calibration board, or the calibration board is placed directly in different positions and postures (at least 4 different postures; the camera's intrinsic parameters (focal length, principal point, etc.) are solved by the corner projection relationship under different postures. Three postures will lead to underdetermined equations, making it impossible to uniquely determine the intrinsic parameters; 4 or more postures can ensure that the equations are overdetermined, and the optimal solution is obtained by the least squares method). Multiple sets of calibration board images are captured by the second vision component and multiple sets of calibration board images are captured by the third vision component. The robot's joint angles are recorded simultaneously, and the calibration board's position is calculated through forward kinematics. The pose is solved in two steps:

[0122] Step 1: For each calibration board image, extract the pixel coordinates of the checkerboard corner points and establish the projection relationship from the world coordinate system (fixed to the calibration board plane) to the pixel coordinate system:

[0123]

[0124] in This is the camera intrinsic parameter matrix (focal length, principal point, distortion coefficients, etc.). Using extrinsic parameters (rotation matrix and translation vector), the intrinsic parameter matrix is ​​solved by mapping the corner points of multiple images. With distortion coefficient;

[0125] Step 2: Fix the calibrated internal parameters For each image, calculate the extrinsic parameter matrix from the world coordinate system to the camera coordinate system. Since the world coordinate system is fixed on the calibration plate, and the calibration plate is in Given the pose, the following can be obtained through a coordinate transformation chain:

[0126]

[0127] For the second vision component, repeat the above calibration process to obtain... Similarly, for the third visual component, we obtain... .

[0128] S302: In S301, the second visual coordinate system has been obtained using Zhang's calibration method. and the third visual coordinate system To the robot's base coordinate system Transformation matrix and However, due to factors such as dynamic deformation of the robotic arm, joint clearance, and vision system delay, dynamic errors will occur between the actual pose and the theoretical pose of the end effector during high-speed movement. To compensate for these errors, this step constructs a high-dimensional dynamic mapping model to correlate the error characteristics during movement with the actual deviation, specifically:

[0129] Construction of nonlinear error feature vectors: A high-dimensional feature vector F is defined to describe the error state during motion, including statistical and dynamic features.

[0130]

[0131] in The cumulative sum of horizontal / vertical errors (this cumulative sum is directly related to the path deviation trend of the robotic arm from the material picking position to the current position. The longer the path, the more significant the error accumulation, and the greater the impact on the end assembly deviation. It is the core basis for predicting the final pose error of the end effector). The peak-to-peak value of the error (reflects the fluctuation range of the error, and is directly related to the joint clearance of the robotic arm and the return error of the transmission mechanism; the larger the peak-to-peak value, the higher the instability of the mechanical motion). The standard deviation of the error (the degree of dispersion of the error, which is related to the smoothness of the robotic arm's movement; a large standard deviation means that there are random disturbances (such as vibration) during the movement, and these need to be carefully adapted in the model). For the end effector in The horizontal / vertical velocity (the magnitude of the velocity directly affects the centrifugal force and elastic deformation of the robotic arm joints. Small deformations of the joints during high-speed movement can amplify pose errors, and this dynamic effect needs to be quantified through velocity characteristics). For the end effector in The horizontal / vertical acceleration (acceleration is related to the inertial force of the robotic arm's movement. When it accelerates or decelerates rapidly, it is easy to generate instantaneous vibration errors. This feature can capture the impact of such dynamic disturbances on the error and ensure that the model is adapted to different motion conditions). These features can characterize the cumulative effect, fluctuation range, dispersion degree and dynamic correlation of the error, and provide discrimination information for the model.

[0132] The mapping model based on multiple linear regression is expressed as follows:

[0133]

[0134] in For third visual components in The actual deviation is obtained by comparing the S301 calibration results with the actual pose of the end effector. This is the weight matrix (size 3×10). The bias vector (size 3×1) is used. The advantage of using a linear model is that the parameters are easy to solve and the calculation speed is fast, making it suitable for real-time compensation.

[0135] To obtain model parameters and Perform the following calibration procedure:

[0136] Multiple typical motion trajectories are planned within the robot's workspace, covering different speeds, accelerations, and postures. The robotic arm is controlled to move along these trajectories hundreds of times (simulating the actual working process). Joint angle and visual data are collected simultaneously. The true deviation at each moment is calculated through forward kinematics and visual detection.

[0137]

[0138] Simultaneous calculation of feature vectors ;

[0139] Building the training set ,in Given the total number of samples (typically several thousand to tens of thousands; a high-dimensional feature vector F contains 10 dimensions, and the number of samples needs to be more than 100 times the number of feature dimensions to ensure the model's generalization ability; several thousand samples can meet basic accuracy, tens of thousands of samples can further improve dynamic error adaptability, while hundreds of thousands of samples will significantly increase training time, exceeding the real-time requirements of industry), represent the data in matrix form:

[0140]

[0141] in Introducing augmented matrices ,in It is a column vector of all 1s, used to absorb the bias. The parameter vector is The least squares objective is The closed-form solution is ;

[0142] The above solution is optimal under the Gaussian noise assumption and is computationally stable (in actual implementation, an L2 regularization term can be added to avoid matrix ill-conditioning. The reason for choosing L2 regularization is that it has a more gentle effect on suppressing feature collinearity in multiple linear regression and will not over-compress the weights; the regularization parameter is recommended to be 0.001-0.01, which can be adjusted according to the convergence of the training set residuals to ensure that model accuracy is not lost while suppressing matrix ill-conditioning). During operation, the feature vector is calculated based on the current motion state, substituted into the model to obtain the compensation amount, and the compensation amount is applied to the measurement results of the third vision to achieve dynamic error correction.

[0143] S303: In S302, a high-dimensional dynamic mapping model has been constructed and calibrated. For new assembly tasks, it is only necessary to extract the corresponding error feature vectors during execution. Substituting this into the model allows for the prediction of the assembly error of the end effector at the endpoint. For a new assembly task trajectory, during the movement of the robotic arm, the control unit collects data such as joint angles, velocity, and acceleration in real time. Based on the horizontal and vertical offset errors calculated in the above steps, and according to the feature vector structure defined in S302, the following calculations are performed:

[0144]

[0145] These features comprehensively reflect the error accumulation, fluctuation range, and dynamic characteristics during the motion process, and transform the feature vector of the new task. Substitute into the calibrated mapping model:

[0146]

[0147] in and These are the model parameters obtained by solving S302 using the least squares method. , , The predicted endpoints are respectively The horizontal offset, vertical offset, and rotation angle deviation in the coordinate system are predicted to be ( , , The value represents the pose error that the end effector may have when it reaches the assembly target point without compensation. This result can be directly used to correct the target pose in advance in the motion command to offset the prediction error. If the prediction error exceeds the allowable range, the motion parameters can be adjusted in advance or the task can be paused to avoid assembly failure.

[0148] S401: Convert the Cartesian space deviation into correction values ​​for each joint of the robotic arm. This allows for pre-compensation in motion control, thereby improving final assembly accuracy, based on the current joint angle of the robotic arm. Computational end effector in Jacobian matrix in coordinate system :

[0149]

[0150] The Jacobian matrix describes the linear mapping relationship between joint velocities and the linear and angular velocities of the end effector in Cartesian space. This matrix can be obtained analytically through the kinematic model of the robotic arm (such as the DH parameter method) (suitable for scenarios where the precise link length, joint offset, and other DH parameters of the robotic arm are known), or approximately solved by numerical differentiation (suitable for scenarios where the DH parameters are unknown or there are assembly errors, and derived by calculating the change in end-effector displacement through small joint angle perturbations).

[0151] The predicted value in S303 Deviation in coordinate system By transforming the matrix Switch to In the coordinate system, it is expanded into a six-degree-of-freedom correction vector (linear displacement + angular displacement):

[0152]

[0153] The translation is measured in millimeters, and the rotation in radians. To ensure stable solution of joint corrections even with model errors or singularities, a damped least squares method is employed.

[0154]

[0155] in The damping coefficient is used to improve the condition number of the matrix and prevent the solution from diverging near singular configurations. The identity matrix is ​​used to automatically enhance stability near singular points, and the damping coefficient is... To dynamically change with singularity, the Jacobian matrix is ​​calculated. singular values and Define singularity index The dynamic damping coefficient is:

[0156]

[0157] in This is the basic damping coefficient (the default value under normal configuration). As a regulating factor, controlling The rate of change with singularity (the higher the joint stiffness (e.g., metal joint)). The larger the value (e.g., 5-10), the faster the damping coefficient changes with singularity, resulting in lower joint stiffness (e.g., flexible joints). The smaller the value of k (e.g., 1-3), the less likely the damping coefficient will cause motion shock due to sudden changes in the damping coefficient. When the robotic arm approaches a singularity ( (smaller) Automatically increases, improving the stability of the solution, especially when far from singular points ( (larger) Automatic reduction ensures correction accuracy; the corrected joint angle is:

[0158]

[0159] To prevent the joint from exceeding its mechanical limits, an inspection is performed. Is it within the permitted physical range? If the limit is exceeded, the scale will be adjusted proportionally. ,in For scaling factors, ensure All joint limits are met;

[0160] The damped least squares method can stably solve the joint correction amount even when there are model errors or singular configurations. The dynamic damping coefficient strategy takes into account both accuracy and stability, and the joint limit protection ensures the physical feasibility of the correction amount and avoids mechanical damage.

[0161] S402: In S401, the joint angle correction amount has been calculated using the damped least squares method to compensate for the predicted end-point assembly error. Since there is a response delay between the robotic arm receiving the control command and the end effector stabilizing, issuing the compensation command only when the target point is reached will cause compensation lag and affect assembly accuracy. Therefore, it is necessary to determine the timing of the command issuance in advance during the robotic arm's movement to ensure that the compensation takes effect stably before the end effector reaches the target point. During the robotic arm's movement (through remaining distance calculation and pre-compensation distance calculation), the straight-line distance from the current position of the end effector to the center of the third vision component's field of view (i.e., the assembly target point) is calculated in real time.

[0162]

[0163] in( , , ) is the current end The position below, ( , , ) is the target assembly point at The lower position;

[0164] Define the pre-compensation distance (pre-compensation distance calculation). Calculate the minimum time step required for displacement compensation based on the control cycle. This time step must match the servo cycle of the robotic arm (such as the 1ms motion control cycle mentioned in S102 of this embodiment) and the driver delay (including command transmission and motor current loop response time). If the time step is less than the servo cycle + driver delay, the compensation command cannot be executed before the robotic arm reaches the target point, resulting in compensation lag. If it is greater than this sum, excessive premature accumulation of compensation will lead to endpoint overshoot. Therefore, the minimum time step is determined based on the servo cycle + driver delay. The pre-compensation distance is the product of the instantaneous speed, the time step, and the control cycle, i.e.:

[0165]

[0166] in This represents the current instantaneous velocity of the end effector (obtained by transforming the joint velocity using the Jacobian matrix). The maximum response time is the time required for the end effector to stabilize from the issuance of a control command (which can be obtained through step response testing, including driver delay, robotic arm dynamic response delay, etc.).

[0167] The condition for issuing the instruction is when the following conditions are met. At that time, the control unit immediately transmits the joint angle correction amount via a real-time Ethernet protocol (such as EtherCAT, PROFINET IRT, etc. The core reason for choosing these two protocols is that their cycle can be as low as 1ms, matching the servo cycle of the robotic arm, and they support distributed clock synchronization, which can ensure that the correction command is sent to the driver within 1ms, meeting the real-time requirements of pre-compensation, while also having broad industrial compatibility). The commands are sent to the robotic arm's servo drive and superimposed on the existing motion commands:

[0168]

[0169] This allows error compensation to be completed before the terminal reaches the target point;

[0170] Through the above, the endpoint error caused by the delay in the issuance of compensation instructions is avoided. The dynamic pre-compensation distance calculation adapts to different movement speeds and trajectories. Real-time Ethernet communication ensures that instructions are issued and executed quickly. This forms a closed loop with the joint correction calculation of S401, significantly improving assembly accuracy.

[0171] S403: Following the pre-compensation command issued by S402, the robotic arm's end effector has approached the assembly target point along the corrected trajectory. To ensure final assembly accuracy and fitting quality, a third vision component is used for precise positioning to achieve final fine-tuning. Simultaneously, a mechanism is used to complete assembly confirmation and finished product transfer. After performing pre-compensation, the third vision component simultaneously performs high-resolution imaging of the mid-frame assembly groove and the grasped button (resolution selected as 2560×1440; this resolution allows the edge of the assembly groove (approximately 0.5mm wide) to occupy about 30-40 pixels in the image, meeting the positioning requirement of ±0.05mm assembly tolerance; resolutions below 1920×1080 would be insufficient, and resolutions above 4K would increase data processing time). Through high-precision corner detection and edge fitting algorithms, the groove and button are extracted respectively. Precise pose in coordinate system:

[0172] Groove pose, ;

[0173] Button position and orientation. ;

[0174] According to both Given the current pose, calculate the relative pose difference:

[0175]

[0176] This difference represents the final, minute adjustment required to achieve the fit.

[0177] The final movement command of the end effector is generated using a proportional control (P control) algorithm:

[0178]

[0179] in The proportional gain matrix can be tuned according to response speed and stability requirements. Tuning methods include the conventional trial-and-error method used in industrial control or the Ziegler-Nichols tuning method. The trial-and-error method involves gradually increasing the gain until a slight oscillation occurs, then adjusting it back by 10-20% to determine the optimal value. The Ziegler-Nichols tuning method calculates the overshoot and rise time based on the step response curve. Both methods are directly adaptable to the button interlocking scenarios in this solution, requiring no additional tuning algorithm development. The control unit sends speed commands to the servo driver via real-time Ethernet to fine-tune the end effector in Cartesian space. This drives the robotic arm to perform sub-millimeter-level fine adjustments until the relative positional difference between the button and the groove meets the assembly tolerance (e.g., <±0.05mm and <±0.05°; the design gap between the button and the groove is usually 0.1mm. A translational tolerance of ±0.05mm ensures a tight fit without loosening, and a rotational tolerance of ±0.05° prevents the button from failing to embed due to posture deviation or uneven force after embedding). When the third vision detects that the button has been precisely embedded in the groove and the relative positional difference is within the allowable range, the assembly is considered successful. The second fixing component releases the middle frame, and the first gripping component removes the finished middle frame from the assembly station and places the finished product on the third steering component for the next process.

[0180] It should be noted that the above-mentioned control unit selection and communication method are conventional technical means in the field of industrial automation. In specific implementation, commercially available mature hardware and protocol stacks can be directly selected for communication transmission with robotic arm servo drivers, first vision units, second vision units, etc., so they will not be described in detail.

[0181] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dual-vision data collaborative communication method based on path triggering and coordinate mapping, characterized in that, include: S1: Based on the motion trajectory of the robotic arm end effector in a unified spatial reference coordinate system, when it enters the preset trigger area, a trigger signal is generated, the first vision unit is simultaneously activated to acquire images, and the motion data of the robotic arm is simultaneously recorded. S2: Perform time synchronization processing on image acquisition and motion data acquisition to form a visual-motion dataset; S3: Process images in the visual-motion dataset, extract features of the target object, and calculate its relative pose parameters in the first visual unit coordinate system; S4: Establish the mapping relationship between the coordinate systems of each vision unit and the unified spatial reference coordinate system, construct a dynamic mapping model based on motion data and relative pose parameters, and predict the pose error of the robotic arm end effector. S5: Convert the pose error into a joint space correction amount and issue the correction amount before moving to the target assembly point; S6: The second vision unit is used to accurately position the target object and the assembly datum to achieve final assembly.

2. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 1, characterized in that, In S1, a state machine-based triggering logic is adopted. The state machine includes an idle state, a potential triggering state, and a triggering activation state. The state transition is verified by a debouncing timer.

3. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 1, characterized in that, S2 specifically involves: performing clock offset compensation to unify the time reference of the vision acquisition and motion control system; pairing the time-synchronized vision data with the motion data; and calculating the standard deviation of the time difference of all paired data. When the standard deviation exceeds the preset threshold, the synchronization is deemed to have failed and the data is discarded.

4. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 1, characterized in that, S3 specifically involves: performing grayscale conversion and Gaussian filtering preprocessing on the image sequentially; extracting the corner pixel coordinates of the target object using a corner detection algorithm; performing distortion correction and coordinate transformation on the corner pixel coordinates using camera intrinsic parameters to obtain their physical coordinates in the first visual unit coordinate system; and calculating the horizontal offset, vertical offset, and rotation angle of the target object based on the physical coordinates.

5. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 1, characterized in that, In S4, the mapping relationship is established using the checkerboard calibration method, which obtains the transformation matrices from the first visual unit coordinate system and the second visual unit coordinate system to the unified spatial reference coordinate system.

6. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 1, characterized in that, The construction of the dynamic mapping model in S4 includes: constructing a high-dimensional feature vector that includes error accumulation, fluctuation range and dynamic characteristics; using the high-dimensional feature vector as input and the true deviation observed in the second visual unit coordinate system as output, the mapping model is trained using the multiple linear regression method.

7. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 1, characterized in that, In S5, pose error is converted into joint correction amount using the damped least squares method, and a dynamic damping coefficient is introduced to adapt to different singularities of the robotic arm.

8. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 7, characterized in that, The timing for issuing correction values ​​in S5 is based on the pre-compensation distance: obtaining the instantaneous motion speed of the robotic arm's end effector; Calculate the minimum time step required for displacement compensation based on the control cycle; The pre-compensation distance is the product of instantaneous velocity, time step, and control cycle; When the remaining distance from the end of the robotic arm to the target assembly point is less than or equal to the pre-compensation distance, a correction amount is issued.

9. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 1, characterized in that, In S6, the poses of the target object and the assembly reference are acquired simultaneously through the second vision unit, the relative pose difference is calculated, and a proportional control algorithm is used to generate fine-tuning speed commands to drive the robotic arm to complete the final fitting.

10. The dual-vision data collaborative communication method based on path triggering and coordinate mapping according to claim 1, characterized in that, The robotic arm is a six-axis serial robotic arm, with a gripping head based on the principle of negative pressure adsorption installed at its end; The first vision unit is a camera assembly fixed to the side wall of the platform. Its optical axis forms an angle of 45-90° with the movement trajectory of the robotic arm, and its field of view covers the middle area of ​​the path of the robotic arm from the picking position to the assembly position. The second vision unit is a camera assembly aligned with the assembly station. Its installation height is lower than the plane of the robotic arm's movement trajectory, and its optical axis forms a 10-30° depression angle with the normal to the top surface of the platform. The origin of the unified spatial reference coordinate system is located at the center of the first bolt hole on the top surface of the test bench. The X-axis is parallel to the conveying direction of the middle frame, and the Z-axis is perpendicular to the top surface of the test bench and pointing upwards.

Images