A robotic vision positioning method and system for a motorcycle production line

Abstract

The application discloses a kind of robot vision positioning method and system for motorcycle production line, it is related to robot vision positioning technical field, comprising: step one: generating station two-dimensional observation frame sequence;Step two: extract two-dimensional assembly bone point;Step three: execute timing residual arrangement processing, generate two-dimensional stable bone point track set;Step four: carry out assembly phase slice, generate two-dimensional assembly phase fragment set;Step five: construct two-dimensional positioning receiving diagram and carry out drift compensation, generate two-dimensional positioning receiving sequence;Step six: two-dimensional positioning receiving sequence is input into improved PoseFormer model, is associated by two-dimensional receiving anchor back calculation operator, outputs robot three-dimensional positioning guide result;Step seven: correct robot motion trajectory and generate production line positioning execution record.The application is combined with timing residual arrangement processing and improved PoseFormer model, realizes motorcycle assembly robot three-dimensional positioning stable guide.

Description

Technical Field

[0001] This invention relates to the field of robot vision positioning technology, and in particular to a robot vision positioning method and system for motorcycle production lines. Background Technology

[0002] With the continuous improvement of automation levels in motorcycle production lines, robot vision positioning technology for components such as frames, engines, wheels, and forks has received widespread attention. Current robot positioning methods in motorcycle assembly stations mainly rely on fixed fixtures for positioning, manual teaching points, or single-frame 2D image recognition for target point localization. However, these methods commonly suffer from the following problems in practical applications:

[0003] Assembly images captured by ordinary 2D cameras are easily affected by workstation vibrations, slight camera shake, fixture placement deviations, and local reflection occlusions, causing single-frame jumps or overall drift of 2D assembly bone points, affecting positioning stability. Existing 2D vision positioning methods usually only focus on the target point coordinates themselves, lacking phase differentiation of the approach, alignment, and execution processes between the robot end-effector center and the target assembly bone point. This easily leads to the mixing of rapid approach and fine alignment stages, causing fluctuations in 3D positioning guidance results. At the same time, traditional models mostly use ordinary 2D bone point sequences as input, making it difficult to express the hole correspondence, fixture positioning relationship, and robot end-effector execution relationship in motorcycle assembly processes. It is also difficult to write back the drift state of the fixture positioning pin center, frame mounting hole center, and key anchor points of the robot end-effector center to the target assembly bone point. This results in a lack of stable tooling reference constraints for robot motion trajectory correction, affecting the positioning accuracy and execution reliability in continuous assembly processes on the production line.

[0004] Therefore, how to provide a robot vision positioning method and system for motorcycle production lines is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a robot vision positioning method and system for motorcycle production lines. This invention acquires ordinary two-dimensional video streams of the target assembly station on the motorcycle production line, extracts two-dimensional assembly bone points, performs temporal residual arrangement and assembly phase slicing, constructs a two-dimensional positioning sequence, and then uses the two-dimensional anchor return operator in the improved PoseFormer model to anchor return association between the two-dimensional anchor points and the target assembly bone points, outputs the robot's three-dimensional positioning guidance results, corrects the robot's motion trajectory accordingly, and generates a production line positioning execution record, thereby improving the stability and execution reliability of robot assembly positioning under ordinary two-dimensional vision conditions.

[0006] A robot vision positioning method for a motorcycle production line according to an embodiment of the present invention includes the following steps:

[0007] Step 1: Obtain the ordinary two-dimensional video stream of the target assembly station on the motorcycle production line, and perform frame-by-frame splitting and image preprocessing on the ordinary two-dimensional video stream to generate a two-dimensional observation frame sequence of the station.

[0008] Step 2: Extract two-dimensional assembly bone points from the two-dimensional observation frame sequence of the workstation, and configure bone point type markers, two-dimensional coordinates and frame sequence markers for the two-dimensional assembly bone points to generate a two-dimensional assembly bone point sequence;

[0009] Step 3: Construct a two-dimensional bone point residual matrix based on the two-dimensional assembly bone point sequence, and perform temporal residual arrangement processing to generate a two-dimensional stable bone point trajectory set;

[0010] Step 4: Based on the distance change between the center of the robot end-effector and the target assembly bone point, perform assembly phase slicing on the two-dimensional stable bone point trajectory set to generate a two-dimensional assembly phase segment set;

[0011] Step 5: Based on the two-dimensional assembly phase segment set and the motorcycle assembly process relationship, establish two-dimensional assembly acceptance edges, construct a two-dimensional positioning acceptance diagram and perform drift compensation to generate a two-dimensional positioning acceptance sequence.

[0012] Step 6: Input the two-dimensional positioning and acceptance sequence into the improved PoseFormer model, and use the two-dimensional acceptance anchor return operator to perform anchor return association between the two-dimensional acceptance anchor point and the target assembly bone point, and output the robot's three-dimensional positioning guidance result;

[0013] Step 7: Correct the robot's motion trajectory based on the robot's 3D positioning guidance results, and generate a production line positioning execution record.

[0014] Optionally, step one specifically includes:

[0015] Workstation cameras and robot end-effector cameras are set up at the target assembly station on the motorcycle production line to capture two-dimensional images of motorcycle parts during assembly preparation and assembly execution, and generate ordinary two-dimensional video streams.

[0016] The ordinary two-dimensional video stream is split frame by frame according to the acquisition time sequence to obtain a set of two-dimensional video frames, and the acquisition camera mark, frame order mark and acquisition time mark are written into the two-dimensional video frames.

[0017] The set of two-dimensional video frames is preprocessed, including grayscale conversion, image denoising, and scale unification, to generate a preprocessed set of two-dimensional video frames.

[0018] The preprocessed two-dimensional video frame set is sorted according to the acquisition camera marker, acquisition time marker, and frame sequence marker to generate a workstation two-dimensional observation frame sequence.

[0019] Optionally, step two specifically involves:

[0020] Read each two-dimensional observation frame in the two-dimensional observation frame sequence of the workstation, perform edge extraction and contour segmentation on the motorcycle parts in the two-dimensional observation frames, and obtain the contour region of the motorcycle parts.

[0021] Extract the frame mounting hole center, engine hanger hole center, wheel hub shaft hole center, front fork end hole center, fixture positioning pin center and robot end tool center from the outline area of ​​the motorcycle component, as two-dimensional assembly rib points;

[0022] Configure bone point type markers according to the assembly position category corresponding to the two-dimensional assembly bone points, and configure two-dimensional coordinates according to the pixel positions of the two-dimensional assembly bone points in the two-dimensional observation frame.

[0023] Read the frame sequence marker of the two-dimensional observation frame where the two-dimensional assembly bone point is located, and write the frame sequence marker into the corresponding two-dimensional assembly bone point;

[0024] The two-dimensional assembly bone points are arranged according to the frame sequence marker to generate a two-dimensional assembly bone point sequence.

[0025] Optionally, step three specifically includes:

[0026] Read the two-dimensional assembly bone points with the same bone point type in the two-dimensional assembly bone point sequence, extract the two-dimensional coordinates according to the frame order mark, and generate a bone point temporal coordinate chain.

[0027] Based on the two-dimensional coordinates of adjacent frames in the bone point temporal coordinate chain, calculate the changes in the horizontal coordinate and the vertical coordinate, and then sum the absolute values ​​of the changes in the horizontal coordinate and the vertical coordinate to generate the bone point frame residual amplitude.

[0028] Arrange the inter-frame residual magnitudes of bone points corresponding to the same bone point type label according to the frame order label, generate bone point residual rows, and stack multiple bone point residual rows according to the bone point type label order to construct a two-dimensional bone point residual matrix.

[0029] The two-dimensional bone point residual matrix is ​​subjected to temporal residual orchestration processing, which includes synchronous drift reference generation, isolated perturbation bit marking, and continuity coordinate updating.

[0030] In the generation of synchronization drift reference, the inter-frame residual amplitudes of multiple bone points with differences not exceeding the set synchronization amplitude threshold under the same frame sequence mark are grouped into the same frame synchronization residual group. The changes in the horizontal and vertical coordinates of each two-dimensional assembled bone point in the same frame synchronization residual group are read, and the average values ​​are calculated to generate the synchronization drift reference value.

[0031] In the isolated perturbation bit marking, the inter-frame residual amplitude of the bone point corresponding to the current frame sequence mark in the same bone point residual row is compared with the inter-frame residual amplitude of the bone point corresponding to the previous frame sequence mark and the next frame sequence mark, respectively. When the absolute value of the two comparison differences is greater than the set perturbation difference threshold, the position corresponding to the current frame sequence mark is marked as an isolated perturbation bit.

[0032] In the coordinate update process, the horizontal average value of the synchronization drift reference value is subtracted from the horizontal average value of the horizontal coordinate of the two-dimensional assembly bone point under the corresponding frame sequence mark, and the vertical average value of the vertical coordinate is subtracted from the vertical average value of the synchronization drift reference value to generate synchronization correction coordinates. The synchronization correction coordinates corresponding to isolated disturbance bits are replaced with the average coordinates of the synchronization correction coordinates corresponding to adjacent frame sequence marks to generate a two-dimensional stable bone point trajectory set.

[0033] Optionally, step four specifically involves:

[0034] Read the two-dimensional coordinates corresponding to the center of the robot end-effector tool and the target assembly bone point in the two-dimensional stable bone point trajectory set, and calculate the bone point spacing between the center of the robot end-effector tool and the target assembly bone point according to the frame sequence mark;

[0035] The distance change is calculated based on the bone point spacing corresponding to adjacent frame sequence marks, and an assembly distance change sequence is generated according to the frame sequence marks;

[0036] In the assembly distance change sequence, consecutive frame segments in which the bone point spacing continuously decreases and the distance change is greater than a set approach change threshold are marked as approach phase segments.

[0037] In the assembly distance change sequence, consecutive frame segments where the bone point spacing is less than the set alignment distance threshold and the distance change amount is less than the set alignment change threshold are marked as alignment phase segments.

[0038] In the assembly distance change sequence, consecutive frame segments where the bone point spacing is less than the set execution distance threshold and the two-dimensional coordinate change of the robot end tool center is less than the set execution stability threshold are marked as execution phase segments.

[0039] Use the start and end frame sequence marks of the near phase segment, aligned phase segment, and executed phase segment as slice boundaries;

[0040] The two-dimensional stable bone point trajectory set is assembled into phase slices according to the slice boundary. The coordinates of the two-dimensional assembly bone points within the corresponding frame sequence range are extracted from the two-dimensional stable bone point trajectory set and written into the proximity phase marker, alignment phase marker and execution phase marker respectively to generate a two-dimensional assembly phase segment set.

[0041] Optionally, step five specifically includes:

[0042] Read the coordinates of the two-dimensional assembly bone points, the bone point type markers, the phase markers, and the frame sequence markers in the two-dimensional assembly phase segment set, and read the hole position correspondence, fixture positioning relationship, and robot end effector relationship in the motorcycle assembly process relationship;

[0043] Based on the hole correspondence, hole receiving edges are established between the frame mounting hole center and the engine hanger hole center, and between the wheel hub shaft hole center and the front fork end hole center;

[0044] Based on the fixture positioning relationship, a tooling positioning support edge is established between the center of the fixture positioning pin and the center of the frame mounting hole;

[0045] Based on the aforementioned robot end-effector execution relationship, an end-effector execution receiving edge is established between the robot end-effector tool center and the target assembly bone point;

[0046] Under the same frame sequence mark, the two-dimensional assembly bone point is used as the graph node, and the hole position receiving edge, tooling positioning receiving edge and end execution receiving edge are used as the two-dimensional assembly receiving edge to construct the two-dimensional positioning receiving graph under the corresponding frame sequence mark.

[0047] The two-dimensional coordinates of the fixture positioning pin center during the assembly preparation stage are read as the tooling reference coordinates, and the two-dimensional coordinates of the fixture positioning pin center under the frame sequence mark in the approach phase segment, alignment phase segment and execution phase segment are read as the tooling current coordinates.

[0048] Subtract the horizontal coordinate of the tooling reference coordinate from the horizontal coordinate of the current tooling coordinate to obtain the horizontal tooling drift amount; subtract the vertical coordinate of the tooling reference coordinate from the vertical coordinate of the current tooling coordinate to obtain the vertical tooling drift amount.

[0049] Under the same frame sequence mark, subtract the horizontal tooling drift from the horizontal coordinate of each two-dimensional assembly bone point coordinate in the two-dimensional positioning and connection diagram, and subtract the vertical tooling drift from the vertical coordinate of each two-dimensional assembly bone point coordinate to generate a drift-compensated two-dimensional positioning and connection diagram.

[0050] Following the phase order of approach phase segment, alignment phase segment, and execution phase segment, the drift-compensated 2D positioning and connection map corresponding to each frame sequence mark in each phase segment is read, and multiple drift-compensated 2D positioning and connection maps are arranged sequentially according to the frame sequence mark to generate a 2D positioning and connection sequence.

[0051] Optionally, the improved PoseFormer model specifically includes a two-dimensional bone point embedding layer, a receiving spatial encoding layer, a phase temporal encoding layer, an assembly drift reflection layer, and a three-dimensional positioning output layer, wherein a two-dimensional receiving anchor return operator is set in the assembly drift reflection layer;

[0052] The two-dimensional positioning sequence is input into the two-dimensional bone point embedding layer. The two-dimensional bone point embedding layer performs linear mapping on the coordinates of the two-dimensional assembly bone points, and adds the linear mapping result with the bone point type label embedding result and the phase label embedding result to form the two-dimensional bone point embedding feature.

[0053] The two-dimensional bone point embedding features are input into the receiving spatial coding layer. The receiving spatial coding layer performs query mapping, key mapping, and value mapping on the two-dimensional bone point embedding features under the same frame sequence label. The query mapping results and key mapping results are multiplied item by item and then summed to generate the bone point spatial attention score.

[0054] The offsets of the bearing edges corresponding to the hole bearing edge, tooling positioning bearing edge, and end execution bearing edge are added to the bone point space attention score. After normalization by the Softmax function, the score is multiplied by the value mapping result and summed to output the bone point features in the bearing space.

[0055] The receiving spatial bone point features are input into the phase-temporal coding layer. The phase-temporal coding layer reads the receiving spatial bone point features of the same bone point type in the near phase segment, aligned phase segment and executed phase segment according to the frame sequence label. The corresponding features of adjacent frame sequence labels are multiplied one by one and then summed to generate a temporal attention score.

[0056] The phase progressive bias from the near phase segment to the aligned phase segment and from the aligned phase segment to the executed phase segment is added to the temporal attention score. After normalization by the Softmax function, the spatial bone point features under different frame order markers are weighted and summed to output the phase temporal bone point features.

[0057] The phase-time bone point features are input into the assembly drift reflection layer. The assembly drift reflection layer reads the connection relationship between the fixture positioning pin center, the frame mounting hole center, the robot end tool center and the target assembly bone point through the two-dimensional anchor return operator. The anchor return convergence features of the two-dimensional anchor point are written back to the phase-time bone point features corresponding to the target assembly bone point, and the assembly drift reflection features are output.

[0058] The assembly drift reflection features are input into the 3D positioning output layer. The 3D positioning output layer sequentially performs fully connected mapping, layer normalization processing, and GELU activation processing to output the robot's 3D positioning guidance results. The robot's 3D positioning guidance results include the 3D coordinates of the target assembly point and the positioning confidence.

[0059] Optionally, the two-dimensional receiving anchor return operator is specifically:

[0060] The center of the fixture positioning pin, the center of the frame mounting hole, and the center of the robot end tool are determined as two-dimensional receiving anchor points, and the target assembly bone point is determined as the anchor return target bone point. The phase time bone point features corresponding to the two-dimensional receiving anchor point and the anchor return target bone point are read respectively.

[0061] Read the two-dimensional coordinates of the anchor point under the current frame sequence mark and the two-dimensional coordinates under the previous frame sequence mark. Subtract the horizontal coordinate of the previous frame sequence mark from the horizontal coordinate of the current frame sequence mark to generate the horizontal displacement of the anchor point. Subtract the vertical coordinate of the previous frame sequence mark from the vertical coordinate of the current frame sequence mark to generate the vertical displacement of the anchor point. Set both the horizontal and vertical displacement of the anchor point corresponding to the first frame sequence mark to zero.

[0062] The lateral and longitudinal displacements of the anchor point are concatenated to form an anchor point displacement vector, and the anchor point displacement vector is linearly mapped to generate an anchor point offset feature.

[0063] The anchor point offset feature is added to the phase time bone point feature corresponding to the two-dimensional anchor point to generate the anchor point acceptance feature.

[0064] Search for the two-dimensional bonding path from the bonding anchor point to the anchor return target bone point along the bonding edge of the hole, the bonding edge of the tooling positioning, and the bonding edge of the end execution. Count the number of two-dimensional assembly bonding edges traversed by the bonding path to obtain the bonding jump count.

[0065] The anchor point acceptance feature is multiplied by the phase time bone point feature corresponding to the anchor return target bone point, and then summed to generate the anchor return correlation score.

[0066] Multiply the number of hops received by the set hop attenuation coefficient to generate the path attenuation score, and subtract the path attenuation score from the anchor return related score to generate the anchor return modulation score;

[0067] The anchor return modulation scores corresponding to multiple two-dimensional anchor points are input into the Softmax function to obtain the anchor return weights. The anchor point acceptance features corresponding to each two-dimensional anchor point are multiplied by the corresponding anchor return weights and then summed to generate the anchor return convergence features.

[0068] The anchor return convergence feature is input into the Sigmoid function to obtain the anchor return gating value. The anchor return convergence feature and the anchor return gating value are then multiplied element by element and superimposed onto the phase time bone point feature corresponding to the anchor return target bone point to output the assembly drift reflection feature.

[0069] Optionally, step seven specifically includes:

[0070] Read the three-dimensional coordinates and positional confidence of the target assembly point from the robot's three-dimensional positioning guidance results;

[0071] When the positioning confidence is greater than or equal to the set execution confidence threshold, the current position of the robot end-effector center is obtained, and the three-dimensional coordinates of the target assembly point are subtracted from the current position of the robot end-effector center to generate the end-effector positioning offset.

[0072] The position of the robot end-effector center in the robot motion trajectory is corrected according to the end-effector positioning offset, and the robot end-effector center is controlled to move towards the target assembly point according to the corrected robot motion trajectory.

[0073] When the positioning confidence is less than the set execution confidence threshold, the movement of the robot's end effector center is stopped, and a repositioning command is output.

[0074] Based on the robot end-effector center position, target assembly point 3D coordinates, positioning confidence, and execution status before and after robot motion trajectory correction, a production line positioning execution record is generated.

[0075] A robot vision positioning system for a motorcycle production line according to an embodiment of the present invention includes the following modules:

[0076] The video preprocessing module is used to acquire the ordinary two-dimensional video stream of the target assembly station of the motorcycle production line, perform frame-by-frame splitting and image preprocessing on the ordinary two-dimensional video stream, and generate a two-dimensional observation frame sequence of the station.

[0077] The two-dimensional assembly bone point extraction module is used to extract two-dimensional assembly bone points from the two-dimensional observation frame sequence of the workstation, and to configure bone point type markers, two-dimensional coordinates and frame sequence markers for the two-dimensional assembly bone points to generate a two-dimensional assembly bone point sequence.

[0078] The temporal residual orchestration module is used to construct a two-dimensional bone point residual matrix based on the two-dimensional assembly bone point sequence, and perform temporal residual orchestration processing to generate a two-dimensional stable bone point trajectory set.

[0079] The assembly phase slicing module is used to slice the two-dimensional stable bone point trajectory set into assembly phase slices based on the distance change between the center of the robot end tool and the target assembly bone point, and generate a two-dimensional assembly phase segment set.

[0080] The connection diagram construction and compensation module is used to establish two-dimensional assembly connection edges based on the two-dimensional assembly phase segment set and the motorcycle assembly process relationship, construct a two-dimensional positioning connection diagram and perform drift compensation to generate a two-dimensional positioning connection sequence.

[0081] The three-dimensional positioning guidance module is used to input the two-dimensional positioning sequence into the improved PoseFormer model, and to perform anchor return association between the two-dimensional anchor point and the target assembly bone point through the two-dimensional anchor return operator, and output the robot's three-dimensional positioning guidance result.

[0082] The trajectory correction and recording module is used to correct the robot's motion trajectory based on the robot's three-dimensional positioning guidance results and generate a production line positioning execution record.

[0083] The beneficial effects of this invention are:

[0084] This invention converts ordinary two-dimensional video streams into a sequence of two-dimensional observation frames at the workstation, and further extracts two-dimensional assembly bone points, constructs a two-dimensional bone point residual matrix and a two-dimensional stable bone point trajectory set, so that the key assembly positions in the motorcycle production line form a continuous and stable two-dimensional bone point representation, reducing the impact of camera shake, tooling vibration and local false detection on the front-end visual positioning data.

[0085] This invention distinguishes the approach, alignment, and execution processes between the robot end-effector center and the target assembly bone point by constructing an assembly phase slice and a two-dimensional positioning and connection diagram. It also expresses the hole connection, tooling positioning, and robot end-effector execution relationships in the same diagram structure, thereby improving the ability of the two-dimensional positioning and connection sequence to express the actual motorcycle assembly process.

[0086] This invention improves the PoseFormer model to guide 3D positioning of 2D positioning and acceptance sequences. Based on the standard PoseFormer spatiotemporal modeling, it introduces acceptance spatial encoding, phase temporal encoding, and a 2D acceptance anchoring return operator. This enables the model to not only learn the mapping relationship between 2D assembly bone points and the 3D coordinates of the target assembly point, but also to express assembly process constraints using hole acceptance edges, tooling positioning acceptance edges, and end-effector execution acceptance edges. Furthermore, it writes back the anchoring return convergence features of the fixture positioning pin center, the frame mounting hole center, and the robot end-effector tool center to the target assembly bone point. This distinguishes between overall tooling drift, actual component displacement, and robot end-effector approach actions, reducing the transmission of 2D detection errors to 3D positioning results, improving the stability of the target assembly point's 3D coordinates and positioning reliability, and enhancing the reliability of robot motion trajectory correction. Attached Figure Description

[0087] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0088] Figure 1 This is an overall flowchart of a robot vision positioning method for a motorcycle production line proposed in this invention.

[0089] Figure 2 This is a schematic diagram of the distribution of assembly bone points in a motorcycle, according to the robot vision positioning method for a motorcycle production line proposed in this invention.

[0090] Figure 3 This is a schematic diagram of the improved PoseFormer model structure for a robot visual positioning method for a motorcycle production line proposed in this invention.

[0091] Figure 4 This is a schematic diagram of the structure of a robot vision positioning system for a motorcycle production line proposed in this invention. Detailed Implementation

[0092] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0093] refer to Figures 1-3 A robot vision positioning method for a motorcycle production line includes the following steps:

[0094] Step 1: Acquire the ordinary two-dimensional video stream of the target assembly station on the motorcycle production line, split the ordinary two-dimensional video stream frame by frame and perform image preprocessing to generate a sequence of two-dimensional observation frames for the station;

[0095] Step 2: Extract two-dimensional assembly bone points from the two-dimensional observation frame sequence of the workstation, and configure bone point type labels, two-dimensional coordinates and frame sequence labels for the two-dimensional assembly bone points to generate a two-dimensional assembly bone point sequence;

[0096] Step 3: Construct a two-dimensional bone point residual matrix based on the two-dimensional assembly bone point sequence, and perform temporal residual arrangement processing to generate a two-dimensional stable bone point trajectory set;

[0097] Step 4: Based on the distance change between the center of the robot's end-effector and the target assembly bone point, perform assembly phase slicing on the two-dimensional stable bone point trajectory set to generate a two-dimensional assembly phase segment set;

[0098] Step 5: Based on the two-dimensional assembly phase segment set and the motorcycle assembly process relationship, establish two-dimensional assembly acceptance edges, construct a two-dimensional positioning acceptance diagram and perform drift compensation to generate a two-dimensional positioning acceptance sequence.

[0099] Step 6: Input the two-dimensional positioning and acceptance sequence into the improved PoseFormer model, and use the two-dimensional acceptance anchor return operator to perform anchor return association between the two-dimensional acceptance anchor point and the target assembly bone point, and output the robot's three-dimensional positioning guidance result;

[0100] Step 7: Correct the robot's motion trajectory based on the robot's 3D positioning guidance results and generate a production line positioning execution record.

[0101] In this embodiment, step one specifically includes:

[0102] Workstation cameras and robot end-effector cameras are set up at the target assembly station on the motorcycle production line to capture two-dimensional images of motorcycle parts during assembly preparation and assembly execution, and generate ordinary two-dimensional video streams.

[0103] The ordinary two-dimensional video stream is split frame by frame according to the acquisition time sequence to obtain a set of two-dimensional video frames, and the acquisition camera mark, frame order mark and acquisition time mark are written into the two-dimensional video frames.

[0104] Image preprocessing is performed on a set of two-dimensional video frames. Image preprocessing includes grayscale conversion, image denoising, and scale unification to generate a preprocessed set of two-dimensional video frames.

[0105] The preprocessed two-dimensional video frame set is sorted according to the acquisition camera mark, acquisition time mark, and frame sequence mark to generate a workstation two-dimensional observation frame sequence;

[0106] In the specific implementation process, the station camera is fixedly installed on the side front of the target assembly station on the motorcycle production line, with the lens facing the frame mounting holes, engine hanger holes, wheel hub shaft holes, and fixture positioning area. The robot end-effector camera is installed near the center of the robot end-effector tool, with the lens facing the area close to the target assembly bone point between the robot gripper end and the end-effector. The station camera is used to acquire overall two-dimensional images of the motorcycle parts during the assembly preparation stage, while the robot end-effector camera is used to acquire two-dimensional images of the local alignment positions during the assembly execution stage. Video acquisition is triggered by both the conveyor line arrival signal and the robot assembly start signal. Acquisition begins when both the conveyor line arrival signal and the robot assembly start signal are set to 1, with a frame rate of 30 frames per second and an acquisition duration of 2 seconds before assembly start to 3 seconds after assembly execution, forming a normal two-dimensional video stream including the static state before assembly and the dynamic state during assembly. By simultaneously setting a fixed viewing angle and an end-effector viewing angle, the overall posture of the motorcycle parts and the local alignment state of the robot end-effector can be preserved, providing a continuous image source for the extraction of two-dimensional assembly bone points.

[0107] The ordinary 2D video stream is read frame by frame in chronological order of acquisition time. Each frame read generates a 2D video frame record, which includes image data, acquisition camera marker, frame sequence marker, and acquisition time marker. 2D video frames acquired by the workstation camera are written to acquisition camera marker C1, and 2D video frames acquired by the robot's end effector camera are written to acquisition camera marker C2. The frame sequence marker starts from 1 and increments sequentially according to the acquisition order. The acquisition time marker uses a millisecond-level timestamp. Grayscale processing uses a weighted average of 0.299 for the red channel, 0.587 for the green channel, and 0.114 for the blue channel to generate a single-channel grayscale image. Image denoising uses a 3×3 median filter to remove isolated noise points. Scale unification scaling scales the 2D video frames to 1280×720 pixels, with the top-left corner of the image as the 2D coordinate origin. During sorting, the data is first grouped by the camera acquisition marker, then arranged from earliest to latest by the acquisition time marker. Two-dimensional video frames with the same acquisition time marker are arranged from smallest to largest by the frame order marker, thus obtaining the two-dimensional observation frame sequence of the workstation, enabling processing to be carried out under uniform size, uniform coordinates, and uniform time sequence.

[0108] In this embodiment, step two specifically involves:

[0109] Read each two-dimensional observation frame in the two-dimensional observation frame sequence of the workstation, perform edge extraction and contour segmentation on the motorcycle parts in the two-dimensional observation frames, and obtain the contour region of the motorcycle parts.

[0110] Extract the frame mounting hole center, engine hanger hole center, wheel hub shaft hole center, front fork end hole center, fixture positioning pin center, and robot end tool center from the motorcycle component outline area to serve as two-dimensional assembly bone points;

[0111] Configure bone point type labels according to the assembly position category corresponding to the two-dimensional assembly bone points, and configure two-dimensional coordinates according to the pixel position of the two-dimensional assembly bone points in the two-dimensional observation frame;

[0112] Read the frame sequence marker of the two-dimensional observation frame where the two-dimensional assembly bone point is located, and write the frame sequence marker to the corresponding two-dimensional assembly bone point;

[0113] Arrange the two-dimensional assembly bone points according to the frame order marker to generate a two-dimensional assembly bone point sequence;

[0114] In the specific implementation process, edge extraction was performed using the Canny edge detection method, with a low threshold set to 50 and a high threshold set to 150, used to extract the boundaries of the frame tube beam, mounting hole boundaries, and fixture positioning pin boundaries. Contour segmentation was performed using a combination of binarization segmentation and connected component analysis, with a binarization threshold set to 120. Regions with a connected component area less than 80 pixels were removed as noise regions, and the remaining connected components were used as the contour regions of the motorcycle components. For the center of the frame mounting hole, engine hanger hole, wheel hub axle hole, and front fork end hole, the center position was determined by contour circle fitting. When the ratio of the major and minor axes of the circular contour was between 0.75 and 1.25, the fitted circle center was used as the corresponding two-dimensional assembly bone point. For the center of the fixture positioning pin, the center of the minimum circumscribed circle of the outer contour of the positioning pin was extracted as the two-dimensional assembly bone point. For the center of the robot end tool, the center point of the end tool contour was extracted as the two-dimensional assembly bone point. The bone point type markers are written into the frame hole center marker, engine hanger hole center marker, wheel hub axle hole center marker, fork end hole center marker, fixture positioning pin center marker, and robot end tool center marker according to the assembly position category. The two-dimensional coordinates take the upper left corner of the two-dimensional observation frame as the origin, the pixel column number as the x-axis, and the pixel row number as the y-axis. The frame sequence marker is consistent with the frame sequence marker of the corresponding two-dimensional observation frame. Through the above processing, the contours of complex motorcycle parts can be converted into stable two-dimensional assembly bone point representations, reducing the interference of irrelevant edges, local reflections, and background structures on the two-dimensional bone point temporal analysis.

[0115] In this embodiment, step three specifically includes:

[0116] Read two-dimensional assembly bone points with the same bone point type in the two-dimensional assembly bone point sequence, extract two-dimensional coordinates according to frame order markers, and generate bone point temporal coordinate chain;

[0117] Based on the two-dimensional coordinates of adjacent frames in the bone point temporal coordinate chain, calculate the changes in the horizontal and vertical coordinates, and then sum the absolute values ​​of the changes in the horizontal and vertical coordinates to generate the bone point inter-frame residual amplitude.

[0118] Arrange the inter-frame residual magnitudes of bone points corresponding to the same bone point type label according to the frame order label, generate bone point residual rows, and stack multiple bone point residual rows according to the bone point type label order to construct a two-dimensional bone point residual matrix.

[0119] Temporal residual orchestration is performed on the two-dimensional bone point residual matrix. Temporal residual orchestration includes synchronous drift reference generation, isolated perturbation bit marking, and bearing coordinate update.

[0120] In the generation of synchronization drift reference, the inter-frame residual amplitudes of multiple bone points with differences not exceeding the set synchronization amplitude threshold under the same frame sequence mark are grouped into the same frame synchronization residual group. The changes in the horizontal and vertical coordinates of each two-dimensional assembled bone point in the same frame synchronization residual group are read, and the average values ​​are calculated to generate the synchronization drift reference value.

[0121] In the isolated perturbation bit marking, the inter-frame residual amplitude of the bone point corresponding to the current frame sequence mark in the same bone point residual row is compared with the inter-frame residual amplitude of the bone point corresponding to the previous frame sequence mark and the next frame sequence mark, respectively. When the absolute value of the two comparison differences is greater than the set perturbation difference threshold, the position corresponding to the current frame sequence mark is marked as an isolated perturbation bit.

[0122] In the coordinate update process, the horizontal average value of the synchronization drift reference value is subtracted from the horizontal average value of the horizontal coordinate of the two-dimensional assembly bone point under the corresponding frame sequence mark, and the vertical average value of the vertical coordinate is subtracted from the vertical average value of the synchronization drift reference value to generate the synchronization correction coordinate. The synchronization correction coordinate corresponding to the isolated disturbance bit is replaced with the average coordinate of the synchronization correction coordinate corresponding to the adjacent frame sequence mark to generate a two-dimensional stable bone point trajectory set.

[0123] In the specific implementation, the two-dimensional coordinates take the top left corner of the image as the origin, with the horizontal coordinate increasing along the column direction and the vertical coordinate increasing along the row direction. When calculating the changes in the horizontal and vertical coordinates, the horizontal coordinate change is obtained by subtracting the horizontal coordinate of the corresponding two-dimensional assembly bone point from the horizontal coordinate of the corresponding two-dimensional assembly bone point in the next frame sequence marker, and the vertical coordinate change is obtained by subtracting the vertical coordinate of the corresponding two-dimensional assembly bone point in the next frame sequence marker. Therefore, both the horizontal and vertical coordinate changes retain the direction of motion. The inter-frame residual amplitude of the bone point is only used to determine the strength of the change, and is formed by adding the absolute values ​​of the horizontal and vertical coordinate changes to avoid cancellation due to opposite directions. The synchronization amplitude threshold is set to 3 pixels. When the difference in the residual amplitude between multiple two-dimensional assembly bone points under the same frame sequence mark is no more than 3 pixels, it is determined to be a synchronization drift caused by slight shaking of the workstation camera, tooling vibration, or overall offset of the vehicle components. The changes in the horizontal and vertical coordinates in the synchronization residual group of the same frame are not taken as absolute values. Instead, the average values ​​are calculated and used as the horizontal average and vertical average values ​​in the synchronization drift reference value, thereby preserving the synchronization drift direction.

[0124] The perturbation difference threshold is further set to 6 pixels. When the absolute value of the difference between the inter-frame residual amplitude of the bone point corresponding to the current frame sequence mark and the comparison values ​​of the previous and next frame sequence marks is greater than 6 pixels, it is determined to be an isolated perturbation bit. In the coordinate update, the horizontal average value of the synchronization drift reference value is subtracted from the horizontal coordinate of the two-dimensional assembly bone point under the corresponding frame sequence mark, and the vertical average value of the synchronization drift reference value is subtracted from the vertical coordinate of the two-dimensional assembly bone point under the corresponding frame sequence mark to generate synchronization correction coordinates, thereby eliminating the common coordinate offset caused by camera jitter or overall tooling offset within the same frame. For the first and last frame sequence markers, isolated perturbation bit marking is not performed, and the synchronization correction coordinates are directly retained. For isolated perturbation bits of intermediate frame sequence markers, the synchronization correction coordinates corresponding to the previous frame sequence marker and the synchronization correction coordinates corresponding to the next frame sequence marker are averaged according to the horizontal and vertical coordinates, respectively, and the average coordinates are used to replace the synchronization correction coordinates corresponding to the isolated perturbation bits. This reduces single-frame anomalies caused by reflections, occluded edges, and false detections of local contours, so that the two-dimensional assembly bone points form a continuous and stable two-dimensional stable bone point trajectory set in the time direction.

[0125] In this embodiment, step four specifically includes:

[0126] Read the two-dimensional coordinates corresponding to the center of the robot end-effector tool and the target assembly bone point from the two-dimensional stable bone point trajectory set, and calculate the bone point spacing between the center of the robot end-effector tool and the target assembly bone point according to the frame sequence mark;

[0127] The distance change is calculated based on the bone point spacing corresponding to adjacent frame sequence marks, and an assembly distance change sequence is generated according to the frame sequence marks;

[0128] In the assembly distance change sequence, consecutive frame segments in which the bone point spacing continuously decreases and the distance change is greater than a set approach change threshold are marked as approach phase segments.

[0129] In the assembly distance change sequence, consecutive frame segments where the bone point spacing is less than the set alignment distance threshold and the distance change amount is less than the set alignment change threshold are marked as alignment phase segments.

[0130] In the assembly distance change sequence, consecutive frame segments where the bone point spacing is less than the set execution distance threshold and the two-dimensional coordinate change of the robot end tool center is less than the set execution stability threshold are marked as execution phase segments.

[0131] Use the start and end frame sequence marks of the near phase segment, aligned phase segment, and executed phase segment as slice boundaries;

[0132] The two-dimensional stable bone point trajectory set is assembled into phase slices according to the slice boundary. The coordinates of the two-dimensional assembly bone points within the corresponding frame sequence range are extracted from the two-dimensional stable bone point trajectory set and written into the proximity phase marker, alignment phase marker and execution phase marker respectively to generate a two-dimensional assembly phase segment set.

[0133] In the specific implementation process, the target assembly bone point is determined according to the current assembly task. In the engine hoisting and positioning task, the target assembly bone point is the center of the engine lifting lug hole; in the wheel hub assembly task, the target assembly bone point is the center of the wheel hub shaft hole; and in the fork assembly task, the target assembly bone point is the center of the fork end hole. The bone point spacing is represented by the two-dimensional pixel distance between the center of the robot end tool and the target assembly bone point. A proximity change threshold of 5 pixels is set. When the bone point spacing decreases frame by frame for no less than 5 consecutive frames and the distance change is greater than 5 pixels, it is determined to be a proximity phase segment. An alignment distance threshold of 25 pixels and an alignment change threshold of 3 pixels are set. When the bone point spacing is less than 25 pixels and the distance change is less than 3 pixels for no less than 4 consecutive frames, it is determined to be an alignment phase segment. An execution distance threshold of 12 pixels and an execution stability threshold of 2 pixels are set. When the bone point spacing is less than 12 pixels and the two-dimensional coordinate change of the robot end tool center is less than 2 pixels for no less than 3 consecutive frames, it is determined to be an execution phase segment. By performing the above slicing process, the robot approach, assembly alignment, and end-effector stabilization processes are expressed separately, enabling the two-dimensional positioning connection diagram to establish corresponding connection relationships according to different assembly stages. This avoids mixing the rapid approach stage and the fine alignment stage in the modeling, thereby improving the stability of the three-dimensional positioning guidance results.

[0134] In this embodiment, step five specifically includes:

[0135] Read the coordinates of two-dimensional assembly bone points, bone point type markers, phase markers and frame sequence markers in the two-dimensional assembly phase segment set, and read the hole position correspondence, fixture positioning relationship and robot end effector execution relationship in the motorcycle assembly process relationship;

[0136] Based on the hole correspondence, hole bearing edges are established between the frame mounting hole center and the engine hanger hole center, and between the wheel hub axle hole center and the front fork end hole center;

[0137] Based on the fixture positioning relationship, a tooling positioning support edge is established between the center of the fixture positioning pin and the center of the frame mounting hole;

[0138] Based on the robot's end-effector execution relationship, an end-effector execution receiving edge is established between the robot's end-effector tool center and the target assembly bone point;

[0139] Under the same frame sequence mark, the two-dimensional assembly bone point is used as the graph node, and the hole position receiving edge, tooling positioning receiving edge and end execution receiving edge are used as the two-dimensional assembly receiving edge to construct the two-dimensional positioning receiving graph under the corresponding frame sequence mark.

[0140] The two-dimensional coordinates of the fixture positioning pin center during the assembly preparation stage are read as the tooling reference coordinates, and the two-dimensional coordinates of the fixture positioning pin center under the frame sequence mark in the approach phase segment, alignment phase segment and execution phase segment are read as the tooling current coordinates.

[0141] Subtract the horizontal coordinate of the tooling reference coordinate from the horizontal coordinate of the current tooling coordinate to obtain the horizontal tooling drift amount; subtract the vertical coordinate of the tooling reference coordinate from the vertical coordinate of the current tooling coordinate to obtain the vertical tooling drift amount.

[0142] Under the same frame sequence mark, subtract the horizontal tooling drift from the horizontal coordinate of each two-dimensional assembly bone point coordinate in the two-dimensional positioning and connection diagram, and subtract the vertical tooling drift from the vertical coordinate of each two-dimensional assembly bone point coordinate to generate a drift-compensated two-dimensional positioning and connection diagram.

[0143] Following the phase order of approach phase segment, alignment phase segment, and execution phase segment, read the drift-compensated 2D positioning and connection map corresponding to each frame sequence mark in each phase segment, and arrange multiple drift-compensated 2D positioning and connection maps in sequence according to the frame sequence mark to generate a 2D positioning and connection sequence.

[0144] In the specific implementation process, the motorcycle assembly process relationships are pre-written into the process configuration table. The hole correspondence in the process configuration table includes the assembly correspondence between the frame mounting hole center and the engine hanger hole center, and the assembly correspondence between the wheel hub shaft hole center and the front fork end hole center; the fixture positioning relationship includes the positioning constraint relationship between the fixture positioning pin center and the frame mounting hole center; the robot end effector relationship includes the approach, alignment, and execution relationship between the robot end tool center and the target assembly bone point. A two-dimensional positioning connection diagram is formed under each frame sequence mark. The diagram nodes record the coordinates of the two-dimensional assembly bone point, the bone point type mark, and the phase mark. The two-dimensional assembly connection edge records the connection objects of the hole connection edge, the tooling positioning connection edge, and the end effector execution connection edge, so that the single frame diagram structure can simultaneously express the hole assembly relationship, fixture positioning relationship, and robot end effector execution relationship of motorcycle components.

[0145] Furthermore, the average of the two-dimensional coordinates of the fixture positioning pin center in the first 5 frames of the assembly preparation stage is taken as the tooling reference coordinates, which can reduce the impact of single-frame detection fluctuations on the reference position. In the approach phase segment, alignment phase segment, and execution phase segment, the two-dimensional coordinates of the fixture positioning pin center are read frame by frame as the current coordinates of the tooling, and the lateral and longitudinal tooling drift amounts are calculated. During drift compensation, the coordinates of all two-dimensional assembly bone points in the two-dimensional positioning connection diagram are corrected in the same direction under the same frame sequence mark, so that the center of the frame mounting hole, the center of the engine hanger hole, the center of the wheel hub shaft hole, the center of the fork end hole, and the center of the robot end tool are all expressed under the same tooling reference. Through the above processing, the overall coordinate deviation caused by slight offset of the production line fixture, workpiece placement deviation, and slight camera angle jitter can be separated from the assembly bone point coordinates, so that the two-dimensional positioning connection sequence input by the improved PoseFormer model has both temporal continuity, process connection, and tooling reference consistency.

[0146] In this embodiment, the improved PoseFormer model specifically includes a two-dimensional bone point embedding layer, a receiving spatial coding layer, a phase temporal coding layer, an assembly drift reflection layer, and a three-dimensional positioning output layer. A two-dimensional receiving anchor return operator is set in the assembly drift reflection layer.

[0147] The two-dimensional positioning sequence is input into the two-dimensional bone point embedding layer. The two-dimensional bone point embedding layer performs linear mapping on the coordinates of the two-dimensional assembly bone points, and adds the linear mapping result with the bone point type label embedding result and the phase label embedding result to form the two-dimensional bone point embedding feature.

[0148] The two-dimensional bone point embedding features are input into the receiving spatial coding layer. The receiving spatial coding layer performs query mapping, key mapping, and value mapping on the two-dimensional bone point embedding features under the same frame order label. The query mapping results and key mapping results are multiplied one by one and then summed to generate the bone point spatial attention score.

[0149] The offsets of the bearing edges corresponding to the hole bearing edge, tooling positioning bearing edge, and end execution bearing edge are added to the bone point space attention score. After normalization by the Softmax function, the score is multiplied by the value mapping result and summed to output the bone point features in the bearing space.

[0150] The spatial bone point features are input into the phase-temporal coding layer. The phase-temporal coding layer reads the spatial bone point features of the same bone point type in the near phase segment, aligned phase segment and executed phase segment according to the frame sequence label. The corresponding features of adjacent frame sequence labels are multiplied one by one and then summed to generate a temporal attention score.

[0151] The phase progressive bias from the near phase segment to the aligned phase segment and from the aligned phase segment to the executed phase segment is added to the temporal attention score. After normalization by the Softmax function, the spatial bone point features under different frame order markers are weighted and summed to output the phase temporal bone point features.

[0152] The phase-time bone point features are input into the assembly drift reflection layer. The assembly drift reflection layer reads the connection relationship between the fixture positioning pin center, the frame mounting hole center, the robot end tool center and the target assembly bone point through the two-dimensional anchor return operator. The anchor return convergence features of the two-dimensional anchor point are written back to the phase-time bone point features corresponding to the target assembly bone point, and the assembly drift reflection features are output.

[0153] The assembly drift reflection features are input into the 3D localization output layer. The 3D localization output layer sequentially performs fully connected mapping, layer normalization processing and GELU activation processing to output the robot's 3D localization guidance results. The robot's 3D localization guidance results include the 3D coordinates of the target assembly point and the positioning confidence.

[0154] In the specific implementation process, the hidden feature dimension of the improved PoseFormer model was set to 128 dimensions, and the coordinates of the two-dimensional assembly bone points were normalized according to the two-dimensional observation frame size, where the horizontal coordinate was divided by 1280 and the vertical coordinate was divided by 720, so that the two-dimensional coordinates collected by the workstation camera and the robot end-effector camera fell into a uniform scale range. In the two-dimensional bone point embedding layer, the normalized horizontal coordinate is multiplied term by term by the preset horizontal coordinate weight vector, and the normalized vertical coordinate is multiplied term by term by the preset vertical coordinate weight vector. The two multiplication results are then added to the coordinate bias vector to obtain a 128-dimensional linear mapping result. Bone point type markers include frame mounting hole center, engine hanger hole center, wheel hub axle hole center, fork end hole center, clamp positioning pin center, and robot end tool center. Each type of bone point type marker corresponds to a 128-dimensional bone point type marker embedding result. Furthermore, phase markers include proximity phase markers, alignment phase markers, and execution phase markers. Each type of phase marker corresponds to a 128-dimensional phase marker embedding result. The 128-dimensional linear mapping result, the 128-dimensional bone point type marker embedding result, and the 128-dimensional phase marker embedding result are added element by element to generate a 128-dimensional two-dimensional bone point embedding feature. This allows each two-dimensional assembly bone point to simultaneously carry coordinate position, assembly object category, and assembly stage information, avoiding the problem that the standard PoseFormer relies solely on two-dimensional coordinate sequences, resulting in insufficient expression of hole position relationships and assembly phase relationships.

[0155] The spatial coding layer employs four attention heads, each with a feature dimension of 32. This ensures that the combined four attention heads align with the 128-dimensional two-dimensional bone point embedding feature, effectively representing hole acceptance, tooling positioning, end effector execution, and phase correlation features without adding extra dimensions that could cause input mismatch to the subsequent phase-temporal coding layer. The hole acceptance edge offset is set to 0.35 to highlight the assembly correspondence between the frame mounting hole center and the engine hanger hole center, and between the wheel hub axle hole center and the fork end hole center. The tooling positioning edge offset is set to 0.25 to retain the reference traction effect of the fixture positioning pin center on the frame mounting hole center, while preventing excessive pressure from tooling nodes on the target assembly bone point. The end effector execution edge offset is set to 0.45 to strengthen the proximity and execution relationship between the robot's end effector tool center and the target assembly bone point. The temporal window of the phase-temporal encoding layer is set to 4 frames before and after. The phase progressive bias from the approach phase segment to the alignment phase segment is set to 0.30 to enhance the continuous features from rapid approach to fine alignment. The phase progressive bias from the alignment phase segment to the execution phase segment is set to 0.40 to enhance the small stable changes before and after the final assembly execution. The temporal attention score between non-adjacent phases is reduced by 0.50 to suppress cross-stage jumps between the approach phase segment and the execution phase segment. The assembly drift reflection feature output by the assembly drift reflection layer is maintained at 128 dimensions. The 3D localization output layer maps the 128-dimensional assembly drift reflection feature to a 4-dimensional output, where the first three dimensions are the 3D coordinates of the target assembly point, and the last dimension is compressed into a positional confidence using the Sigmoid function, thereby improving the stability of the robot's 3D localization guidance results under scenarios of fixture offset, part placement deviation, and assembly stage switching.

[0156] In this embodiment, the two-dimensional anchor return operator is specifically as follows:

[0157] The center of the fixture positioning pin, the center of the frame mounting hole, and the center of the robot end tool are determined as two-dimensional receiving anchor points, and the target assembly bone point is determined as the anchor return target bone point. The phase time bone point features corresponding to the two-dimensional receiving anchor point and the anchor return target bone point are read respectively.

[0158] Read the two-dimensional coordinates of the anchor point under the current frame sequence mark and the two-dimensional coordinates under the previous frame sequence mark. Subtract the horizontal coordinate of the previous frame sequence mark from the horizontal coordinate of the current frame sequence mark to generate the horizontal displacement of the anchor point. Subtract the vertical coordinate of the previous frame sequence mark from the vertical coordinate of the current frame sequence mark to generate the vertical displacement of the anchor point. Set both the horizontal and vertical displacement of the anchor point corresponding to the first frame sequence mark to zero.

[0159] The lateral and longitudinal displacements of the anchor points are concatenated to form the anchor point displacement vector, and the anchor point displacement vector is linearly mapped to generate the anchor point offset feature.

[0160] The anchor point offset feature is added to the phase time bone point feature corresponding to the two-dimensional anchor point to generate the anchor point acceptance feature.

[0161] Search for the two-dimensional bonding path from the bonding anchor point to the anchor return target bone point along the bonding edge of the hole, the bonding edge of the tooling positioning, and the bonding edge of the end execution. Count the number of two-dimensional assembly bonding edges traversed by the bonding path to obtain the bonding jump count.

[0162] The anchor point acceptance feature is multiplied by the phase time bone point feature corresponding to the anchor return target bone point, and then summed to generate the anchor return correlation score.

[0163] Multiply the number of hops received by the set hop attenuation coefficient to generate the path attenuation score, and subtract the path attenuation score from the anchor return related score to generate the anchor return modulation score;

[0164] The anchor return modulation scores corresponding to multiple two-dimensional anchor points are input into the Softmax function to obtain the anchor return weights. The anchor point acceptance features corresponding to each two-dimensional anchor point are multiplied by the corresponding anchor return weights and then summed to generate the anchor return convergence features.

[0165] The anchor return convergence feature is input into the Sigmoid function to obtain the anchor return gate value. The anchor return convergence feature and the anchor return gate value are multiplied element by element and then superimposed onto the phase time bone point feature corresponding to the anchor return target bone point to output the assembly drift reflection feature.

[0166] In the specific implementation process, the phase-time bone point feature dimension received by the two-dimensional anchoring and return operator is set to 128 dimensions. The lateral displacement and longitudinal displacement of the anchor point are concatenated to form a two-dimensional anchor point displacement vector, which is then converted into a 128-dimensional anchor point offset feature through linear mapping, so that the two-dimensional coordinate displacement can be added to the phase-time bone point feature in the same feature space. Prioritized, the jump count attenuation coefficient is set to 0.20. When the two-dimensional anchoring point and the target bone point pass through one two-dimensional assembly anchoring edge, the path attenuation score is 0.20; when they pass through two two-dimensional assembly anchoring edges, the path attenuation score is 0.40. This allows two-dimensional anchoring points that are closer to the target assembly bone point and have a more direct anchoring relationship to receive higher anchoring weights. The fixture positioning pin center is used to express the tooling reference drift, the frame mounting hole center is used to express the motorcycle component mounting reference, and the robot end-effector tool center is used to express the approach state of the execution end. The three types of two-dimensional anchoring points participate in the anchoring convergence, which can avoid the model performing three-dimensional positioning inference based solely on the single-point change of the target assembly bone point. The anchor return gating value is limited to between 0 and 1 by the Sigmoid function. This controls the intensity of the anchor return convergence feature superimposed on the anchor return target bone point, preventing the anchor point drift information from excessively covering the original phase time bone point features. Through the above improvements, the assembly drift reflection feature can simultaneously retain the assembly phase continuity, tooling reference changes, and robot end effector execution trends, improving the stability of the target assembly point's 3D coordinates and positional reliability output.

[0167] The improved PoseFormer model is built upon the standard PoseFormer model's framework of 2D bone point input, spatial encoding, temporal encoding, and 3D bone point output. It retains the standard PoseFormer's processing logic of first modeling spatial relationships within a single frame and then modeling temporal relationships across frames, while replacing the input with a sequence of ordinary human 2D bone points with the 2D positioning and connection sequences generated by this invention. Compared to the standard PoseFormer model, this invention sets up a 2D bone point embedding layer at the input end, uniformly encoding the 2D assembly bone point coordinates, bone point type markers, and phase markers; in spatial modeling, it sets up a connection spatial encoding layer, transforming hole connection edges, tooling positioning connection edges, and end-effector execution connection edges into connection edge offsets, so that the model no longer relies solely on the geometric distance between bone points; in temporal modeling, it sets up a phase temporal encoding layer, introducing temporal attention to the assembly progression relationships of proximity phase segments, alignment phase segments, and execution phase segments; and before output, it sets up an assembly drift reflection layer, and uses a 2D connection anchor return operator to write back the anchor return convergence features of the fixture positioning pin center, frame mounting hole center, and robot end-effector tool center to the target assembly bone point. The improved PoseFormer model no longer only learns the ordinary mapping relationship from two-dimensional bone points to three-dimensional coordinates, but also incorporates hole acceptance, tooling positioning, robot end effector execution, and assembly phase progression into spatiotemporal attention calculation. This enables the model to distinguish between overall tooling drift, actual component displacement, and end effector approach motion, thereby reducing the transmission of two-dimensional detection errors to three-dimensional positioning results and improving the stability of the three-dimensional coordinates and positioning reliability of the target assembly point.

[0168] During model training, training data was derived from ordinary 2D video streams collected from the target assembly station on the motorcycle production line and robot assembly execution records. The ordinary 2D video streams were processed in the aforementioned steps to generate 2D positioning sequences. The actual spatial coordinates of the assembly points in the robot assembly execution records were used as the 3D coordinate labels for the target assembly points. Samples with completed assembly and an error of no more than 2mm were labeled as high-confidence samples, while samples with assembly deviations greater than 2mm were labeled as low-confidence samples. The training, validation, and test sets were divided in a 7:2:1 ratio, with a batch size of 32, an initial learning rate of 0.0001, and 120 training epochs. The AdamW optimizer was used, with a weight decay of 0.01. The loss function consisted of the sum of the 3D coordinate error loss and the confidence error loss. The 3D coordinate error loss was calculated by averaging the absolute values ​​of the differences between the predicted and labeled 3D coordinates in the horizontal, vertical, and height directions. The confidence error loss was calculated as the binary cross-entropy loss between the predicted and labeled confidence scores. The model was considered converged when the total loss on the validation set decreased by less than 0.001 over 10 consecutive epochs. This training method enables the improved PoseFormer model to learn the three-dimensional positioning mapping relationship of assembly points under ordinary two-dimensional vision conditions, and reduces the impact of fixture offset, phase switching and local bone point false detection on the positioning results through the connection relationship and anchor return mechanism.

[0169] In this embodiment, step seven specifically includes:

[0170] Read the three-dimensional coordinates and positional confidence of the target assembly point from the robot's three-dimensional positioning guidance results;

[0171] When the positioning confidence is greater than or equal to the set execution confidence threshold, the current position of the robot end-tool center is obtained, and the three-dimensional coordinates of the target assembly point are subtracted from the current position of the robot end-tool center to generate the end-positioning offset.

[0172] The position of the robot end-effector center in the robot motion trajectory is corrected according to the end-effector positioning offset, and the robot end-effector center is controlled to move towards the target assembly point according to the corrected robot motion trajectory.

[0173] When the positioning confidence level is less than the set execution confidence level threshold, stop the movement of the robot's end effector center and output a repositioning command;

[0174] Based on the robot end-effector center position, target assembly point 3D coordinates, positioning confidence, and execution status before and after robot motion trajectory correction, a production line positioning execution record is generated.

[0175] In the specific implementation process, the execution confidence threshold is set to 0.85. The robot controller obtains the current position of the robot end-effector center in real time through the robot joint encoder and the calibrated tool coordinate system. The three-dimensional coordinates of the target assembly point and the current position of the robot end-effector center are both represented using the same robot base coordinate system. The end-effector positioning offset includes lateral offset, longitudinal offset, and height offset. The robot controller superimposes the end-effector positioning offset onto the robot end-effector center position of the corresponding interpolation point in the original robot motion trajectory, and regenerates the end-effector movement path using linear interpolation. The interpolation period is set to 10 milliseconds, and the maximum displacement for a single correction is set to 8mm. When it exceeds 8mm, it is divided into multiple consecutive corrections to avoid sudden jumps in the robot end-effector center. When the positioning confidence is less than 0.85, the repositioning command includes stopping the current movement of the robot end-effector center, re-acquiring the ordinary two-dimensional video stream, regenerating the two-dimensional positioning sequence, and re-executing the three-dimensional positioning guidance. The production line positioning execution record includes the workstation number, frame sequence mark, center position of the robot end effector before and after correction, three-dimensional coordinates of the target assembly point, positioning reliability, and execution status. This facilitates tracing the source of positioning errors and improves positioning safety and execution stability during motorcycle assembly.

[0176] refer to Figure 4 A robot vision positioning system for a motorcycle production line includes the following modules:

[0177] The video preprocessing module is used to acquire ordinary two-dimensional video streams of the target assembly station on the motorcycle production line, perform frame-by-frame splitting and image preprocessing on the ordinary two-dimensional video streams, and generate a sequence of two-dimensional observation frames for the station.

[0178] The two-dimensional assembly bone point extraction module is used to extract two-dimensional assembly bone points from the two-dimensional observation frame sequence of the workstation, and to configure bone point type markers, two-dimensional coordinates and frame sequence markers for the two-dimensional assembly bone points to generate a two-dimensional assembly bone point sequence.

[0179] The temporal residual orchestration module is used to construct a two-dimensional bone point residual matrix based on the two-dimensional assembly bone point sequence, and perform temporal residual orchestration processing to generate a two-dimensional stable bone point trajectory set.

[0180] The assembly phase slicing module is used to slice the two-dimensional stable bone point trajectory set into assembly phase slices based on the distance change between the center of the robot end tool and the target assembly bone point, and generate a two-dimensional assembly phase segment set.

[0181] The connection diagram construction and compensation module is used to establish two-dimensional assembly connection edges based on the two-dimensional assembly phase segment set and the motorcycle assembly process relationship, construct a two-dimensional positioning connection diagram and perform drift compensation to generate a two-dimensional positioning connection sequence.

[0182] The 3D positioning guidance module is used to input the 2D positioning sequence into the improved PoseFormer model, and to perform anchor return association between the 2D anchor point and the target assembly bone point through the 2D anchor return operator, and output the robot's 3D positioning guidance result.

[0183] The trajectory correction and recording module is used to correct the robot's motion trajectory based on the robot's 3D positioning guidance results and generate production line positioning execution records.

[0184] Example 1: To verify the feasibility of this invention in practice, it was applied to the engine lifting and positioning station of a motorcycle assembly line. This station is used to align the engine lifting lug hole with the frame mounting hole. A six-axis industrial robot, an engine clamping end effector, a frame positioning fixture, a station camera, and a robot end effector camera are configured on-site. The station camera is installed in front of the assembly station, covering the frame mounting hole, engine lifting lug hole, fixture positioning pin, and the main outline of the frame. The robot end effector camera is installed near the center of the robot end effector, covering the area where the center of the robot end effector and the engine lifting lug hole are close together. Because the motorcycle frame still has a slight positioning deviation after being placed on the conveyor line, and the fixture positioning pin is affected by vibration, resulting in a slight offset of 1mm to 3mm, and there is metallic reflection and partial obstruction near the engine lifting lug hole, traditional positioning methods based on single-frame image hole center recognition easily mistake false detection points for assembly targets, causing trajectory jitter or repeated corrections during the final alignment of the robot end effector.

[0185] In this embodiment, ordinary two-dimensional video streams are continuously acquired from 2 seconds before robot assembly starts to 3 seconds after assembly execution, with a frame rate of 30 frames per second. The ordinary two-dimensional video streams are then processed frame-by-frame splitting, grayscale conversion, 3×3 median filtering, and 1280×720 pixel scaling to generate a two-dimensional observation frame sequence for the workstation. The system extracts the center of the frame mounting hole, engine hanger hole, wheel hub axle hole, front fork end hole, fixture positioning pin, and robot end-effector from this sequence to generate a two-dimensional assembly bone point sequence. To address bone point jumps caused by camera shake, reflections, and partial occlusion, a two-dimensional bone point residual matrix is ​​constructed and temporal residual orchestration is performed to obtain a stable two-dimensional bone point trajectory set. Subsequently, based on the bone point spacing changes between the robot end-effector center and the engine hanger hole center, proximity phase segments, alignment phase segments, and execution phase segments are defined. A two-dimensional positioning and connection map is constructed by combining the hole receiving edge, fixture positioning receiving edge, and end-effector execution receiving edge. The system uses the average coordinates of the center of the fixture positioning pin in the first 5 frames as the reference coordinates of the tooling, performs drift compensation on each subsequent frame, generates a two-dimensional positioning sequence and inputs it into the improved PoseFormer model to obtain the three-dimensional coordinates and positioning confidence of the target assembly point, which are used to correct the robot's motion trajectory.

[0186] To verify the practical effect of this invention, a comparative test was conducted on the single-frame circle fitting two-dimensional positioning method, the standard PoseFormer method, and the method of this invention at the engine hoisting and positioning station on the same motorcycle assembly line. The single-frame circle fitting two-dimensional positioning method uses the center of the frame mounting hole and the center of the engine lifting lug hole in the two-dimensional observation frame as the main positioning objects, directly determining the coordinates of the hole center through contour circle fitting, and guiding the robot's motion trajectory correction accordingly. The standard PoseFormer method takes the continuous frame two-dimensional assembly bone point sequence as input, and outputs the three-dimensional coordinates of the target assembly point through conventional spatial encoding and temporal encoding. The method of this invention further performs temporal residual arrangement, assembly phase slicing, two-dimensional positioning connection map construction, and drift compensation on the basis of the two-dimensional assembly bone point sequence, and achieves three-dimensional positioning guidance by improving the two-dimensional connection anchor return operator in the PoseFormer model. During the test, each method completed 100 engine hoisting and positioning tasks, and the average positioning error, maximum positioning error, and trajectory correction success rate were statistically analyzed, resulting in the results shown in Table 1.

[0187]

[0188] As shown in Table 1 above, the average positioning error of the single-frame circle fitting 2D positioning method is 4.8 mm, and the maximum positioning error reaches 8.9 mm. This indicates that relying solely on the hole center contour fitting result in a single frame image is easily affected by metal reflection, local occlusion, camera shake, and fixture offset, leading to instability in the correction amount of the robot's end-effector center. The standard PoseFormer method uses continuous frame 2D bone points for spatiotemporal modeling, reducing the average positioning error to 3.1 mm and the maximum positioning error to 5.7 mm. However, due to the lack of hole position receiving edges, tooling positioning receiving edges, and end-effector execution receiving edges, it is still difficult to accurately distinguish between the actual displacement of the component and the overall tooling drift. Furthermore, the method of this invention reduces the average positioning error to 1.6 mm and the maximum positioning error to 2.8 mm, indicating that the 2D positioning receiving sequence and the 2D receiving anchor return operator can improve the stability of the 3D coordinate output of the target assembly point. The trajectory correction success rate is determined by the proportion of tests in which the robot motion trajectory correction is successfully completed and the final assembly error is less than or equal to 2 mm out of the total number of tests. In 100 tests, the single-frame circle fitting two-dimensional positioning method met the above conditions 86 times, so the trajectory correction success rate was 86.0%. The standard PoseFormer method, converted according to the test results, had a success rate of 91.0%, while the method of this invention, converted according to the test results, had a success rate of 98.0%, indicating that the present invention has higher assembly execution reliability.

[0189] As can be seen from this embodiment, the present invention is applicable to actual assembly scenarios such as engine hoisting and positioning in motorcycle production lines, and can effectively address problems existing in ordinary two-dimensional visual positioning, such as camera shake, metal reflection, partial occlusion, fixture offset, and frame placement deviation. By extracting two-dimensional assembly bone points, performing temporal residual arrangement, dividing assembly phases, constructing a two-dimensional positioning connection map, and performing drift compensation, the system can obtain a more stable two-dimensional positioning connection sequence. Furthermore, by improving the PoseFormer model and the two-dimensional connection anchor return operator, the tooling datum, hole connection relationship, and robot end effector state are used together for three-dimensional positioning guidance. This verifies the technical effect of the present invention in improving the stability of robot visual positioning, reducing positioning errors, and improving trajectory correction reliability under ordinary two-dimensional video stream conditions, demonstrating good engineering feasibility.

[0190] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A robot vision positioning method for a motorcycle production line, characterized in that, Includes the following steps: Step 1: Obtain the ordinary two-dimensional video stream of the target assembly station on the motorcycle production line, and perform frame-by-frame splitting and image preprocessing on the ordinary two-dimensional video stream to generate a two-dimensional observation frame sequence of the station. Step 2: Extract two-dimensional assembly bone points from the two-dimensional observation frame sequence of the workstation, and configure bone point type markers, two-dimensional coordinates and frame sequence markers for the two-dimensional assembly bone points to generate a two-dimensional assembly bone point sequence; Step 3: Construct a two-dimensional bone point residual matrix based on the two-dimensional assembly bone point sequence, and perform temporal residual arrangement processing to generate a two-dimensional stable bone point trajectory set; Step 4: Based on the change in distance between the center of the robot's end-effector and the target assembly bone point, perform assembly phase slicing on the two-dimensional stable bone point trajectory set to generate a two-dimensional assembly phase segment set; Step 5: Based on the two-dimensional assembly phase segment set and the motorcycle assembly process relationship, establish two-dimensional assembly acceptance edges, construct a two-dimensional positioning acceptance diagram and perform drift compensation to generate a two-dimensional positioning acceptance sequence. Step 6: Input the two-dimensional positioning and acceptance sequence into the improved PoseFormer model, and use the two-dimensional acceptance anchor return operator to perform anchor return association between the two-dimensional acceptance anchor point and the target assembly bone point, and output the robot's three-dimensional positioning guidance result; Step 7: Correct the robot's motion trajectory based on the robot's 3D positioning guidance results, and generate a production line positioning execution record.

2. The robot vision positioning method for a motorcycle production line according to claim 1, characterized in that, Step one specifically involves: Workstation cameras and robot end-effector cameras are set up at the target assembly station on the motorcycle production line to capture two-dimensional images of motorcycle parts during assembly preparation and assembly execution, and generate ordinary two-dimensional video streams. The ordinary two-dimensional video stream is split frame by frame according to the acquisition time sequence to obtain a set of two-dimensional video frames, and the acquisition camera mark, frame order mark and acquisition time mark are written into the two-dimensional video frames. The set of two-dimensional video frames is preprocessed, including grayscale conversion, image denoising, and scale unification, to generate a preprocessed set of two-dimensional video frames. The preprocessed two-dimensional video frame set is sorted according to the acquisition camera marker, acquisition time marker, and frame sequence marker to generate a workstation two-dimensional observation frame sequence.

3. The robot vision positioning method for a motorcycle production line according to claim 1, characterized in that, Step two specifically involves: Read each two-dimensional observation frame in the two-dimensional observation frame sequence of the workstation, perform edge extraction and contour segmentation on the motorcycle parts in the two-dimensional observation frames, and obtain the contour region of the motorcycle parts. Extract the frame mounting hole center, engine hanger hole center, wheel hub shaft hole center, front fork end hole center, fixture positioning pin center and robot end tool center from the outline area of ​​the motorcycle component, as two-dimensional assembly rib points; Configure bone point type markers according to the assembly position category corresponding to the two-dimensional assembly bone points, and configure two-dimensional coordinates according to the pixel positions of the two-dimensional assembly bone points in the two-dimensional observation frame. Read the frame sequence marker of the two-dimensional observation frame where the two-dimensional assembly bone point is located, and write the frame sequence marker into the corresponding two-dimensional assembly bone point; The two-dimensional assembly bone points are arranged according to the frame sequence marker to generate a two-dimensional assembly bone point sequence.

4. The robot vision positioning method for a motorcycle production line according to claim 1, characterized in that, Step three specifically involves: Read the two-dimensional assembly bone points with the same bone point type in the two-dimensional assembly bone point sequence, extract the two-dimensional coordinates according to the frame order mark, and generate a bone point temporal coordinate chain. Based on the two-dimensional coordinates of adjacent frames in the bone point temporal coordinate chain, calculate the changes in the horizontal coordinate and the vertical coordinate, and then sum the absolute values ​​of the changes in the horizontal coordinate and the vertical coordinate to generate the bone point frame residual amplitude. Arrange the inter-frame residual magnitudes of bone points corresponding to the same bone point type label according to the frame order label, generate bone point residual rows, and stack multiple bone point residual rows according to the bone point type label order to construct a two-dimensional bone point residual matrix. The two-dimensional bone point residual matrix is ​​subjected to temporal residual orchestration processing, which includes synchronous drift reference generation, isolated perturbation bit marking, and continuity coordinate updating. In the generation of synchronization drift reference, the inter-frame residual amplitudes of multiple bone points with differences not exceeding the set synchronization amplitude threshold under the same frame sequence mark are grouped into the same frame synchronization residual group. The changes in the horizontal and vertical coordinates of each two-dimensional assembled bone point in the same frame synchronization residual group are read, and the average values ​​are calculated to generate the synchronization drift reference value. In the isolated perturbation bit marking, the inter-frame residual amplitude of the bone point corresponding to the current frame sequence mark in the same bone point residual row is compared with the inter-frame residual amplitude of the bone point corresponding to the previous frame sequence mark and the next frame sequence mark, respectively. When the absolute value of the two comparison differences is greater than the set perturbation difference threshold, the position corresponding to the current frame sequence mark is marked as an isolated perturbation bit. In the coordinate update process, the horizontal average value of the synchronization drift reference value is subtracted from the horizontal average value of the horizontal coordinate of the two-dimensional assembly bone point under the corresponding frame sequence mark, and the vertical average value of the vertical coordinate is subtracted from the vertical average value of the synchronization drift reference value to generate synchronization correction coordinates. The synchronization correction coordinates corresponding to isolated disturbance bits are replaced with the average coordinates of the synchronization correction coordinates corresponding to adjacent frame sequence marks to generate a two-dimensional stable bone point trajectory set.

5. The robot vision positioning method for a motorcycle production line according to claim 1, characterized in that, Step four specifically involves: Read the two-dimensional coordinates corresponding to the center of the robot end-effector tool and the target assembly bone point in the two-dimensional stable bone point trajectory set, and calculate the bone point spacing between the center of the robot end-effector tool and the target assembly bone point according to the frame sequence mark; The distance change is calculated based on the bone point spacing corresponding to adjacent frame sequence marks, and an assembly distance change sequence is generated according to the frame sequence marks; In the assembly distance change sequence, consecutive frame segments in which the bone point spacing continuously decreases and the distance change is greater than a set approach change threshold are marked as approach phase segments. In the assembly distance change sequence, consecutive frame segments where the bone point spacing is less than the set alignment distance threshold and the distance change amount is less than the set alignment change threshold are marked as alignment phase segments. In the assembly distance change sequence, consecutive frame segments where the bone point spacing is less than the set execution distance threshold and the two-dimensional coordinate change of the robot end tool center is less than the set execution stability threshold are marked as execution phase segments. Use the start and end frame sequence marks of the near phase segment, aligned phase segment, and executed phase segment as slice boundaries; The two-dimensional stable bone point trajectory set is assembled into phase slices according to the slice boundary. The coordinates of the two-dimensional assembly bone points within the corresponding frame sequence range are extracted from the two-dimensional stable bone point trajectory set and written into the proximity phase marker, alignment phase marker and execution phase marker respectively to generate a two-dimensional assembly phase segment set.

6. The robot vision positioning method for a motorcycle production line according to claim 1, characterized in that, Step five specifically involves: Read the coordinates of the two-dimensional assembly bone points, the bone point type markers, the phase markers, and the frame sequence markers in the two-dimensional assembly phase segment set, and read the hole position correspondence, fixture positioning relationship, and robot end effector relationship in the motorcycle assembly process relationship; Based on the hole correspondence, hole receiving edges are established between the frame mounting hole center and the engine hanger hole center, and between the wheel hub shaft hole center and the front fork end hole center; Based on the fixture positioning relationship, a tooling positioning support edge is established between the center of the fixture positioning pin and the center of the frame mounting hole; Based on the robot end-effector execution relationship, an end-effector execution receiving edge is established between the robot end-effector tool center and the target assembly bone point; Under the same frame sequence mark, the two-dimensional assembly bone point is used as the graph node, and the hole position receiving edge, tooling positioning receiving edge and end execution receiving edge are used as the two-dimensional assembly receiving edge to construct the two-dimensional positioning receiving graph under the corresponding frame sequence mark. The two-dimensional coordinates of the fixture positioning pin center during the assembly preparation stage are read as the tooling reference coordinates, and the two-dimensional coordinates of the fixture positioning pin center under the frame sequence mark in the approach phase segment, alignment phase segment and execution phase segment are read as the tooling current coordinates. Subtract the horizontal coordinate of the tooling reference coordinate from the horizontal coordinate of the current tooling coordinate to obtain the horizontal tooling drift amount; subtract the vertical coordinate of the tooling reference coordinate from the vertical coordinate of the current tooling coordinate to obtain the vertical tooling drift amount. Under the same frame sequence mark, subtract the horizontal tooling drift from the horizontal coordinate of each two-dimensional assembly bone point coordinate in the two-dimensional positioning and connection diagram, and subtract the vertical tooling drift from the vertical coordinate of each two-dimensional assembly bone point coordinate to generate a drift-compensated two-dimensional positioning and connection diagram. Following the phase order of approach phase segment, alignment phase segment, and execution phase segment, the drift-compensated 2D positioning and connection map corresponding to each frame sequence mark in each phase segment is read, and multiple drift-compensated 2D positioning and connection maps are arranged sequentially according to the frame sequence mark to generate a 2D positioning and connection sequence.

7. The robot vision positioning method for a motorcycle production line according to claim 1, characterized in that, The improved PoseFormer model specifically includes a two-dimensional bone point embedding layer, a receiving spatial encoding layer, a phase temporal encoding layer, an assembly drift reflection layer, and a three-dimensional positioning output layer. The assembly drift reflection layer is equipped with a two-dimensional receiving anchor return operator. The two-dimensional positioning sequence is input into the two-dimensional bone point embedding layer. The two-dimensional bone point embedding layer performs linear mapping on the coordinates of the two-dimensional assembly bone points, and adds the linear mapping result with the bone point type label embedding result and the phase label embedding result to form the two-dimensional bone point embedding feature. The two-dimensional bone point embedding features are input into the receiving spatial coding layer. The receiving spatial coding layer performs query mapping, key mapping, and value mapping on the two-dimensional bone point embedding features under the same frame sequence label. The query mapping results and key mapping results are multiplied item by item and then summed to generate the bone point spatial attention score. The offsets of the bearing edges corresponding to the hole bearing edge, tooling positioning bearing edge, and end execution bearing edge are added to the bone point space attention score. After normalization by the Softmax function, the score is multiplied by the value mapping result and summed to output the bone point features in the bearing space. The receiving spatial bone point features are input into the phase-temporal coding layer. The phase-temporal coding layer reads the receiving spatial bone point features of the same bone point type in the near phase segment, aligned phase segment and executed phase segment according to the frame sequence label. The corresponding features of adjacent frame sequence labels are multiplied one by one and then summed to generate a temporal attention score. The phase progressive bias from the near phase segment to the aligned phase segment and from the aligned phase segment to the executed phase segment is added to the temporal attention score. After normalization by the Softmax function, the spatial bone point features under different frame order markers are weighted and summed to output the phase temporal bone point features. The phase-time bone point features are input into the assembly drift reflection layer. The assembly drift reflection layer reads the connection relationship between the fixture positioning pin center, the frame mounting hole center, the robot end tool center and the target assembly bone point through the two-dimensional anchor return operator. The anchor return convergence features of the two-dimensional anchor point are written back to the phase-time bone point features corresponding to the target assembly bone point, and the assembly drift reflection features are output. The assembly drift reflection features are input into the 3D positioning output layer. The 3D positioning output layer sequentially performs fully connected mapping, layer normalization processing, and GELU activation processing to output the robot's 3D positioning guidance results. The robot's 3D positioning guidance results include the 3D coordinates of the target assembly point and the positioning confidence.

8. The robot vision positioning method for a motorcycle production line according to claim 7, characterized in that, The two-dimensional anchor return operator is specifically: The center of the fixture positioning pin, the center of the frame mounting hole, and the center of the robot end tool are determined as two-dimensional receiving anchor points, and the target assembly bone point is determined as the anchor return target bone point. The phase time bone point features corresponding to the two-dimensional receiving anchor point and the anchor return target bone point are read respectively. Read the two-dimensional coordinates of the anchor point under the current frame sequence mark and the two-dimensional coordinates under the previous frame sequence mark. Subtract the horizontal coordinate of the previous frame sequence mark from the horizontal coordinate of the current frame sequence mark to generate the horizontal displacement of the anchor point. Subtract the vertical coordinate of the previous frame sequence mark from the vertical coordinate of the current frame sequence mark to generate the vertical displacement of the anchor point. Set both the horizontal and vertical displacement of the anchor point corresponding to the first frame sequence mark to zero. The lateral and longitudinal displacements of the anchor point are concatenated to form an anchor point displacement vector, and the anchor point displacement vector is linearly mapped to generate the anchor point offset feature. The anchor point offset feature is added to the phase time bone point feature corresponding to the two-dimensional anchor point to generate the anchor point acceptance feature. Search for the two-dimensional bonding path from the bonding anchor point to the anchor return target bone point along the bonding edge of the hole, the bonding edge of the tooling positioning, and the bonding edge of the end execution. Count the number of two-dimensional assembly bonding edges traversed by the bonding path to obtain the bonding jump count. The anchor point acceptance feature is multiplied by the phase time bone point feature corresponding to the anchor return target bone point, and then summed to generate the anchor return correlation score. Multiply the number of hops received by the set hop attenuation coefficient to generate the path attenuation score, and subtract the path attenuation score from the anchor return related score to generate the anchor return modulation score; The anchor return modulation scores corresponding to multiple two-dimensional anchor points are input into the Softmax function to obtain the anchor return weights. The anchor point acceptance features corresponding to each two-dimensional anchor point are multiplied by the corresponding anchor return weights and then summed to generate the anchor return convergence features. The anchor return convergence feature is input into the Sigmoid function to obtain the anchor return gating value. The anchor return convergence feature and the anchor return gating value are then multiplied element by element and superimposed onto the phase time bone point feature corresponding to the anchor return target bone point to output the assembly drift reflection feature.

9. A robot vision positioning method for a motorcycle production line according to claim 1, characterized in that, Step seven specifically involves: Read the three-dimensional coordinates and positional confidence of the target assembly point from the robot's three-dimensional positioning guidance results; When the positioning confidence is greater than or equal to the set execution confidence threshold, the current position of the robot end-effector center is obtained, and the three-dimensional coordinates of the target assembly point are subtracted from the current position of the robot end-effector center to generate the end-effector positioning offset. The position of the robot end-effector center in the robot motion trajectory is corrected according to the end-effector positioning offset, and the robot end-effector center is controlled to move towards the target assembly point according to the corrected robot motion trajectory. When the positioning confidence is less than the set execution confidence threshold, the movement of the robot's end effector center is stopped, and a repositioning command is output. Based on the robot end-effector center position, target assembly point 3D coordinates, positioning confidence, and execution status before and after robot motion trajectory correction, a production line positioning execution record is generated.

10. A robot vision positioning system for a motorcycle production line, comprising executing the robot vision positioning method for a motorcycle production line as described in any one of claims 1 to 9, characterized in that, Includes the following modules: The video preprocessing module is used to acquire the ordinary two-dimensional video stream of the target assembly station of the motorcycle production line, perform frame-by-frame splitting and image preprocessing on the ordinary two-dimensional video stream, and generate a two-dimensional observation frame sequence of the station. The two-dimensional assembly bone point extraction module is used to extract two-dimensional assembly bone points from the two-dimensional observation frame sequence of the workstation, and to configure bone point type markers, two-dimensional coordinates and frame sequence markers for the two-dimensional assembly bone points to generate a two-dimensional assembly bone point sequence. The temporal residual orchestration module is used to construct a two-dimensional bone point residual matrix based on the two-dimensional assembly bone point sequence, and perform temporal residual orchestration processing to generate a two-dimensional stable bone point trajectory set. The assembly phase slicing module is used to slice the two-dimensional stable bone point trajectory set into assembly phase slices based on the distance change between the center of the robot end tool and the target assembly bone point, and generate a two-dimensional assembly phase segment set. The connection diagram construction and compensation module is used to establish two-dimensional assembly connection edges based on the two-dimensional assembly phase segment set and the motorcycle assembly process relationship, construct a two-dimensional positioning connection diagram and perform drift compensation to generate a two-dimensional positioning connection sequence. The three-dimensional positioning guidance module is used to input the two-dimensional positioning sequence into the improved PoseFormer model, and to perform anchor return association between the two-dimensional anchor point and the target assembly bone point through the two-dimensional anchor return operator, and output the robot's three-dimensional positioning guidance result. The trajectory correction and recording module is used to correct the robot's motion trajectory based on the robot's three-dimensional positioning guidance results and generate a production line positioning execution record.

Images