Automatic feeding control method of wire harness bundling machine based on multi-scene positioning

Abstract

This invention relates to the field of automatic control technology for wire harness cable ties, specifically an automatic feeding control method for wire harness cable ties based on multi-scenario positioning. The method includes: acquiring wire harness image sequences, wire harness point cloud data, and material arrival signals at the feeding station; processing the image sequences using an improved visual geometric inference algorithm, optimizing three-dimensional pose estimation based on prior knowledge of wire harness physical deformation, and analyzing the geometric pose and the position of the ties; performing surface fitting and feature enhancement on the point cloud data to extract key positioning feature point cloud clusters; inputting the above information into a multi-sensor fusion decision model to generate positioning guidance information, generating a collision-free trajectory and grasping posture through a motion planner, and converting these into servo commands to drive the feeding mechanism to complete the picking, positioning, and placement operations. This method can improve the feeding positioning accuracy and operational stability of wire harness cable ties, and adapt to the wire harness feeding needs of multiple scenarios.

Description

Technical Field

[0001] This invention relates to the field of automatic control technology for wire harness tying machines, and in particular to an automatic feeding control method for wire harness tying machines based on multi-scenario positioning. Background Technology

[0002] Automatic feeding control of wire harness tying machines is a crucial step in the wire harness processing. In existing technologies, a single vision sensor is typically used to acquire wire harness images. Conventional visual geometric algorithms are then used to analyze the wire harness pose and the position of the tying point. Simultaneously, a three-dimensional structured light sensor is used to acquire wire harness point cloud data. After simple filtering, surface features are extracted, and then combined with the material arrival signal to achieve control of the tying machine's feeding mechanism.

[0003] Conventional visual geometry algorithms do not consider the physical deformation characteristics of wire harnesses. When performing 3D pose estimation, they are easily affected by the flexible deformation of the wire harness, resulting in large pose analysis deviations and insufficient positioning accuracy of the tie-in points. The wire harness point cloud data only undergoes simple filtering without surface fitting and feature enhancement, making it difficult to effectively extract key positioning features from the wire harness surface, resulting in fuzzy and incomplete feature point recognition. At the same time, in existing technologies, data collected by multiple sensors are mostly used independently without effective fusion, leading to low confidence in the generated feeding positioning guidance information. This, in turn, results in unreasonable motion trajectory planning of the feeding mechanism, making collisions more likely and end-effector grasping posture deviations more likely. This affects feeding efficiency and operational stability, failing to meet the high-precision and high-stability requirements of automatic feeding of wire harness cable ties in various scenarios. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and propose an automatic feeding control method for wire harness cable tie machines based on multi-scenario positioning.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an automatic feeding control method for wire harness cable tie machines based on multi-scenario positioning, comprising: Acquire a multi-source sensing data set of the material loading station, the multi-source sensing data set including a wire harness image sequence acquired by a multi-angle industrial camera, wire harness point cloud data acquired by a three-dimensional structured light sensor, and material arrival signal acquired by a proximity sensor. An improved visual geometric inference algorithm is applied to the wire harness image sequence to parse the geometric pose of the wire harness and the position of the ligation point. The improved visual geometric inference algorithm optimizes the three-dimensional pose estimation process based on the prior physical deformation of the wire harness. Perform surface fitting and feature enhancement processing on the wire harness point cloud data to extract key positioning feature point cloud clusters on the wire harness surface; The geometric pose of the wire harness, the position of the knot to be tied, the key positioning feature point cloud cluster, and the material arrival signal are input into the multi-sensor fusion decision model to generate high-confidence wire harness loading and positioning guidance information. Based on the wire harness feeding positioning guidance information, a collision-free motion trajectory and end-grabbing posture of the cable tie machine feeding mechanism are generated by a motion planner. The collision-free motion trajectory and end-grabbing posture are converted into a servo control command sequence to drive the cable tie feeding mechanism to complete the picking, positioning and placement of the wire harness.

[0006] As a further aspect of the present invention, an improved visual geometric inference algorithm is applied to the wire harness image sequence to parse the geometric pose of the wire harness and the location of the ligation point, including: The image sequence of the wire harness is processed frame by frame, and the pixel regions and categories of the wire harness in the image are identified by a pre-trained semantic segmentation network. The categories include the main body of the wire harness, branch points, bending segments, and candidate regions for tying points. For the identified main pixel region of the wire harness, the improved visual geometric reasoning algorithm is used, combined with prior knowledge of the wire harness diameter, to inversely derive the three-dimensional centerline space curve of the wire harness in the camera coordinate system from the pixel coordinates of the two-dimensional image. For the identified candidate regions of the puncture point, the improved visual geometric reasoning algorithm is used to calculate their coordinates in three-dimensional space through multi-view geometric constraints. Based on the precise three-dimensional spatial coordinates corresponding to the three-dimensional centerline spatial curve and the candidate area to be tied, and combined with the preset cable tie installation process rules, the geometric pose of the wire harness and the position of the tying point are calculated. The geometric pose includes the direction, bending angle and spatial position of the wire harness.

[0007] As a further aspect of the present invention, the improved visual geometric inference algorithm optimizes the 3D pose estimation process based on the prior knowledge of the physical deformation of the wire harness, including: In the improved visual geometric reasoning algorithm, a physical deformation model of the wire harness is preset, which describes the bending, twisting and deformation range of the wire harness under the action of gravity and tension. When inverting the three-dimensional centerline space curve from the pixel coordinates of a two-dimensional image, the physical deformation model is introduced as a constraint condition, transforming the inversion problem into an optimal estimation problem under deformation constraints. Solving the optimal estimation problem yields a three-dimensional centerline spatial curve that not only satisfies multi-view projection consistency in visual geometry but also conforms to the physical deformation model in physics. This results in a geometric pose of the wire harness that is more closely matched to its actual spatial form and is physically reliable.

[0008] As a further aspect of the present invention, solving the optimal estimation problem includes: Construct an objective function that includes a data fitting term and a physical deformation regularization term. The data fitting term measures the difference between the 3D curve projected onto the image from each viewpoint and the observed pixel region. The physical deformation regularization term measures the difference between the 3D curve and the ideal deformation state predicted by the physical deformation model. The minimum value of the objective function is solved iteratively using a numerical optimization algorithm. In each iteration, the coordinates of the control points of the three-dimensional curve are adjusted to reduce both projection error and physical deformation deviation. When the iteration converges or reaches the maximum number of iterations, the final optimized three-dimensional centerline space curve is obtained as the pose estimation result.

[0009] As a further aspect of the present invention, surface fitting and feature enhancement processing are performed on the wire bundle point cloud data to extract key localization feature point cloud clusters on the wire bundle surface, including: The line bundle point cloud data is subjected to outlier filtering and voxel downsampling to obtain denoised sparse point cloud data; On the sparse point cloud data, the moving least squares method is used to perform local surface fitting on the wire bundle surface to estimate the normal vector and curvature of each point. Based on the normal vector and curvature, feature regions in the point cloud are identified, including edge regions with abrupt curvature changes, corner regions with drastic changes in normal vector, and flat regions. Cluster analysis is performed on the identified feature regions to aggregate point clouds that are spatially adjacent and have similar features into clusters, forming multiple key positioning feature point cloud clusters. Each point cloud cluster corresponds to a geometric feature on the surface of the wire harness that can be used for positioning.

[0010] As a further aspect of the present invention, the resolved geometric pose of the wire harness, the position of the anchor point, the key positioning feature point cloud cluster, and the material arrival signal are input into a multi-sensor fusion decision model to generate high-confidence wire harness loading and positioning guidance information, including: The multi-sensor fusion decision model includes a visual geometric information encoder, a point cloud feature encoder, and a signal state encoder. The visual geometric information encoder receives the geometric pose of the wire bundle and the position of the knot to be tied, and encodes them into a geometric information feature vector; The point cloud feature encoder receives the key localization feature point cloud cluster, extracts its global and local features, and encodes them into point cloud feature vectors. The signal status encoder receives the material arrival signal and processes it into a status feature vector. The geometric information feature vector, point cloud feature vector, and state feature vector are concatenated and input into a fusion network with a self-attention mechanism. The fusion network calculates the correlation weights between features from different sources and aggregates the weighted features. The output of the fusion network passes through a fully connected decision layer to generate wire harness loading and positioning guidance information, which includes the final wire harness grasping coordinates, the desired grasping direction, the estimated grasping force, and the placement target point.

[0011] As a further aspect of the present invention, based on the wire harness feeding and positioning guidance information, a motion planner generates a collision-free motion trajectory and end-effector gripping posture for the cable tie machine feeding mechanism, including: The final wire harness grabbing coordinates and the desired grabbing direction in the wire harness loading and positioning guidance information are used as the target state of the motion planner. Obtain the kinematic model of the cable tie feeding mechanism, the current joint state, and the 3D model of obstacles in the workspace; In the motion planner, an improved stochastic path planning algorithm is used to search for a smooth path from the current state to the target state in the workspace, taking into account the constraints of the kinematic model and the obstacle avoidance requirements. The smooth path obtained by the search is discretized into a series of continuous path points. For each path point, inverse kinematics is solved to obtain the angles of each joint of the cable tie feeding mechanism, thus forming the collision-free motion trajectory in the joint space. Simultaneously, based on the desired grasping direction and the configuration of the end effector, the end-effector's grasping posture at the grasping point is calculated.

[0012] As a further aspect of the present invention, the motion planner employs an improved stochastic path planning algorithm to search for a smooth path from the current state to the target state within the workspace, taking into account the constraints of the kinematic model and obstacle avoidance requirements. This includes: In the configuration space of the improved stochastic path planning algorithm, the end position and orientation of the cable tie feeding mechanism are used as nodes; When randomly sampling new nodes, a target bias strategy is introduced to sample the target state with a certain probability in order to accelerate convergence; When expanding the tree to connect new nodes, it not only checks whether the straight path collides with obstacles, but also checks whether the process of moving from the current node to the new node satisfies the constraints of joint velocity and acceleration under the kinematic model. Once the extended tree is successfully connected to the target state, the generated initial path is post-processed to smooth it, redundant nodes are removed, and path points are interpolated to ensure the smoothness and executability of the path.

[0013] As a further aspect of the present invention, the collision-free motion trajectory and end-effector gripping posture are converted into a servo control command sequence to drive the cable tie feeding mechanism to complete the picking, positioning, and placement operations of the wire harness, including: The collision-free motion trajectory is parameterized in time, and a timestamp and the desired end velocity and acceleration are assigned to each path point on the trajectory. Based on the time-parameterized trajectory and the end-grabbing posture, the desired position, speed and torque of each servo motor of the cable tie feeding mechanism in each control cycle are obtained through real-time inverse kinematics calculation. The desired position, speed, and torque of each servo motor in each control cycle are encapsulated into a specific servo control instruction sequence according to the specified communication protocol; The servo control command sequence is sent to the motion controller of the cable tie feeding mechanism in real time. The motion controller drives each servo motor to execute, so that the end effector moves along the planned trajectory and completes the picking up of the wire harness at the target point in the desired posture, and then moves to the placement point to complete the placement.

[0014] As a further aspect of the present invention, based on the normal vector and curvature, feature regions in the point cloud are identified. These feature regions include edge regions with abrupt curvature changes, corner regions with drastic changes in normal vector, and flat regions, including: Based on the calculated curvature value of each point, a first curvature threshold is set, and points with curvature greater than the first curvature threshold are initially marked as belonging to the edge region; In the neighborhood of each point, calculate the angle between its normal vector and the normal vectors of all neighboring points in the neighborhood, count the number of neighboring points whose angle is greater than a preset angle threshold in the preset neighborhood, and record this number as the normal vector mutation degree. Set a threshold for the normal vector mutation degree, and mark points whose normal vector mutation degree is greater than the threshold as belonging to the corner region; For the remaining points that are not marked as edge regions or corner regions, calculate the standard deviation of the curvature values ​​of all points in their neighborhood, set a curvature standard deviation threshold, and mark points whose curvature standard deviation is less than the curvature standard deviation threshold as belonging to flat regions. Spatial connectivity clustering analysis is performed on the initially labeled points to aggregate points that are spatially continuous and have the same labeling category, ultimately forming clearly defined edge regions, corner regions, and flat regions.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: An improved visual geometric inference algorithm is used to process the wire harness image sequence. Based on the prior knowledge of the wire harness's physical deformation, the 3D pose estimation process is optimized to resolve the geometric pose of the wire harness and the position of the binding point. This technical solution can adapt to the flexible deformation characteristics of the wire harness, avoiding pose estimation errors caused by conventional visual geometric algorithms ignoring the physical deformation of the wire harness. This makes the resolution of the wire harness's geometric pose and the position of the binding point more closely match the actual state of the wire harness, reducing positioning errors. This allows the cable tie machine's feeding mechanism to accurately identify the operation position, avoiding operational errors caused by inaccurate positioning and improving the accuracy of the feeding operation.

[0016] Surface fitting and feature enhancement processing are performed on the wire harness point cloud data to extract key positioning feature point cloud clusters from the wire harness surface. These key positioning feature point cloud clusters, along with the parsed wire harness geometric pose, the position of the anchor point, and the material arrival signal, are input into a multi-sensor fusion decision model to generate high-confidence wire harness loading positioning guidance information. Surface fitting and feature enhancement processing can enhance the recognizability of key features on the wire harness surface, solving the problems of fuzzy and incomplete feature extraction in conventional point cloud processing, making the key positioning feature point cloud clusters clearer and more accurate. The fusion of multi-sensor data can integrate multi-dimensional perception information, making up for the perception limitations of a single sensor, and solving the problem of low confidence in guidance information caused by the independent use of multi-sensor data in existing technologies. This makes the generated positioning guidance information more comprehensive and reliable, thereby making the collision-free motion trajectory and end-effector grasping posture generated by the motion planner more consistent with actual working conditions, improving the stability and smoothness of the loading mechanism operation. Attached Figure Description

[0017] Figure 1 This is a flowchart of the automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to the present invention; Figure 2 This is a flowchart illustrating the process of analyzing geometric pose and tie-point position using an improved visual geometric reasoning algorithm for wire harness image sequence in step S2 of the automatic feeding control method for wire harness cable tie machines based on multi-scene positioning in this invention. Figure 3 This is a diagram showing the three-dimensional geometric pose of the wire harness and the positioning effect of the point to be tied in step S2 of the automatic feeding control method for wire harness cable ties based on multi-scene positioning of the present invention. Figure 4 This is a flowchart of step S2 of the automatic feeding control method for wire harness cable tie machine based on multi-scene positioning of the present invention, which is a flowchart of the process of optimizing the three-dimensional pose estimation process of the improved visual geometric reasoning algorithm based on the prior of physical deformation. Figure 5 To illustrate step S3 of the automatic feeding control method for wire harness cable tie machine based on multi-scene positioning of the present invention, a wire harness point cloud preprocessing and key feature point cloud cluster extraction effect diagram is shown, which shows the complete processing flow of the wire harness three-dimensional point cloud from raw data to key positioning features. Figure 6 To illustrate step S5 of the automatic feeding control method for wire harness cable tie machine based on multi-scene positioning of the present invention, a collision-free motion trajectory planning effect diagram of the cable tie machine feeding mechanism is shown. Figure 7 This is a schematic diagram illustrating the contact between the end effector's gripping posture and the wire harness in step S5 of the automatic feeding control method for wire harness cable ties based on multi-scene positioning of the present invention. Figure 8 This is a diagram illustrating the servo control command sequence and execution timing in step S6 of the automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] See Figure 1 This invention provides an automatic feeding control method for wire harness cable tie machines based on multi-scenario positioning, the specific method including: S1: Acquire a multi-source sensing data set for the material loading station. This set includes a sequence of wire harness images acquired by a multi-angle industrial camera, wire harness point cloud data acquired by a 3D structured light sensor, and material arrival signals acquired by a proximity sensor.

[0021] S2: Execute an improved visual geometric inference algorithm on the wire harness image sequence. The algorithm parses the geometric pose of the wire harness and the position of the ligation point. The improved visual geometric inference algorithm optimizes the three-dimensional pose estimation process based on the prior physical deformation of the wire harness.

[0022] S3: Perform surface fitting and feature enhancement processing on the wire harness point cloud data to extract the key localization feature point cloud clusters on the wire harness surface.

[0023] S4: The parsed wire harness geometric pose, the position of the point to be tied, the key positioning feature point cloud cluster, and the material arrival signal are input into a multi-sensor fusion decision model. This model generates high-confidence wire harness loading and positioning guidance information through internal information encoding, weighting, and aggregation mechanisms.

[0024] S5: Based on the wire harness feeding positioning guidance information, the collision-free motion trajectory and end gripping posture of the cable tie machine feeding mechanism are generated by calling the motion planner.

[0025] S6: The collision-free motion trajectory and end-effector gripping posture are converted into a servo control command sequence to drive the machine's servo system, thereby precisely driving the cable tie feeding mechanism to complete the specified operation of the wire harness. The specified operation can be picking up, positioning, and placing.

[0026] Figure 2 A flowchart illustrating the improved visual geometric inference algorithm for analyzing geometric pose and puncture point location for wire harness image sequences.

[0027] In one embodiment of the present invention, reference is made to... Figure 2 In the automatic feeding control method for wire harness cable ties based on multi-scene positioning of the present invention, in step S2, an improved visual geometric inference algorithm is executed on the wire harness image sequence. The algorithm parses the geometric pose of the wire harness and the position of the tying point. The improved visual geometric inference algorithm for wire harness image sequence parses the geometric pose and the position of the tying point specifically includes: S21: Process the wire harness image sequence frame by frame, and identify the pixel regions and categories of the wire harness in the image through a pre-trained semantic segmentation network. The categories include the main body of the wire harness, branch points, bending segments, and candidate regions for tying points.

[0028] S22: For the identified main pixel region of the wire harness, an improved visual geometric reasoning algorithm is used, combined with prior knowledge of the wire harness diameter, to inversely deduce the three-dimensional centerline space curve of the wire harness in the camera coordinate system from the pixel coordinates of the two-dimensional image.

[0029] S23: For the identified candidate regions of the puncture point, the improved visual geometric reasoning algorithm is used to calculate its coordinates in three-dimensional space through multi-view geometric constraints.

[0030] S24: Based on the precise three-dimensional spatial coordinates corresponding to the three-dimensional centerline spatial curve and the candidate area to be tied, and combined with the preset cable tie installation process rules, calculate the geometric pose of the wire harness and the position of the tying point. The geometric pose includes the direction, bending angle and spatial position of the wire harness.

[0031] In practice, an improved visual geometric inference algorithm is applied to the wire harness image sequence to resolve the geometric pose of the wire harness and the location of the ligation point. This process follows the steps below: The wire harness image sequence is processed frame by frame. A pre-trained semantic segmentation network is used to identify the pixel regions and their categories of the wire harness in the image. The categories include the main body of the wire harness, branch points, bending segments, and candidate regions for tying points. In some embodiments, the semantic segmentation network adopts a deep convolutional neural network-based architecture and is trained on image datasets containing multiple wire harness types using transfer learning to improve the accuracy of pixel region segmentation. For the identified main body pixel region of the wire harness, an improved visual geometric inference algorithm is used in conjunction with prior knowledge of the wire harness diameter to inversely deduce the three-dimensional centerline space curve of the wire harness in the camera coordinate system from the pixel coordinates of the two-dimensional image. The improved visual geometric inference algorithm establishes a mapping from two-dimensional pixel points to three-dimensional space points through the correspondence between multi-view images.

[0032] In practical implementation, when retrieving the three-dimensional centerline space curve from the pixel coordinates of a two-dimensional image, the centerline pixel coordinates of the main pixel region of the bundle are first extracted. Then, three-dimensional reconstruction is performed by combining camera calibration parameters and multi-view geometric constraints. Prior knowledge of the bundle diameter is used as an additional constraint to adjust the radius of the reconstructed curve, making the three-dimensional centerline space curve more consistent with the actual physical dimensions. For the identified candidate regions of the knot, an improved visual geometric inference algorithm is used to calculate their coordinates in three-dimensional space through multi-view geometric constraints. In some embodiments, the multi-view geometric constraints are based on feature point matching and triangulation principles, determining the corresponding pixel points of the candidate regions of the knot from images from at least two different perspectives, and solving for the three-dimensional coordinates by minimizing the reprojection error. The formula for calculating the three-dimensional space coordinates of the candidate regions of the knot can be expressed as: ; in: This represents the coordinate vector of the candidate region for tethering points in three-dimensional space. Indicates the first The pixel coordinates of the candidate region for the insertion point in each view image. Represents the camera projection function. Indicates the first Camera projection matrix for each viewpoint, Indicates the number of viewpoints.

[0033] It is understandable that this formula minimizes the error of projecting 3D coordinates onto images from various viewpoints through an optimization process. Based on the precise 3D spatial coordinates corresponding to the 3D centerline curve and the candidate region for the tying point, the geometric pose of the wire harness and the position of the tying point are calculated in combination with the preset cable tie installation process rules.

[0034] In practical implementation, the cable tie installation process rules include the minimum distance requirement between the point to be tied and the branch point or bend of the wire harness, the parallelism requirement between the cable tie installation direction and the wire harness direction, and the geometric pose including the wire harness direction, bending angle, and spatial position. The direction is described by the first derivative of the three-dimensional centerline space curve, i.e., the tangent direction; the bending angle is obtained by the integral of the curve curvature; and the spatial position is determined by the three-dimensional coordinates of key points on the curve. Optionally, the location of the point to be tied is selected from the candidate area based on distance constraints and directional consistency to determine the final cable tie installation point.

[0035] Figure 4 A flowchart illustrating the process of optimizing 3D pose estimation based on physical deformation priors to improve the visual geometric inference algorithm.

[0036] In one embodiment of the present invention, reference is made to... Figure 4 In the automatic feeding control method for wire harness cable tie machines based on multi-scene positioning of the present invention, in step S2, the improved visual geometric inference algorithm optimizes the three-dimensional pose estimation process based on the prior physical deformation of the wire harness, including: S25: In the improved visual geometric reasoning algorithm, a physical deformation model of the wire harness is preset. The physical deformation model describes the bending, twisting and deformation range of the wire harness under the action of gravity and tension.

[0037] S26: When inverting the three-dimensional centerline space curve from the pixel coordinates of a two-dimensional image, a physical deformation model is introduced as a constraint condition, transforming the inversion problem into an optimal estimation problem under deformation constraints.

[0038] S27: Solve the optimal estimation problem to obtain the final optimized three-dimensional centerline space curve as the pose estimation result. This three-dimensional centerline space curve not only satisfies the consistency of multi-view projection in visual geometry, but also conforms to the physical deformation model in physics, thus obtaining a wire harness geometric pose with higher matching degree with the actual spatial shape of the wire harness and physical reliability.

[0039] Step S27: Solving the optimal estimation problem may include: S271: Construct an objective function that includes a data fitting term and a physical deformation regularization term. The data fitting term measures the difference between the 3D curve projected onto the image at each viewpoint and the observed pixel region. The physical deformation regularization term measures the difference between the 3D curve and the ideal deformation state predicted by the physical deformation model.

[0040] S272: The minimum value of the objective function is solved iteratively using a numerical optimization algorithm. In each iteration, the coordinates of the control points of the three-dimensional curve are adjusted to reduce both projection error and physical deformation deviation.

[0041] S273: End the iteration until the iteration converges or the maximum number of iterations is reached.

[0042] In practical implementation, the improved visual geometric inference algorithm optimizes the three-dimensional pose estimation process based on the prior physical deformation of the wire harness. The improved visual geometric inference algorithm presets the physical deformation model of the wire harness, which describes the bending, twisting and deformation range of the wire harness under the action of gravity and tension.

[0043] In some embodiments, the physical deformation model can be a parametric mechanical model, such as discretizing the wire harness into a series of mass points connected by springs and dampers, whose bending and torsional stiffness are defined by material properties, thereby simulating the static equilibrium state of the wire harness under its own weight and external forces. When inverting the three-dimensional centerline space curve from the pixel coordinates of a two-dimensional image, the physical deformation model is introduced as a constraint, transforming the inversion problem into an optimal estimation problem under deformation constraints. Solving this optimal estimation problem involves constructing an objective function that includes a data fitting term and a physical deformation regularization term. The data fitting term measures the difference between the three-dimensional curve projected onto the image at each viewpoint and the observed pixel region, while the physical deformation regularization term measures the difference between the three-dimensional curve and the ideal deformation state predicted by the physical deformation model. The specific form of the objective function can be expressed as: ; in: This represents the total optimal energy. This indicates the total number of angles of view of the industrial camera. Indicates the first The number of line bundle pixels matched in each viewpoint image Indicates the first From the perspective of the first The two-dimensional coordinates of the observed line bundle pixels. Represents the three-dimensional centerline space curve in terms of parameters The corresponding 3D point coordinates Represents the transition from three-dimensional space to the third dimension. Projection transformation of the image plane of each view camera This represents a robust loss function used to reduce the impact of incorrect matching points. This represents the physical deformation regularization term, which is calculated based on the physical deformation model of the curve. Evaluation of physical quantities such as bending energy and torsional energy. This represents the regularization coefficient used to balance the effects of the data fitting term and the physical deformation regularization term.

[0044] Understandably, minimizing this objective function means finding a three-dimensional curve whose projection from multiple viewpoints matches the observed image as closely as possible, while its physical deformation also conforms to the reasonable range described by the physical deformation model. A numerical optimization algorithm is used to iteratively solve for the minimum value of the objective function. In each iteration, the coordinates of the control points of the three-dimensional curve are adjusted to simultaneously reduce projection errors and physical deformation deviations. In some embodiments, the numerical optimization algorithm employs the Levenberg-Marquardt algorithm or the Gauss-Newton algorithm, iteratively updating the objective function by calculating the gradient or Jacobian matrix of the curve's control point coordinates.

[0045] When the iteration converges or reaches the maximum number of iterations, the final optimized 3D centerline space curve is obtained as the pose estimation result. This 3D centerline space curve not only satisfies multi-view projection consistency in visual geometry but also conforms to the physical deformation model in physical geometry, thus obtaining a wire harness geometric pose with higher matching degree to the actual spatial shape of the wire harness and greater physical reliability. Optional, physical deformation regularization term. The specific calculations can be achieved by comparing the curvature and torsion distributions of the current 3D centerline space curve with the corresponding distributions of the ideal curve simulated by the physical deformation model under the same constraints. It can be understood that by embedding the physical deformation prior into the optimization framework as a regularization term, the pose estimation process can effectively resist interference from visual occlusion, monotonous texture, or image noise, guiding the solution to converge in a direction that better conforms to physical laws.

[0046] Figure 3 This image shows the 3D geometric pose of a wire harness and the localization of the anchor points. The smooth red curve represents the 3D centerline of the wire harness, characterizing its direction, bending angle, and spatial position. The highlighted yellow dots represent precise anchor points, calculated using an improved visual geometric inference algorithm combined with multi-view constraints and prior physical deformation. This image visually demonstrates the high-precision analytical capability for the wire harness's geometric pose and anchor points, proving the engineering feasibility of the visual localization process.

[0047] In one embodiment of the present invention, step S3 performs surface fitting and feature enhancement processing on the wire harness point cloud data to extract key localization feature point cloud clusters on the wire harness surface, specifically including: S31: Perform outlier filtering and voxel downsampling on the wire harness point cloud data to obtain denoised sparse point cloud data.

[0048] S32: On sparse point cloud data, the moving least squares method is used to perform local surface fitting on the wire bundle surface to estimate the normal vector and curvature of each point.

[0049] S33: Based on normal vectors and curvature, feature regions in point clouds are identified, including edge regions with abrupt curvature changes, corner regions with drastic changes in normal vectors, and flat regions.

[0050] S34: Perform cluster analysis on the identified feature regions, and aggregate point clouds that are spatially adjacent and have similar features into clusters to form multiple key positioning feature point cloud clusters. Each point cloud cluster corresponds to a geometric feature on the surface of the wire harness that can be used for positioning.

[0051] In one embodiment, step S33 may specifically include: S331: The specific process of identifying feature regions based on normal vectors and curvature includes setting a first curvature threshold based on the calculated curvature value of each point, and initially marking points with curvature greater than the first curvature threshold as belonging to the edge region.

[0052] S332: Within the neighborhood of each point, calculate the angle between its normal vector and the normal vectors of all adjacent points in the neighborhood, count the number of adjacent points within the preset neighborhood whose angle is greater than a preset angle threshold, and record this number as the normal vector mutation degree.

[0053] S333: Set a threshold for the normal vector mutation degree, and mark points whose normal vector mutation degree is greater than the threshold as belonging to the corner region.

[0054] S334: For the remaining points that are not marked as edge regions or corner regions, calculate the standard deviation of the curvature values ​​of all points in their neighborhood, set a curvature standard deviation threshold, and mark the points whose curvature standard deviation is less than the curvature standard deviation threshold as belonging to flat regions.

[0055] S335: Perform spatial connectivity clustering analysis on the initially labeled points to aggregate points that are spatially continuous and have the same labeling category, forming clearly defined edge regions, corner regions, and flat regions.

[0056] In practice, surface fitting and feature enhancement processing are performed on the wire harness point cloud data to extract key localization feature point cloud clusters on the wire harness surface. This process begins with outlier filtering and voxel downsampling of the original wire harness point cloud data. Discrete noise points are removed by statistical filtering or radius filtering methods. Voxel grid filters are used to downsample the point cloud, reducing the number of point clouds while maintaining the overall shape of the wire harness, resulting in denoised sparse point cloud data.

[0057] Understandably, this step aims to improve subsequent processing efficiency and suppress noise interference. On the sparse point cloud data, a moving least squares method is used to perform local surface fitting on the wire bundle surface, constructing a local neighborhood for each point in the point cloud. Within this neighborhood, a polynomial surface is fitted using weighted least squares. Based on the fitted local surface, the normal vector and curvature of each point are estimated. In some embodiments, the polynomial surface is typically a quadratic surface. The normal vector of a point is given by the normal direction of the fitted surface, and the curvature of a point is obtained by calculating the mean of the principal curvatures or the Gaussian curvature of the surface at that point. Based on the calculated normal vector and curvature, feature regions in the point cloud are identified. Feature regions include edge regions with abrupt curvature changes, corner regions with drastic changes in normal vectors, and flat regions.

[0058] In specific implementation, a first curvature threshold is set based on the calculated curvature value of each point. Points with curvature greater than the first curvature threshold are initially marked as belonging to edge regions. Within the neighborhood of each point, the angle between its normal vector and the normal vectors of all adjacent points in the neighborhood is calculated. The number of adjacent points within the preset neighborhood with an angle greater than a preset angle threshold is counted and recorded as the normal vector abrupt change degree. A normal vector abrupt change degree threshold is set, and points with a normal vector abrupt change degree greater than the threshold are marked as belonging to corner regions. For the remaining points not marked as edge regions or corner regions, the standard deviation of the curvature values ​​of all points in their neighborhood is calculated. A curvature standard deviation threshold is set, and points with a curvature standard deviation less than the threshold are marked as belonging to flat regions. See Table 1 for a simplified point cloud feature recognition judgment: Table 1: Point Cloud Feature Region Determination Table Optionally, calculate the abrupt change in the normal vector of a point P. The formula can be expressed as: ; in: This represents the degree of abrupt change in the normal vector of point P. Let P be the set of neighborhood points. Represents a point within the neighborhood. and Let represent the unit normal vectors of points P and q, respectively. It is an indicator function that takes the value 1 when the condition inside the parentheses is true and 0 otherwise. This indicates the preset angle threshold. Calculate the absolute value of the angle between the normal vectors of two points.

[0059] Spatial connectivity clustering analysis is performed on the initially labeled points to aggregate points that are geographically close and have the same labeling category in three-dimensional space, ultimately forming clearly defined edge regions, corner regions, and flat regions. In some embodiments, Euclidean distance-based clustering algorithms such as DBSCAN or connected component analysis can be used to achieve spatial clustering.

[0060] In practice, after clustering analysis of the identified feature regions, point clouds that are spatially adjacent and have similar features are aggregated into clusters, forming multiple key positioning feature point cloud clusters. Each point cloud cluster corresponds to a geometric feature on the wire harness surface that can be used for positioning, such as a protruding corner, a distinct edge, or a flat mounting surface. It can be understood that through the above steps of surface fitting, feature calculation, threshold determination, and spatial clustering, stable and discriminative key positioning feature point cloud clusters can be structurally extracted from the original 3D point cloud.

[0061] Figure 5 To illustrate step S3 of the automatic feeding control method for wire harness cable ties based on multi-scene positioning of the present invention, an image showing the preprocessing of wire harness point cloud and extraction of key feature point cloud clusters is provided. This image demonstrates the complete processing flow of the wire harness 3D point cloud from raw data to key positioning features. The gray scattered points represent the original collected point cloud, while the blue highlighted areas represent the key positioning feature point cloud clusters obtained after outlier filtering, voxel downsampling, surface fitting, normal vector and curvature calculation, and feature clustering, corresponding to three types of positioning features: edges, corners, and flat areas. Figure 5 This demonstrates that the point cloud processing stage can reliably extract effective features, providing reliable input for multi-sensor fusion.

[0062] In step S4 of this invention, the parsed wire harness geometric pose, the position of the point to be tied, the key positioning feature point cloud cluster, and the material arrival signal are input into the multi-sensor fusion decision model to generate high-confidence wire harness loading and positioning guidance information.

[0063] The multi-sensor fusion decision model includes a visual geometric information encoder, a point cloud feature encoder, and a signal state encoder.

[0064] In one embodiment of the present invention, step S4 may specifically include: S41: The visual geometric information encoder receives the geometric pose of the wire harness and the position of the knot to be tied, and encodes them into a geometric information feature vector.

[0065] S42: The point cloud feature encoder receives key localization feature point cloud clusters, extracts their global and local features, and encodes them into point cloud feature vectors.

[0066] S43: The signal status encoder receives the material arrival signal and processes it into a status feature vector.

[0067] S44: The geometric information feature vector, point cloud feature vector, and state feature vector are concatenated and input into a fusion network with a self-attention mechanism. The fusion network calculates the correlation weights between features from different sources and aggregates the weighted features.

[0068] S45: The output of the fusion network passes through a fully connected decision layer to generate wire harness loading positioning guidance information that includes the final wire harness grasping coordinates, expected grasping direction, estimated grasping force, and placement target point.

[0069] In one embodiment of the present invention, in the automatic feeding control method for wire harness cable tie machines based on multi-scene positioning, step S5, based on the wire harness feeding positioning guidance information, generates a collision-free motion trajectory and end-effector gripping posture of the cable tie machine feeding mechanism through a motion planner, including: S51: Use the final wire harness grabbing coordinates and the desired grabbing direction in the wire harness loading positioning guidance information as the target state of the motion planner.

[0070] S52: Obtain the kinematic model of the cable tie feeding mechanism, the current joint state, and the 3D model of obstacles in the workspace.

[0071] S53: In the motion planner, an improved stochastic path planning algorithm is adopted. Under the premise of considering the kinematic model constraints and obstacle avoidance requirements, a smooth path from the current state to the target state is searched in the workspace, and the smooth path obtained by the search is discretized into a series of continuous path points.

[0072] S54: For each path point, perform inverse kinematics solution to obtain the angles of each joint of the cable tie feeding mechanism, thus forming a collision-free motion trajectory in the joint space.

[0073] S55: At the same time, based on the desired grasping direction and the configuration of the end effector, the end grasping posture of the end effector at the grasping point is calculated.

[0074] In one embodiment, during step S53, the following steps may be performed: S531: In the configuration space of the improved stochastic path planning algorithm, the end position and orientation of the cable tie feeding mechanism are used as nodes.

[0075] S532: When randomly sampling new nodes, a target bias strategy is introduced to sample towards the target state with a certain probability in order to accelerate convergence.

[0076] S533: When expanding the tree to connect new nodes, not only is it checked whether the straight path collides with obstacles, but also whether the process of moving from the current node to the new node satisfies the constraints of joint velocity and acceleration under the kinematic model.

[0077] S534: After the extended tree is successfully connected to the target state, the generated initial path is post-processed to smooth it, redundant nodes are removed, and path points are interpolated to ensure the smoothness and executability of the path.

[0078] In practice, the parsed wire harness geometric pose, the position of the point to be tied, the key positioning feature point cloud cluster, and the material arrival signal are input into the multi-sensor fusion decision model to generate high-confidence wire harness loading positioning guidance information. The multi-sensor fusion decision model includes a visual geometric information encoder, a point cloud feature encoder, and a signal state encoder. The visual geometric information encoder receives the geometric pose of the wire harness and the position of the point to be tied and encodes them into a geometric information feature vector.

[0079] In some embodiments, the visual geometric information encoder can be a multilayer perceptron network. Its input is a set of discrete point coordinates, tangent directions, and the three-dimensional coordinates of the point to be anchored on the three-dimensional centerline space curve of the wire bundle. The output is a fixed-dimensional geometric information feature vector. The point cloud feature encoder receives key positioning feature point cloud clusters and extracts their global and local features, encoding them into point cloud feature vectors. In specific implementations, the point cloud feature encoder can employ a PointNet++-based neural network structure to perform hierarchical feature learning on each key positioning feature point cloud cluster, ultimately aggregating them to obtain a point cloud feature vector representing the geometric characteristics of that cluster. The signal state encoder receives material arrival signals and processes them into state feature vectors. Material arrival signals are typically binary or analog signals from proximity sensors. The signal state encoder maps the signal values ​​to a high-dimensional vector space through an embedding layer. The geometric information feature vector, point cloud feature vector, and state feature vector are concatenated and input into a fusion network with a self-attention mechanism. The fusion network calculates the correlation weights between features from different sources and aggregates the weighted features. The formula for calculating the correlation weights using the self-attention mechanism can be expressed as: ; in: , , These are the query matrix, key matrix, and value matrix, respectively, obtained by linear transformation of the concatenated feature vectors. is the dimension of the key vector, where k represents the key; the softmax function is used to normalize the calculated attention score into weights, and the final output is the weighted aggregated fusion feature. The output of the fusion network passes through a fully connected decision layer to generate wire harness loading and positioning guidance information containing the final wire harness grasping coordinates, expected grasping direction, estimated grasping force, and placement target point. See Table 2 for the structure of one wire harness loading and positioning guidance information: Table 2: Wire Harness Loading Positioning Guidance Information Table It is understandable that introducing kinematic constraint checks can prevent the planning of theoretically collision-free trajectories that the mechanism cannot track and execute. The smooth path obtained by the search is discretized into a series of continuous path points. Inverse kinematics is performed on each path point to obtain the angles of each joint of the cable tie feeding mechanism, thus forming a collision-free motion trajectory in joint space. In some embodiments, the inverse kinematics solution adopts a numerical iterative method to calculate the angle values ​​of each joint of the robot for the end effector pose of each path point. At the same time, the end effector grasping posture at the grasping point is calculated based on the desired grasping direction and the configuration of the end effector. Optionally, the end effector grasping posture is represented by a rotation matrix or quaternion. Its calculation must ensure that the grasping direction of the end effector is aligned with the desired grasping direction and that the actuator itself does not interfere with the environment.

[0080] Figure 6 To illustrate step S5 of the automatic feeding control method for wire harness cable ties based on multi-scene positioning of the present invention, a diagram showing the collision-free motion trajectory planning effect of the cable tie feeding mechanism is provided. The red semi-transparent area represents obstacles, and the green curve represents the collision-free smooth trajectory generated by the improved random path planning algorithm. The path starts from the initial position, bypasses obstacles, and reaches the grab point and placement point, satisfying kinematic constraints and obstacle avoidance requirements. This diagram visually demonstrates that a safe and executable motion trajectory can be generated, solving the problems of easy collisions and unreasonable trajectories in traditional solutions.

[0081] Figure 7 To illustrate step S5 of the automatic feeding control method for wire harness cable ties based on multi-scenario positioning of the present invention, a schematic diagram of the end effector's gripping posture in contact with the wire harness is shown. The thick gold line represents the wire harness, the gray structure represents the gripper-type end effector, and the red arrow indicates the desired gripping direction. This diagram shows the state where the gripping direction is aligned with the wire harness surface, the grippers do not interfere, and the posture matches the process requirements, proving that a stable and reliable gripping posture can be calculated, ensuring accurate completion of the picking action.

[0082] In one embodiment of the present invention, in the automatic feeding control method for wire harness cable tie machine based on multi-scene positioning, step S6 converts the collision-free motion trajectory and end-effector gripping posture into a servo control command sequence to drive the cable tie machine feeding mechanism to complete the specified operation of the wire harness, such as picking, positioning, and placing. Specifically, this includes: S61 performs time parameterization on the collision-free motion trajectory, assigning a timestamp and the desired end velocity and acceleration to each path point on the trajectory.

[0083] S62, based on the time-parameterized trajectory and end-grabbing posture, calculates the desired position, speed and torque of each servo motor of the cable tie feeding mechanism in each control cycle through real-time inverse kinematics.

[0084] S63 encapsulates the desired position, speed, and torque of each servo motor in each control cycle into a specific servo control instruction sequence according to the specified communication protocol.

[0085] S64 sends the servo control command sequence to the motion controller of the cable tie feeding mechanism in real time. The motion controller drives each servo motor to execute, so that the end effector moves along the planned trajectory and completes the picking up of the wire harness at the target point in the desired posture, and then moves to the placement point to complete the placement.

[0086] In practice, the collision-free motion trajectory and end-effector gripping posture are converted into a servo control command sequence to drive the cable tie feeding mechanism to complete the picking, positioning and placement of the wire harness. This conversion and execution process follows a defined procedure, and the collision-free motion trajectory is time-parameterized to assign a timestamp and the desired end-effector velocity and acceleration to each path point on the trajectory.

[0087] In some embodiments, time parameterization employs an S-shaped velocity planning algorithm to calculate a smooth time-position curve for the collision-free motion trajectory across the entire joint space, given a total motion time, maximum velocity, maximum acceleration, and jerk constraints. This assigns a specific arrival time, instantaneous velocity, and instantaneous acceleration value to each discrete path point. Based on the time-parameterized trajectory and the end effector gripping posture, real-time inverse kinematics calculations are used to obtain the desired position, velocity, and torque of each servo motor of the cable tie feeding mechanism in each control cycle. In specific implementations, real-time inverse kinematics calculations are based on the kinematic model of the cable tie feeding mechanism. For each control cycle, the desired angle of each joint is solved based on the end effector pose (including position and gripping posture) given by the time-parameterized trajectory at the current moment. Combined with the instantaneous velocity and acceleration given by the trajectory, the desired velocity and desired torque of each joint are solved using kinematic differential relations.

[0088] It is understandable that this calculation process needs to be completed within each control cycle to ensure real-time control. The desired position, speed, and torque of each servo motor within each control cycle are encapsulated into a specific servo control command sequence according to a specified communication protocol. This servo control command sequence typically includes fields such as target position, target speed, target torque, and control mode. Optionally, a checksum and sequence number must also be added during the encapsulation process to ensure reliable communication. The formula for generating the servo control command sequence can be expressed as: ; in: Indicates the first The servo control instruction package generated in each control cycle A function that encapsulates data according to a specified communication protocol. Indicates the first The expected position angle vectors of each joint are calculated in each control cycle. Indicates the first The expected velocity vectors of each joint are calculated in each control cycle. Indicates the first The expected torque vector of each joint is calculated in each control cycle. This indicates the preset control mode code. The servo control command sequence is sent to the motion controller of the cable tie feeding mechanism in real time. The motion controller drives each servo motor to execute, so that the end effector moves along the planned trajectory and completes the picking up of the wire harness at the target point in the desired posture, and then moves to the placement point to complete the placement.

[0089] In some embodiments, the servo control command sequence is periodically sent via a real-time industrial Ethernet bus. After receiving the commands, the motion controller closes the position loop, speed loop, or torque loop in the underlying servo driver to precisely control the motor movement. Optionally, during critical phases of the pick-and-place action, the torque control mode in the servo control command sequence is activated to achieve smooth gripping and precise placement, avoiding damage to the wire harness or workpiece.

[0090] Figure 8 To illustrate step S6 of the automatic feeding control method for wire harness and cable tie machines based on multi-scenario positioning of the present invention, a servo control command sequence and execution timing diagram is shown. The upper diagram shows the position command curve, and the lower diagram shows the speed command curve; both are smooth S-shaped plans, meeting the control requirements of a servo system. This diagram demonstrates the final execution stage of the present invention, which transforms the upper-level trajectory into a directly executable servo control command sequence, forming a complete closed loop of "perception—decision—planning—execution," proving that the solution is feasible and engineerable.

[0091] It is understandable that the complete chain from time parameterization and inverse kinematics solution to instruction encapsulation and transmission realizes the transformation of the high-level motion intentions planned by the upper layer into specific action instructions that can be executed by the lower-level drive unit.

[0092] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An automatic feeding control method for wire harness cable tie machines based on multi-scenario positioning, characterized in that, The method includes: S1: Acquire a multi-source sensing data set of the material loading station. The multi-source sensing data set includes a sequence of wire harness images acquired by a multi-angle industrial camera, wire harness point cloud data acquired by a three-dimensional structured light sensor, and material arrival signals acquired by a proximity sensor. S2: Execute an improved visual geometric inference algorithm on the wire harness image sequence. The algorithm parses the geometric pose of the wire harness and the position of the ligation point. The improved visual geometric inference algorithm optimizes the three-dimensional pose estimation process based on the prior physical deformation of the wire harness. S3: Perform surface fitting and feature enhancement processing on the wire harness point cloud data to extract key positioning feature point cloud clusters on the wire harness surface; S4: Input the parsed geometric pose of the wire harness, the position of the point to be tied, the key positioning feature point cloud cluster, and the material arrival signal into the multi-sensor fusion decision model to generate high-confidence wire harness loading and positioning guidance information. S5: Based on the wire harness feeding positioning guidance information, by calling the motion planner, generate the collision-free motion trajectory and end gripping posture of the cable tie feeding mechanism; S6: Convert the collision-free motion trajectory and end-effector gripping posture into a servo control command sequence to drive the cable tie feeding mechanism to complete the specified operation of the wire harness.

2. The automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to claim 1, characterized in that, In step S2, an improved visual geometric inference algorithm is applied to the wire harness image sequence to parse the geometric pose of the wire harness and the location of the ligation point, including: S21: The wire harness image sequence is processed frame by frame, and the pixel regions and categories of the wire harness in the image are identified by a pre-trained semantic segmentation network. The categories include the wire harness body, branch points, bending segments and candidate regions for tying points. S22: For the identified main pixel area of ​​the wire harness, the improved visual geometric reasoning algorithm is used, combined with the prior knowledge of the wire harness diameter, to inversely deduce the three-dimensional centerline space curve of the wire harness in the camera coordinate system from the pixel coordinates of the two-dimensional image. S23: For the identified candidate regions of the puncture point, the improved visual geometric reasoning algorithm is used to calculate its coordinates in three-dimensional space through multi-view geometric constraints; S24: Based on the precise three-dimensional spatial coordinates corresponding to the three-dimensional centerline spatial curve and the candidate area to be tied, and combined with the preset cable tie installation process rules, calculate the geometric pose of the wire harness and the position of the tying point. The geometric pose includes the direction, bending angle and spatial position of the wire harness.

3. The automatic feeding control method for wire harness cable tie machine based on multi-scenario positioning according to claim 2, characterized in that, In step S2, the improved visual geometric inference algorithm optimizes the 3D pose estimation process based on the prior physical deformation of the wire harness, including: S25: In the improved visual geometric reasoning algorithm, a physical deformation model of the wire harness is preset, which describes the bending, twisting and deformation range of the wire harness under the action of gravity and tension. S26: When inverting the three-dimensional centerline space curve from the pixel coordinates of a two-dimensional image, the physical deformation model is introduced as a constraint condition, transforming the inversion problem into an optimal estimation problem under deformation constraints. S27: Solve the optimal estimation problem and obtain the three-dimensional centerline space curve as the pose estimation result. The three-dimensional centerline space curve not only satisfies the multi-view projection consistency in visual geometry, but also conforms to the physical deformation model in physics, thereby obtaining the geometric pose of the wire harness that has a higher degree of matching with the actual spatial shape of the wire harness and is physically reliable.

4. The automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to claim 3, characterized in that, Solving the optimal estimation problem includes: S271: Construct an objective function that includes a data fitting term and a physical deformation regularization term. The data fitting term measures the difference between the three-dimensional curve projected onto the image at each viewpoint and the observed pixel region. The physical deformation regularization term measures the difference between the three-dimensional curve and the ideal deformation state predicted by the physical deformation model. S272, The minimum value of the objective function is solved iteratively using a numerical optimization algorithm. In each iteration, the coordinates of the control points of the three-dimensional curve are adjusted to reduce both projection error and physical deformation deviation. S273. When the iteration converges or the maximum number of iterations is reached, the final optimized three-dimensional centerline space curve is obtained as the pose estimation result.

5. The automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to claim 1, characterized in that, Step S3: Perform surface fitting and feature enhancement processing on the wire harness point cloud data to extract key localization feature point cloud clusters on the wire harness surface, including: S31, perform outlier filtering and voxel downsampling on the wire bundle point cloud data to obtain denoised sparse point cloud data; S32, on the sparse point cloud data, the moving least squares method is used to perform local surface fitting on the wire bundle surface to estimate the normal vector and curvature of each point; S33, based on the normal vector and curvature, identify the feature regions in the point cloud, including edge regions with abrupt curvature changes, corner regions with drastic changes in normal vector, and flat regions; S34, perform cluster analysis on the identified feature regions, and aggregate point clouds that are spatially adjacent and have similar features into clusters to form multiple key positioning feature point cloud clusters. Each point cloud cluster corresponds to a geometric feature on the surface of the wire harness that can be used for positioning.

6. The automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to claim 1, characterized in that, In step S4, the parsed geometric pose of the wire harness, the position of the anchor point, the key positioning feature point cloud cluster, and the material arrival signal are input into a multi-sensor fusion decision model to generate high-confidence wire harness loading positioning guidance information. The multi-sensor fusion decision model includes a visual geometric information encoder, a point cloud feature encoder, and a signal state encoder. This step specifically includes: S41, the visual geometric information encoder receives the geometric pose of the wire bundle and the position of the knot to be tied, and encodes them into a geometric information feature vector; S42, the point cloud feature encoder receives the key localization feature point cloud cluster, extracts its global and local features, and encodes them into a point cloud feature vector; S43, the signal status encoder receives the material arrival signal and processes it into a status feature vector; S44, the geometric information feature vector, point cloud feature vector and state feature vector are concatenated and input into a fusion network with a self-attention mechanism. The fusion network calculates the correlation weights between features from different sources and aggregates the weighted features. S45, the output of the fusion network passes through a fully connected decision layer to generate the wire harness loading positioning guidance information, which includes the final wire harness grasping coordinates, the desired grasping direction, the estimated grasping force, and the placement target point.

7. The automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to claim 1, characterized in that, Step S5, based on the wire harness feeding and positioning guidance information, generates a collision-free motion trajectory and end-effector gripping posture for the cable tie machine feeding mechanism through a motion planner, including: S51, take the final wire harness grabbing coordinates and the expected grabbing direction in the wire harness loading positioning guidance information as the target state of the motion planner; S52, acquire the kinematic model of the cable tie feeding mechanism, the current joint state, and the 3D model of obstacles in the workspace; S53, In the motion planner, an improved stochastic path planning algorithm is adopted. Under the premise of considering the constraints of the kinematic model and the obstacle avoidance requirements, a smooth path from the current state to the target state is searched in the workspace, and the smooth path obtained by the search is discretized into a series of continuous path points. S54. For each path point, perform inverse kinematics solution to obtain the angles of each joint of the cable tie feeding mechanism, thereby forming the collision-free motion trajectory in the joint space. S55, simultaneously, based on the desired grasping direction and the configuration of the end effector, the end-effector's grasping posture at the grasping point is calculated.

8. The automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to claim 7, characterized in that, In the motion planner, step S53 employs an improved stochastic path planning algorithm to search for a smooth path from the current state to the target state within the workspace, taking into account the kinematic model constraints and obstacle avoidance requirements. This includes: S531, in the configuration space of the improved random path planning algorithm, the end position and orientation of the cable tie feeding mechanism are used as nodes; S532, when randomly sampling new nodes, a target bias strategy is introduced to sample the target state with a certain probability in order to accelerate convergence; S533, when expanding the tree to connect new nodes, not only does it check whether the straight path collides with obstacles, but it also checks whether the process of moving from the current node to the new node under the kinematic model satisfies the constraints of joint velocity and acceleration. S534, after the extended tree is successfully connected to the target state, the generated initial path is post-processed and smoothed, redundant nodes are removed, and path points are interpolated to ensure the smoothness and executability of the path.

9. The automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to claim 1, characterized in that, In step S6, the collision-free motion trajectory and end-effector gripping posture are converted into a servo control command sequence to drive the cable tie feeding mechanism to complete the picking, positioning, and placement of the wire harness, including: S61, perform time parameterization on the collision-free motion trajectory, and assign a timestamp and the desired end velocity and acceleration to each path point on the trajectory; S62, based on the time-parameterized trajectory and the end-grabbing posture, the desired position, speed and torque of each servo motor of the cable tie feeding mechanism in each control cycle are obtained through real-time inverse kinematics calculation; S63, encapsulate the desired position, speed and torque of each servo motor in each control cycle into a specific servo control instruction sequence according to the specified communication protocol; S64, the servo control command sequence is sent to the motion controller of the cable tie feeding mechanism in real time. The motion controller drives each servo motor to execute, so that the end effector moves along the planned trajectory and completes the picking up of the wire harness at the target point in the desired posture, and then moves to the placement point to complete the placement.

10. The automatic feeding control method for wire harness cable tie machine based on multi-scene positioning according to claim 5, characterized in that, In step S33, based on the normal vector and curvature, feature regions in the point cloud are identified. These feature regions include edge regions with abrupt curvature changes, corner regions with drastic changes in normal vector, and flat regions, including: S331, based on the calculated curvature value of each point, a first curvature threshold is set, and points with curvature greater than the first curvature threshold are initially marked as belonging to the edge region; S332, in the neighborhood of each point, calculate the angle between its normal vector and the normal vectors of all neighboring points in the neighborhood, count the number of neighboring points whose angle is greater than a preset angle threshold in the preset neighborhood, and record this number as the normal vector mutation degree. S333, Set a normal vector mutation threshold, and mark points whose normal vector mutation is greater than the normal vector mutation threshold as belonging to the corner region; S334, For the remaining points that are not marked as edge regions or corner regions, calculate the standard deviation of the curvature values ​​of all points in their neighborhood, set a curvature standard deviation threshold, and mark the points whose curvature standard deviation is less than the curvature standard deviation threshold as belonging to flat regions. S335 performs spatial connectivity clustering analysis on the initially labeled points, aggregating points that are spatially continuous and have the same labeling category, ultimately forming clearly defined edge regions, corner regions, and flat regions.

Images