Manipulator dispensing path optimization method and device based on 3D vision

Abstract

The invention relates to the technical field of 3D vision, and discloses a manipulator dispensing path optimization method and device based on 3D vision, and the method comprises the steps: carrying out the multi-angle three-dimensional vision scanning of a dispensing workpiece, and obtaining the point cloud data of the dispensing workpiece; performing semantic segmentation on the point cloud data, and decomposing the dispensing workpiece into a plurality of functional parts; according to the semantic identifier and the surface feature of the functional component, identifying a target area to be dispensed and a dispensing path constraint condition; generating an initial dispensing path based on the target area to be dispensed and the dispensing path constraint condition; and performing iterative linear secondary adjustment control on the initial dispensing path to generate a motion control instruction. According to the invention, the self-adaptive optimization adjustment of the dispensing path is realized, the problem that the traditional dispensing system cannot cope with the actual state change of the workpiece is solved, and the adaptability and stability of the dispensing system are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of 3D vision technology, and in particular to a method and device for optimizing a robotic arm's dispensing path based on 3D vision. Background Art

[0002] Traditional dispensing path planning relies primarily on CAD models and manual instruction, making it difficult to address practical issues such as workpiece position deviation and surface deformation in industrial production environments, severely limiting dispensing accuracy and efficiency. Particularly for complex curved workpieces, traditional explicit 3D modeling methods suffer from insufficient accuracy and a lack of semantic information, making them unable to meet the demands of high-precision dispensing processes. Furthermore, during the dispensing process, subtle changes in the workpiece's surface geometry, ambient temperature fluctuations, and other external interference factors can cause the actual dispensing path to deviate from the intended path.

[0003] In recent years, deep learning-based 3D visual reconstruction technology has made significant progress. Implicit neural representation methods can more accurately restore workpiece surface details than traditional explicit reconstruction methods. However, current implicit neural representation methods typically reconstruct the workpiece as a whole, ignoring the workpiece's semantic structural information. This results in poor recognition of the workpiece's functional components and accurate dispensing path planning. Especially when dealing with industrial workpieces with multiple materials and complex geometries, the lack of semantic information in the reconstruction results often makes it impossible to accurately identify areas suitable for dispensing and areas to avoid, resulting in unstable dispensing quality, material waste, or product defects. Summary of the Invention

[0004] The present invention provides a method and device for optimizing the dispensing path of a robot based on 3D vision. The present invention realizes adaptive optimization and adjustment of the dispensing path, solves the problem that traditional dispensing systems are unable to cope with changes in the actual state of workpieces, and greatly improves the adaptability and stability of the dispensing system.

[0005] In a first aspect, the present invention provides a method for optimizing a robotic dispensing path based on 3D vision, the method comprising: Performing a multi-angle three-dimensional visual scan on the dispensing workpiece to obtain point cloud data of the dispensing workpiece; Performing semantic segmentation on the point cloud data to decompose the dispensing workpiece into multiple functional components; Identifying target areas to be dispensed and dispensing path constraints based on semantic identification and surface features of the functional components; generating an initial dispensing path based on the target area to be dispensed and the dispensing path constraint conditions; Iterative linear quadratic regulation control is performed on the initial dispensing path to generate motion control instructions.

[0006] In a second aspect, the present application provides a 3D vision-based robot dispensing path optimization device, comprising: a scanning module for multi-angle three-dimensional vision scanning of the dispensing workpiece to obtain point cloud data of the dispensing workpiece; a semantic segmentation module for semantic segmentation of the point cloud data to decompose the dispensing workpiece into a plurality of functional components; an identification module for identifying target regions to be dispensed and dispensing path constraint conditions according to the semantic labels and surface features of the functional components; a generation module for generating an initial dispensing path based on the target regions to be dispensed and the dispensing path constraint conditions; an adjustment control module for performing iterative linear quadratic adjustment control on the initial dispensing path to generate motion control instructions.

[0007] In the technical solution provided by the present application, multi-angle three-dimensional vision scanning and semantic segmentation are used to realize accurate reconstruction and semantic identification of the workpiece, solving the problem of lack of workpiece semantic information in traditional 3D reconstruction, and providing a more accurate workpiece model and semantic features for dispensing path planning. The dispensing path planning method based on workpiece semantic information and surface features realizes high-precision dispensing area identification and constraint condition generation for complex curved surface workpieces, and compared with traditional methods, can more accurately adapt to the geometric changes of the workpiece surface, improving the dispensing quality and material utilization. The combination of iterative linear quadratic adjustment control and adaptive sliding mode control effectively suppresses the model uncertainty and external disturbances in the dispensing process of the five-axis robot, significantly enhancing the robustness and path tracking accuracy of the system. Through the real-time 3D vision feedback mechanism, the workpiece position and attitude changes and surface deformation can be dynamically perceived, realizing adaptive optimization and adjustment of the dispensing path, solving the problem that traditional dispensing systems cannot cope with actual state changes of the workpiece, and greatly improving the adaptability and stability of the dispensing system.

[0008] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and achieved by the structure particularly pointed out in the description, claims, and drawings.

[0009] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are shown as follows. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 An embodiment schematic diagram of the 3D vision-based robot dispensing path optimization method in the embodiments of the present application; Figure 2 This is a schematic diagram of an embodiment of a 3D vision-based robotic dispensing path optimization device in an embodiment of the present invention. DETAILED DESCRIPTION

[0011] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0012] The terms "including," "having," and any variations thereof, as used in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or device.

[0013] To facilitate understanding of this embodiment, a 3D vision-based robot dispensing path optimization method disclosed in an embodiment of the present invention is first introduced in detail. Figure 1 As shown, this method includes the following steps: 101. Perform multi-angle three-dimensional visual scanning on the dispensing workpiece to obtain point cloud data of the dispensing workpiece; It is understandable that the execution subject of the present invention can be a 3D vision-based robot dispensing path optimization device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0014] Specifically, a three-dimensional perception system with high coverage and high perspective redundancy is designed around the five-axis robot workbench, which is composed of multiple high-resolution industrial cameras and is arranged around the workpiece to be measured at equal angles to ensure that every area of the workpiece surface is observed simultaneously by multiple perspectives during the scanning process, improving the overall three-dimensional information redundancy and image matching reliability. Eight 4096x3072 pixel cameras are arranged at an angle of 45° on the edge of the workbench, and these cameras are integrated into a unified vision control system. By controlling the exposure time and trigger signal, all cameras can synchronize image acquisition at the same time, eliminating image timing errors caused by changes in lighting or mechanical vibration and obtaining consistent multi-perspective image data. The multi-perspective image data is preprocessed, and each image is sequentially processed with Gaussian filtering to suppress noise, enhanced local contrast through adaptive histogram equalization, and detailed contours extracted using edge enhancement algorithms such as the Sobel operator. By setting an edge gradient threshold, the surface features of the workpiece are highlighted, and a sequence of preprocessed images is obtained. The preprocessed image sequence is input into a stereo matching algorithm based on the semi-global matching strategy. Based on setting a reasonable disparity search range and aggregation window, the pixel disparity map between image pairs is calculated, and a smoothing regularization term is introduced to suppress matching jumps, making the generated target disparity map have both local accuracy and global consistency. After obtaining the target disparity map, the internal and external parameters of each camera are combined with the previously calibrated parameters, and the image disparity is inverted into the depth value of the three-dimensional space point using the triangulation principle, completing the conversion of pixel coordinates to world coordinates and generating an initial point cloud dataset. The initial point cloud data is processed to remove noise and fill holes. Discrete points are removed using radius filtering or statistical outlier rejection algorithms, and missing information is completed in low-density areas using interpolation algorithms to obtain a dense and complete point cloud model. The dense and complete point cloud data is processed to calculate the normal vector and analyze the curvature. The normal vector is calculated using least squares plane fitting in the neighborhood of each point, and the curvature analysis is based on the spatial structure change between neighborhood points to obtain the Gaussian curvature and mean curvature distribution through eigenvalue decomposition, obtaining key point cloud data.

[0015] 102. performing semantic segmentation on the point cloud data to decompose the dispensing workpiece into multiple functional components; Specifically, the point cloud data is input into a multi-layer convolutional neural network structure, which uses a point cloud convolution module to perform local neighborhood sampling on three-dimensional point distribution, and extracts high-dimensional feature representation of spatial structure and surface geometry through a number of layers of gradually deepening convolution operations, wherein the convolution kernel size and the number of channels increase layer by layer to adapt to multi-layer information fusion from detailed features to overall shape, to obtain a deep semantic feature vector corresponding to each point. The point cloud deep features are subjected to spatial pyramid pooling processing, the feature maps at different scales are partitioned according to the spatial position for pooling, and the local small-range and large-range feature responses are extracted respectively to construct a multi-scale semantic representation structure and improve the recognition ability of the network for large-scale planes and small-scale edge details. The multi-scale features are input into a fully connected classification layer, and each point is assigned an initial semantic label representing the preliminary attribution of the point in the global workpiece structure, such as a frame, a shell, a mounting surface, a corner area, etc., and a point cloud semantic sketch is formed. Due to the irregularity of the point cloud and the interference of the edge noise, the point cloud semantic sketch is subjected to global consistency optimization. Based on the initial label, a graph structure is constructed, wherein each point is a graph node, and the edges between the nodes define the weight according to the Euclidean distance or the feature space distance. The similarity and adjacency relationship between points are modeled through the edge weight of the graph structure, and then the label smoothing operation is performed on the graph structure. On the premise of keeping the labels of high-confidence areas unchanged, the graph regularization mechanism is used to complete and correct the low-confidence areas to obtain a label graph with optimized semantic consistency. Based on the optimized label, the whole point cloud data is divided into semantic regions with clear boundaries and topological connectivity, i.e. candidate functional components, and topological relationship analysis is performed on each component, including connectivity judgment, boundary curvature inspection, neighborhood continuity identification, etc. Through these analysis, the functional role and geometric attribute of each candidate component in the workpiece structure are identified. The boundaries of each candidate component are finely processed by increasing sampling and edge reconstruction in the local normal vector change area to enhance the accuracy and continuity of the component partition, and the point cloud data is divided into a plurality of functional units with clear boundaries, stable structure and accurate semantic labels.

[0016] 103. Identifying the target area and the dispensing path constraint condition according to the semantic identification and the surface feature of the functional component; Specifically, on the basis of the completed semantic segmentation and the three-dimensional point cloud of the functional component identification, the surface points of each component are analyzed jointly with the Gaussian curvature and the average curvature to reveal the micro-geometric variation trend. By establishing a normal vector variation model in the local neighborhood of each point and calculating the principal curvature direction and numerical difference, the Gaussian curvature (as a product feature of surface curvature intensity) and the average curvature (as a mean feature of the overall bending degree of the surface) of the point are obtained respectively, forming a curvature distribution map covering the surface of the functional component. The original semantic identification of the functional component is introduced into the feature fusion model, and different surface feature weights are configured for different semantic regions (for example, the curvature sensitivity is enhanced for the edge region of the assembly hole, and the normal vector smoothness is enhanced for the shell plane region), so that the surface features complete the structure sensitivity enhancement under the guidance of the semantic prior. The fused feature response value is matched with a set of process-related dispensing adaptability thresholds, and the entire surface is divided into regions according to the curvature, normal vector variation rate and neighborhood stability, etc. A batch of candidate regions with high dispensing adaptability are selected. The normal vector continuity detection is performed on the candidate regions to determine whether there are sharp posture changes, corner mutations and other discontinuous regions that are not suitable for continuous dispensing in the region; at the same time, based on the normal vector discontinuity and local curvature mutation, edge detection and feature line extraction are implemented to extract the key structure lines and boundary corner points of the region boundary. On this basis, the candidate regions that are close to each other, consistent in direction or continuous in boundary are merged, and the regions with significant boundary changes are segmented to ensure that the extracted target regions for dispensing have continuity in geometry and functional independence in semantics. The reachable analysis and workspace mapping of the mechanical arm are performed on each target region to determine whether the region can be accessed by the dispensing head in a suitable posture under the current five-axis mechanism, and to exclude unreachable regions caused by joint limits, motion singularity or space occlusion; at the same time, the path playback and environment modeling collision detection are performed on the dispensing head by combining the virtual simulation module to exclude path point sets with interference or insufficient angle, and to generate safe dispensing distance constraints (i.e. the minimum and maximum distances between the nozzle and the workpiece surface) and posture constraints (i.e. the maximum angle between the nozzle axis and the surface normal vector) with safety redundancy, to obtain a safe dispensing region. According to the actual morphological characteristics of the workpiece surface and the specific process requirements (such as the type of glue used, the required line width, the glue thickness, etc.), the dispensing control parameters and motion execution parameters of each path segment are set, including dispensing speed, glue discharge frequency, nozzle deflection angle and glue amount dynamic adjustment coefficient, etc. These parameters are dynamically adjusted according to the curvature variation of the target region surface, for example, the motion speed is reduced or the glue discharge frequency is increased at the curvature mutation point to ensure the continuity and adhesion quality of the glue trace. Finally, the obtained dispensing path constraint conditions include the boundary and normal limit in geometric space, and are fused with the dynamic characteristics of the dispensing process and execution device.

[0017] 104. generating an initial dispensing path based on the target region to be dispensed and dispensing path constraint conditions; Specifically, the target region to be dispensed is equidistantly sampled to generate a series of dispensing position candidate point sets covering the target surface. The spacing of these sampling points is adjusted according to the glue line width, process required line density and surface curvature to ensure that the distance between points is uniformly distributed within the allowable range of the structure. After sampling, each candidate point is subjected to path constraint adaptability detection, i.e. whether the point meets a series of geometric and process constraint conditions such as the minimum distance limit between the nozzle and the workpiece surface, the angle limit between the nozzle axis and the normal vector, the edge avoidance requirement, the dispensing speed threshold limit, etc. to eliminate candidate points that do not meet the conditions and obtain an effective dispensing position set after screening. On this basis, the A* algorithm in the heuristic search strategy is used to perform path search on the effective dispensing position set. By introducing the cost function between each path point, combining path geometric length, corner change rate and glue amount distribution continuity as search heuristic factors, a path candidate set that meets the reachability and process continuity between points is constructed. The A* algorithm can effectively avoid the constraint boundary and select a locally optimal connection strategy in areas with sharp curvature changes to improve the structural rationality of the path. The preliminary path candidate set is input into the particle swarm optimization algorithm as the initial solution for global optimization search. In this optimization process, each path is encoded as a particle, and the fitness of the particle is determined by the weighted combination value of the four objective functions: path total length, turning smoothness index (such as second derivative change rate), dispensing line glue amount distribution consistency error and nozzle and surface following error. In the optimization process, while ensuring the search diversity between particles, the individual experience and the global optimal solution are mutually driven to gradually approach the optimal path sequence, obtaining the optimal path that is globally optimal, locally smooth, uniformly distributed in glue amount and fits the workpiece surface. After optimization, the path point sequence is input into the curve fitting module, and a quintic B-spline function is selected to model the path as a whole. The quintic B-spline has high curvature continuity and smoothness, and by flexibly adjusting the control point position and number, high-precision fitting of the original path is achieved. The system performs equidistant interpolation distribution on the key control points in the path sequence, and then performs local control point densification and refinement according to the curvature change intensive area to adapt to the path adjustment requirements of high curvature areas, forming a continuous, smooth and process controllable dispensing curve. After completing the curve construction, the geometric derivative information of the quintic B-spline curve is used to perform acceleration and deceleration planning, and the nozzle attitude is designed according to the curve tangent vector and normal vector to ensure that each path point not only has accurate position, but also carries corresponding speed size, speed change rate and nozzle attitude parameters, ensuring motion stability, attitude continuity and dispensing consistency during path execution. The final generated initial dispensing path is a complete path sequence that is structurally continuous, motionally smooth, attitude-matched and process-adapted.

[0018] 105. performing an iterative linear quadratic regulation control on the initial dispensing path to generate the motion control instructions.

[0019] Specifically, the initial dispensing path is converted from the Cartesian space form to the joint space trajectory recognizable by the five-axis robot, which is achieved by inverse kinematics calculation, mapping each spatial pose point in the path to the corresponding five joint angle values, and then deriving the joint angular velocity, angular acceleration, and end-of-path dispensing nozzle pose based on the time parameter to obtain a robot dynamics state expression model containing joint angle, angular velocity, and end-of-path pose. In the vicinity of the working point of the current path segment, the nonlinear dynamics model is linearized, the local linear approximation of the state change of the system is obtained by using the small disturbance assumption, and the linear time-varying system description is obtained. Based on the Lagrange mechanics principle, the dynamic characteristic parameter matrix is constructed, including the inertia matrix, the Coriolis torque item, the gravitational potential energy item, and the static / dynamic friction torque item, ensuring that the linear model not only expresses the local physical dynamics response of the system, but also accurately captures the time-varying nature of the dynamic characteristic changes in the path execution process. After the linear model is established, according to the accuracy and stability requirements of the dispensing trajectory control, a weighted quadratic performance index function is defined, which includes position tracking error, joint velocity error, and control input cost. The three are set as the minimum target, and the weight matrix is set to control the punishment intensity of each item, with the position error weight much higher than the control energy weight to ensure that the dispensing nozzle trajectory is highly consistent with the set path. The controller performs a typical iterative linear quadratic regulator calculation process. Under the given target function and system model, forward prediction is performed first, i.e. the future state is predicted based on the current control input and the dynamics model to obtain the complete state trajectory, and then backward recursion is performed based on the state trajectory to obtain the control gain matrix and feedback law through iterative derivation of the Riccati equation, generating a locally optimal control increment sequence that can achieve optimal adjustment of the current state to the target state. Due to model uncertainty and external disturbances in the system during execution, a local solution is not enough to cover the global path control accuracy requirements, so the local control increment sequence is fed back to the original dynamics model for a complete state update prediction, and the error between the predicted state and the target state is calculated. Based on this error, the linearization point and system parameters are updated, and the linearization process, state prediction, and Riccati equation recursion steps are re-executed. This cycle continuously reduces the prediction error and adjusts the control gain matrix in each iteration, and finally completes the solution of the global optimal control strategy under the convergence condition, obtaining a set of optimal control laws that can accurately track the entire path and dynamically compensate for disturbances. Based on the optimal control law, a composite control structure is designed, with the nominal control term generated by the global path planning as the main control input, and a robust control term as the external disturbance and modeling error compensation mechanism, where the robust control term is based on the sliding mode surface structure or adaptive term to suppress uncertainty.In the control execution layer, the torque command and joint speed command are transmitted to the servo driver of each joint of the robot at a high frequency using a real-time communication protocol such as EtherCAT or Profinet, each control cycle responds within 1 millisecond, and the actual output state of the driver is continuously monitored through the feedback channel, and the state is updated in a closed loop with the controller, so as to realize dynamic accurate control of the whole system, stable tracking of the path and high consistency of the dispensing process output.

[0020] In this embodiment, according to different requirements of trajectory error, speed deviation and control energy in the optimization objective function, the weight coefficients of terminal state and intermediate process are set, the influence degree of each control step on the overall path execution quality is determined, and combined with the dynamic structure of the system, an augmented description matrix is formed, which can reflect the sensitivity of the state change of the robot, the change of the control input and the deviation of the target. On this basis, the recursive framework of Riccati equation is constructed, and the control cost propagation state corresponding to each time node is calculated by step-by-step deduction from the end time point of the path. This process constitutes a time-reversed dynamic optimization sequence, which gradually feeds back the influence of the final target deviation to the current control decision, and establishes an updateable control gain basis for each node. The time-varying control cost mapping sequence is combined with the current robot dynamics model to generate the feedback control gain parameters and the feedforward adjustment term at each time. This control law enables the controller to automatically generate the optimal control amount at each time point according to the current state, which is used to drive each joint to complete the next movement. In order to verify the actual effect of the control law, the current state is input into the feedback controller, and the forward calculation is performed combined with the dynamics model to predict the system state at future time points and generate a trajectory sequence, which represents the motion prediction result of the robot under the current control strategy. The optimized trajectory is compared with the expected ideal path, the trajectory error is analyzed, and a state correction mechanism is constructed based on the change rule of error distribution in different time periods. This mechanism realizes the secondary optimization of control output by adjusting the response amplitude of some key links in the control input. The corrected state trajectory is input into the control law module to recalculate the control input, improve the accuracy of path tracking and the stability of control process. The whole process takes error minimization as the core and state update as the driving force, and the prediction, feedback and correction stages are executed in a loop until the control sequence converges, that is, the control amount changes very little in continuous iteration, the system is stable and has execution, and a set of locally optimal control sequence with optimality of current state and matching of system dynamics is output to guide the robot to execute the actual dispensing path.

[0021] In the embodiment of the present application, the precise reconstruction and semantic recognition of the workpiece are realized through multi-angle three-dimensional visual scanning and semantic segmentation, solving the problem of lack of workpiece semantic information in traditional 3D reconstruction, and providing a more accurate workpiece model and semantic feature for dispensing path planning. The dispensing path planning method based on workpiece semantic information and surface features realizes high-precision dispensing area recognition and constraint condition generation for complex curved surface workpieces, and compared with traditional methods, can more accurately adapt to the geometric changes of the workpiece surface, improving the dispensing quality and material utilization. The composite control strategy combining iterative linear quadratic regulation control and adaptive sliding mode control effectively suppresses the model uncertainty and external disturbance in the dispensing process of the five-axis robot, significantly enhancing the robustness and path tracking accuracy of the system. Through the real-time 3D visual feedback mechanism, the workpiece position and posture changes and surface deformation can be dynamically perceived, realizing adaptive optimization and adjustment of the dispensing path, solving the problem that the traditional dispensing system cannot cope with the actual state changes of the workpiece, and greatly improving the adaptability and stability of the dispensing system.

[0022] In a specific embodiment, the process of step 101 can specifically include the following steps: A plurality of industrial cameras are evenly distributed around the five-axis robot workbench, and the dispensing workpiece is synchronously imaged from different angles by the plurality of industrial cameras to obtain multi-view image data; The multi-view image data is subjected to Gaussian filtering, adaptive histogram equalization and edge enhancement processing to obtain pre-processed multi-view images; The pre-processed multi-view images are input into a stereo matching algorithm for disparity calculation to obtain a target disparity map; According to the camera calibration parameters and the target disparity map, coordinate conversion is performed through the principle of triangulation to obtain initial point cloud data; The initial point cloud data is subjected to noise filtering and hole filling processing to obtain dense and complete point cloud data, and the dense and complete point cloud data is subjected to normal vector calculation and curvature analysis to obtain the point cloud data of the dispensing workpiece.

[0023] Specifically, according to the geometric layout of the robot workspace, the attitude variation range of the workpiece, and the surface feature complexity involved in the dispensing task, a multi-camera distribution structure with spatial coverage and angle redundancy is designed. In a ring distribution manner, several high-resolution industrial cameras are installed on the edge of the robot workbench or the upper support position, observing the key parts of the workpiece surface from multiple directions at the same time. In order to avoid visual blind area and improve the accuracy of parallax reconstruction, a certain angle is maintained between these cameras, which are arranged at equal intervals, so that each surface area is observed by at least three cameras. At the control system level, a unified trigger module is used to synchronize the control of all industrial cameras, ensuring that all cameras complete image acquisition at the same time node, eliminating time jitter caused by environmental light changes or equipment response differences, and obtaining consistent multi-view image data. For each image, Gaussian filtering is performed to filter out high-frequency noise components in the image, reducing interference caused by lighting, sensor response nonlinearity, or background texture, thereby enhancing the structural clarity of the image. After filtering, adaptive histogram equalization is used to enhance the local contrast of the image, especially on workpiece surfaces with both bright and dark areas. The adaptive processing automatically adjusts the gray scale mapping range for different image areas, thereby improving the overall detail performance. To enhance key visual elements such as workpiece boundaries and concave-convex features, edge enhancement operations are performed, including gradient operator or edge detection operator processing, such as using a directional edge template to emphasize gray level changes in the image, making the output image have stronger feature expression ability in the texture level. After the above multi-level processing, the multi-view images have uniform lighting style, balanced gray scale distribution, and clear boundary features, providing high-quality input conditions for three-dimensional reconstruction. After image preprocessing, the images are input into the stereo matching module for disparity calculation. The corresponding relationship between the same physical point in different views is established, and the disparity value is calculated accordingly to infer the depth information. To improve matching accuracy, a semi-global stereo matching algorithm is used, which introduces a global path smoothing mechanism based on local cost aggregation, suppressing noise propagation while maintaining clear edges. During algorithm execution, a matching cost matrix is constructed based on pixel intensity or gradient information between image pairs, then the possible disparity values are matched between left and right views within a certain search range, and the final optimal disparity map is calculated by combining path cost smoothing rules. The disparity map represents the pixel displacement size between left and right views at different pixel points. Based on the target disparity map, combined with the intrinsic and extrinsic parameters of the camera, the two-dimensional disparity information on the image is converted into the position coordinates of the three-dimensional space point through the principle of triangulation. The intrinsic parameters of the camera describe the lens focal length, principal point coordinates, distortion parameters, and other optical properties, while the extrinsic parameters describe the position and orientation of the camera in the world coordinate system. These parameters are obtained in advance through Zhang Zhengyou calibration or chessboard-based calibration, and remain stable during the entire system operation.The coordinate conversion process reflects the parallax value as a depth value and back-projects the pixel points into a three-dimensional space to obtain dense initial point cloud data. The initial point cloud data is subjected to noise filtering and hole filling processing. Outlier points in the point cloud are removed using filtering methods based on density judgment or statistical analysis, such as radius filtering, statistical outlier removal, etc., effectively removing points that are sparse or spatially abnormal. In areas where the local point cloud is sparse or missing, an interpolation completion mechanism is introduced to construct a local fitting surface based on the geometric structure of adjacent points and fill in the missing points, generating a dense point cloud model with better topological continuity. On the basis of this high-quality point cloud, normal vector calculation is performed to estimate the surface tangent characteristics of each point based on the spatial distribution within its neighborhood, and the normal direction of all points is uniformly corrected to have the ability to express the direction of the curved surface. Finally, curvature analysis is performed to extract the Gaussian curvature and mean curvature of each point based on the second-order characteristics of the normal vector change rate and spatial structure, and a curvature distribution map reflecting the micro-geometric undulations of the workpiece surface is constructed.

[0024] In a specific embodiment, the process of performing step 102 can specifically include the following steps: Input the point cloud data of the point gluing workpiece into a multi-layer convolutional neural network for feature extraction to obtain point cloud deep features; Perform spatial pyramid pooling processing on the point cloud deep features to obtain multi-scale feature representations; Input the multi-scale feature representations into a fully connected layer for classification processing to obtain initial semantic labels of the point cloud data; Construct a graph structure based on the initial semantic labels, and perform edge weight calculation and label smoothing based on the graph structure to obtain labels that are optimized for semantic consistency; According to the labels that are optimized for semantic consistency, divide the point cloud data into regions with clear boundaries to obtain candidate components of the point gluing workpiece; Perform topological relationship analysis and boundary refinement processing on the candidate components to obtain a plurality of functional components with clear semantic labels.

[0025] Specifically, the three-dimensional point cloud data of the dispensing workpiece is input into a deep neural network with point cloud structure processing capability. Traditional rule-based image convolution network structure is difficult to adapt to the non-structural characteristics of point cloud data. A multi-layer convolutional neural network specially designed for point cloud data is adopted to extract local features based on points. A plurality of local coordinate systems are constructed through in-neighborhood sampling and normalization operations, and layer-by-layer convolution calculations are performed based on the local coordinate systems to extract the expression ability of the point cloud in multiple dimensions such as spatial geometry, surface normal and curvature change. The network adopts a stacked structure to capture the geometric adjacency relationship between points in the shallow layer, aggregate the structural features of the patch scale in the middle layer, and extract the global spatial layout and semantic pattern in the deep layer to obtain a point cloud deep feature representation with high semantic density. Spatial pyramid pooling processing is performed on the point cloud deep features. Spatial structure information at different scales is introduced into the feature representation to enhance the robustness of the network to size changes, local density unevenness and structural asymmetry. The entire point cloud data is divided into a plurality of sub-regions in different spatial division manners, and then the features in each sub-region are pooled. The local feature distribution is compressed by maximum pooling, average pooling or weighted pooling, and the pooling results at different scales are combined to form a unified multi-scale feature description vector. The multi-scale feature representation is input into a fully connected network structure for point-level semantic classification processing. Through a plurality of fully connected layers decreasing layer by layer, each point is mapped to a semantic prediction vector, each dimension of the vector corresponds to a potential semantic category, and each point is assigned an initial semantic label according to the dimension where the maximum response value is located. The labels include assembly faces, shells, connecting edges, screw hole regions, glue line channels and other common structural units in industrial dispensing tasks. Since this classification is based on independent mapping in the feature space and has not yet considered the coherence and spatial consistency between local structures, a point relationship graph structure is constructed after the label output to optimize the initial semantic distribution. The construction of the graph structure is based on the relationship between the point and its spatial neighborhood, each point is regarded as a node in the graph, and the edges are connected through the geometric distance, feature similarity or normal angle between the point and its neighboring points, each edge is attached with a weight value reflecting the credibility of the point in the semantic propagation of its neighborhood. On this basis, the label smoothing operation is performed, the high-confidence label region is taken as the propagation source through the graph regularization mechanism, the label information is transmitted and harmonized along the edge weight, and the low-confidence region is corrected. This process effectively eliminates isolated points and boundary jumps in the initial semantic labels, improves the consistency and spatial coherence of the overall semantics, and forms a more stable semantic label output structure. After the semantic labels are stable, the point cloud data is divided into a plurality of local regions with clear boundaries, each region contains a set of adjacent and same semantic points, and is regarded as a candidate part. The division operation is based on the consistency of the semantic categories and combines the spatial connectivity, curvature continuity and boundary clarity to ensure that the division result has structural integrity and boundary distinguishability.After the division, topological relationship analysis is performed on each candidate component to construct a topological structure diagram between components by analyzing connection boundaries, contact areas, relative position relationships, and direction consistency between adjacent components. Meanwhile, boundary refinement is performed inside each candidate component to enhance boundary point distribution density, reconstruct edge point connection order, eliminate boundary sawtooth phenomena, and the like, so as to improve the structural quality and geometric expression accuracy of the component contour. After all the processing is completed, each component has complete three-dimensional point cloud expression, clear boundary contour, and is assigned a clear semantic identifier, such as a main support surface, a dispensing track surface, an assembly slot, or a constraint frame.

[0026] In a specific embodiment, the process of performing step 103 can specifically include the following steps: Gaussian curvature and average curvature calculation is performed on each point in the functional component to obtain a surface curvature distribution map; Surface feature weight is obtained by performing weight configuration on the surface features according to the semantic identifier of the functional component, and the candidate region to be dispensed is obtained by performing region division according to the surface feature weight and a preset dispensing adaptability threshold; Region boundary and feature line information is obtained by performing normal vector continuity detection and edge feature extraction on the candidate region, and the target region to be dispensed is obtained by performing region merging and segmentation processing according to the region boundary and feature line information; Safety dispensing regions are obtained by performing reachability analysis and collision detection on the target region to be dispensed to determine dispensing distance constraints and posture constraints; Dispensing path constraint conditions are obtained by setting glue amount control parameters and motion speed parameters based on the surface characteristics and process requirements of the safety dispensing regions.

[0027] Specifically, on the basis of completing semantic segmentation and obtaining the three-dimensional point cloud of each functional component, a surface fitting model is established for each point in its local neighborhood, and the Gaussian curvature and mean curvature of the point are calculated based on this. The Gaussian curvature reflects the product characteristics of the surface at the principal curvature direction of the point, and is a key indicator for describing convexity and regional curvature properties. The mean curvature describes the overall bending trend of the point in all directions, and the combination of the two forms a curvature distribution map covering the entire component, which depicts the degree of geometric undulation, edge concave-convex area and transition zone smoothness of the component surface. According to the semantic identifier carried by each functional component, the curvature data is weighted and configured, that is, different curvature importance factors are set for different categories of functional components. For example, for the shell surface that needs to be glued on the plane, the surface curvature stability and low value area weight are higher, while for the linear feature dispensing area, the response sensitivity of the curvature change rate should be improved. Through the feature weighting mechanism based on semantic prior, a set of surface feature weight maps reflecting geometric adaptability and structural functionality are generated. The weight map is compared with the previously set dispensing adaptability threshold, and a conditional judgment is performed on all points, and the points are grouped and classified according to the set response rule, so as to divide a set of candidate dispensing areas with good geometric continuity, surface stability and structural integrity. For the delineated candidate area, a normal vector continuity detection operation is performed, the angle change between the normal vector direction of each point and the normal vector of the surrounding points is calculated in the neighborhood of each point, and the continuity threshold is used as the basis for judgment to identify the boundary sections with normal vector mutation or inconsistent direction. At the same time, combined with the curvature change trend and the normal vector distribution gradient, the feature lines of these boundary sections are extracted, and the feature line set representing the region edge, structure mutation and functional boundary is extracted through continuous point connection, direction difference clustering and other ways. Based on the extracted boundary information and feature line trend, the candidate area is further processed by merging and segmentation. For adjacent regions with similar boundaries, consistent feature line trends and continuous normal vectors, fusion processing is performed to form larger structure consistent regions; while for regions with significant boundary fracture or feature line crossing, fracture segmentation operation is performed to ensure that the target region has boundary closure, functional consistency and curvature continuity in structure, and the target dispensing region with complete geometric definition and structural semantics is obtained.For these target areas, kinematic accessibility analysis and path feasibility judgment are completed, the position, size and attitude information of the target area are mapped to the robot joint space, and the kinematic accessibility calculation is performed based on the robot motion range, joint limit, mechanism singularity and other parameters to identify the unreachable points and redundant degree of freedom space in the path planning process; At the same time, a virtual collision model is constructed, the dispensing head, robot body and workpiece are simulated in the simulation environment, and the dynamic collision possibility in the dispensing path execution process is evaluated, the areas with structural interference, nozzle inclination angle too large or insufficient angle are excluded, and a set of safe dispensing areas meeting the kinematic execution conditions, sufficient attitude control space, reasonable glue dispensing distance and sufficient safety margin are obtained. On this basis, according to the local curvature distribution, normal stability, area characteristics and dispensing process requirements of each safe area, the glue amount control parameters and motion speed parameters required during path execution are automatically set, such as reducing the motion speed in the area with sharp curvature change to avoid glue stringing, adjusting the nozzle angle in the area with large surface normal fluctuation to improve the adhesion stability, setting the equal glue spacing in the large flat area to control the glue uniformity, and setting the path and boundary minimum distance combined with the edge avoidance mechanism, A set of dispensing path constraint conditions are constructed, including path segment maximum speed, minimum radius, nozzle distance to target surface distance, attitude angle range and glue frequency change limit, etc., which are used to guide the whole process of path generation, optimization and control scheduling to ensure the stable, efficient and accurate execution of dispensing task under high complexity surface and multi-structure interference.

[0028] In a specific embodiment, the process of performing step 104 can specifically include the following steps: Equidistant sampling is performed on the target area to be dispensed to obtain dispensing position candidate points, and the dispensing position candidate points are screened according to the dispensing path constraint conditions to obtain a set of effective dispensing positions; An A* algorithm is used to search paths for the set of effective dispensing positions to obtain a set of path candidates meeting the constraint conditions; The set of path candidates meeting the constraint conditions is input into a particle swarm optimization algorithm, and the path length, turning smoothness, glue uniformity and surface following accuracy are taken as optimization objectives for global optimization to obtain an optimized path sequence; A quintic B-spline curve is constructed based on the optimized path sequence, and the control points are uniformly distributed and locally adjusted based on the quintic B-spline curve to obtain a smooth and continuous dispensing curve; The smooth and continuous dispensing curve is subjected to acceleration and deceleration planning and attitude planning, and each path point is assigned a speed parameter and a tool attitude parameter to obtain an initial dispensing path.

[0029] Specifically, within the identified and determined target region range, a uniform two-dimensional or three-dimensional sampling grid is established based on spatial geometric coordinates. The grid distributes sample points on the target surface according to a set step size, ensuring consistent spacing between each sample point. The uniform distribution of points ensures the basic connectivity and uniformity of the subsequent dispensing path. In areas where the surface curvature changes dramatically or the edges are complex, the system appropriately reduces the sampling step size to improve local path adaptability and retains high-density point columns at key boundaries to support subsequent path refinement reconstruction. After generating the initial candidate point set, each point is subjected to multidimensional screening according to the previously set dispensing path constraint conditions, including determining whether the minimum distance between the point and the workpiece edge meets the safety boundary limit, whether the angle between the nozzle and the surface normal is within the attitude constraint range, whether the curvature around the point exceeds the allowable path curvature threshold, whether the normal continuity around the point meets the path smoothness requirement, and whether the region where the point is located has the required dispensing distance control space. Through process and geometric comprehensive judgment, invalid points that do not meet the execution conditions are screened out, and a set of valid dispensing positions that fully meet the dispensing execution specifications is obtained. A* algorithm is used to search for a path in the space graph constructed by the valid points. In the graph construction process, each valid point is considered as a node in the graph, and the connection relationship between adjacent points is defined as a feasible edge. A heuristic cost function is constructed based on factors such as Euclidean distance between points, surface normal difference, and curvature change rate. This cost function integrates path length, surface smoothness, and structural safety indicators. When executing path search, A* algorithm prioritizes the direction with the smallest total cost, quickly constructing a path sequence from the starting point to the end point that meets all constraint conditions in the point set. This sequence consists of multiple control points and has connectivity, obstacle avoidance, and preliminary smoothness in structure, forming a set of path candidates. To improve the overall performance of the path, the path candidate set generated by the A* algorithm is input into the particle swarm optimization algorithm for global optimization. In the particle swarm model, each path is represented as a particle, and each dimension of the particle represents a control point coordinate in the path. The positions and velocities of all particles are initialized, and then the particles are guided to search for the optimal solution in the search space in a group collaborative manner through the iteration process. In each iteration, the fitness score of each path is calculated based on four optimization objectives, including path length, angle change amplitude of path turning points, consistency of glue distribution between adjacent path segments, and path fitting accuracy on the target surface. These scores are combined into the overall performance indicator of the path according to the pre-set weight coefficients, and are used to update the global optimal and individual optimal states of the particles. After multiple iterations, a set of optimized path sequences is obtained, which are structurally stable, have consistent glue distribution, good surface following performance, and continuous turning angles. After optimization, the optimized path point series is input into the curve fitting module, and a five-order B-spline function is selected for overall path modeling.Compared with low-order spline curves, the quintic B-spline has a higher continuity order, which can effectively suppress path fluctuations and improve the smoothness and adaptability of the path in the local curvature variation area. During the curve construction process, the control points are resampled to make them equidistantly distributed along the path. Then, the control points are encrypted in the high variation area according to the local curvature information to improve the fitting accuracy, and the control points are thinned in the low variation area to reduce the number of control points, forming a point gluing path curve that is geometrically continuous, structurally smooth, and overall controllable, providing an ideal input trajectory for robot motion control. The quintic B-spline curve is executed for acceleration and deceleration planning and posture planning operations. The acceleration and deceleration planning generates a speed value corresponding to each path point based on the path segment length, curvature variation, and glue spraying time parameters, ensuring that the robot can achieve flexible start and stop and smooth speed transition during execution. The posture planning dynamically adjusts the tool's attitude angle based on the relationship between the path tangent vector and the surface normal vector, ensuring that the nozzle's orientation and the surface remain stable and consistent throughout the path, avoiding glue deviation or unevenness caused by angle errors. Each path point is assigned a clear position coordinate, speed instruction, and attitude instruction, forming an initial point gluing path data set.

[0030] In a specific embodiment, the process of performing step 105 can specifically include the following steps: M1: Convert the initial point gluing path to the joint space trajectory of the five-axis robot, and construct a robot dynamics model containing joint angles, angular velocities, and point gluing nozzle position and attitude; M2: Linearize the robot dynamics model at the current working point, and construct a linear time-varying system matrix containing inertia matrix, Coriolis force moment, gravity moment, and friction force moment using the Lagrange algorithm; M3: Based on the linear time-varying system matrix, take the position tracking error, velocity error, and control input as the penalty term of the quadratic cost function, and set the weight matrix to obtain the optimal control objective function; M4: Based on the optimal control objective function, iteratively calculate the control gain matrix by forward calculating the state equation and backward recursively calculating the Riccati equation to obtain the local optimal control sequence; M5: Apply the local optimal control sequence to the robot dynamics model to predict the system state, then update the linearization parameters based on the prediction error, and repeat steps M2 to M4 until the control sequence converges to obtain the global optimal control strategy; M6: Based on the global optimal control strategy, construct a composite control law containing a nominal control term and a robust control term, and transmit the torque command and speed command to the robot drive system through a real-time communication protocol to output the motion control command.

[0031] Specifically, each Cartesian space coordinate point in the initial dispensing path is converted into a joint space trajectory that the robot itself can execute, which is accomplished by a reverse kinematics solver module. According to the position and attitude changes of the dispensing path in three-dimensional space, the required joint angle combination of the five-axis robot at each time is back calculated. At the same time, the joint angular velocity and the end point dispensing nozzle attitude change are calculated by interpolation between path segments at each time step to obtain a complete joint space trajectory containing position, velocity and attitude information. A complete nonlinear dynamics model of the robot is constructed based on this, which describes the mapping relationship between joint input and end motion. During the trajectory tracking process of the system, in order to improve the control accuracy and real-time performance, the nonlinear dynamics model is locally linearized at the current time point, taking the current system state as the linearization base point. The partial derivative information of the model is extracted to construct a local first-order approximation model, forming a linear expression structure of the state transition equation and the input control equation. After linearization, the Lagrange modeling method is used to introduce the geometric parameters and dynamic parameters of the robot structure into the calculation to establish a complete dynamics matrix that contains inertia terms, Coriolis terms, gravity terms and friction terms. These matrices dynamically change with time and state, so they are organized as linear time-varying system matrices, which express the dynamic response characteristics of the robot under different states. Based on the linear time-varying system matrix, the position tracking error, the velocity error and the control input are taken as the penalty terms of the quadratic cost function with a weight matrix. The position error reflects the tracking accuracy of the dispensing path, the velocity error controls the system dynamic transition behavior, and the control input penalty term is used to limit the energy consumption and instability risk caused by the rapid change of the actuator input. All error terms correspond to the set weight matrix, and these weights determine the relative importance of each term in the objective function, forming a set of adjustable, optimized and measurable multi-objective performance index system. To solve this control objective function and obtain the optimal control instruction sequence of the current path segment, the iLQR method is executed by using the iterative optimization strategy. This method performs forward recursion through the system state equation, i.e. starting from the current initial state, the state trajectory of the system in the future time steps is predicted under the condition of given initial control input. Then backward recursion is performed, i.e. starting from the terminal time point, the control gain evolution process is solved in reverse, the control gain matrix is updated through the recursion method, and the control law is corrected to minimize the state evolution cost of the system between the current position and the target position, obtaining a set of optimal control increment sequence that can be implemented within a local time window. This control sequence has dynamic consistency, local optimality and path tracking convergence in the mathematical sense.After feeding this local control sequence back into the robot's dynamic model, forward state prediction is performed again based on the updated control input. An error analysis is performed between the new predicted state and the ideal path state. The parameters of the linearized model are adjusted in real time based on the error results, and a new system matrix and cost function are reconstructed. The entire process of linearization, objective function construction, control gain recursion, and state prediction is repeated. This process is iteratively executed until the difference between two consecutive control sequences falls below a preset threshold, indicating that the control strategy converges and the global optimal control strategy under the path execution conditions is obtained. Based on the global optimal control strategy, a composite control law is designed. This control law consists of a nominal control term and a robust control term. The nominal control term is directly derived from the optimal control sequence and mainly performs path tracking and attitude control tasks. The robust control term is constructed based on system uncertainty estimation and external disturbance modeling. Its purpose is to correct state offsets caused by modeling errors, environmental changes, or sensor disturbances in real time, thereby enhancing the stability and anti-interference capability of the control system. The system converts the composite control law into control instructions, sends the torque input and speed setting at each time point to the robot's underlying drive system through a high-frequency real-time communication protocol (such as EtherCAT or Profinet), and completes data transmission, status update and error feedback within the servo control cycle, thereby realizing closed-loop linkage control between the controller and the robot body, and ultimately outputting a continuous, stable and dynamically optimal motion control instruction sequence for the entire dispensing task.

[0032] In a specific embodiment, the execution step is based on the optimal control objective function, and the process of iteratively calculating the control gain matrix by forward calculating the state equation and backward recursive Riccati equation to obtain the local optimal control sequence can specifically include the following steps: According to the optimal control objective function, the terminal state cost matrix and the intermediate state cost matrix are set, and the terminal state cost matrix and the intermediate state cost matrix are combined with the linear time-varying system matrix to obtain an augmented matrix system; Based on the augmented matrix system, the Riccati recursion equation is constructed, the terminal Riccati matrix value is set, and the Riccati recursion equation is solved by the reverse time recursion method to obtain the time-varying Riccati matrix sequence; The feedback control gain matrix and feedforward control term are calculated based on the time-varying Riccati matrix sequence and the linear time-varying system matrix to obtain the state feedback control law. Input the linear time-varying system matrix and the current system state into the state feedback control law, perform forward recursion of the state equation, and obtain the optimized trajectory state sequence; Perform trajectory error analysis on the optimized trajectory state sequence, calculate the difference between the trajectory deviation and the expected trajectory, and construct a state correction term based on the difference to obtain the corrected state sequence; The modified state sequence is substituted into the state feedback control law, and the control input sequence is recalculated to perform iterative optimization to obtain a locally optimal control sequence.

[0033] Specifically, according to the set optimal control objective function, the state convergence accuracy requirement at the terminal time and the trajectory control stability requirement in the intermediate process are determined under the framework of overall path planning and control objective trade-off. Two key matrices are set in the cost function, one is the terminal state cost matrix, which is used to define the punishment degree of the end state deviation from the target state at the end of the trajectory, and the other is the intermediate state cost matrix, which is used to measure the cost intensity of state error and control input amplitude at each time step. The weights of the two matrices are designed according to the complexity and dynamic change of the actual dispensing path. The terminal state weight emphasizes the accuracy of target positioning, while the intermediate state weight considers the stability, energy efficiency and tracking consistency of the system in operation. The two cost matrices and the time-varying system matrix obtained in the linearization step are combined to form an augmented matrix system, which includes the mapping relationship between system state and control variable, and also embeds the logical path affected by the cost function of state evolution. Based on the augmented matrix system, Riccati recursive equation is constructed, which is a structure optimization mechanism through reverse time recursion. The goal is to quantify the influence of future state deviation on current control input and feedback to the current decision, so as to realize an optimization logic of front and back information coupling. An initial terminal cost matrix is set at the end of time as the initial input of Riccati recursion, and then the recursion is performed from the end time step to the front, and the optimal cost matrix at the current time is updated based on the current state transition information and cost propagation rule at each step. This matrix sequence can accurately reflect the influence of state error at each time point on the global objective function, thereby providing algebraic support for the controller to develop a reasonable dynamic adjustment mechanism. The time-varying cost matrix sequence and the corresponding system matrix are jointly operated to extract a feedback control gain matrix and a feedforward adjustment item at each time. The construction of this feedback control law enables the control input to respond to the size and direction of the current state deviation from the target state in real time, and adjusts the current control amplitude through feedback gain, so as to gradually approach the optimal trajectory. The strength of the feedback control law depends on the error amplitude and the system stability constraint, so in the high dynamic path area, the control law can automatically enhance the adjustment amplitude, while in the stable area, it maintains moderate control to reduce system energy consumption. In the establishment of the control law, the linear time-varying system model and the current state vector are input into the controller to perform the forward recursion process of the state equation, that is, starting from the current system state, a series of future state evolution trajectories are calculated step by step according to the feedback control law, forming a state prediction sequence under the action of the optimal control law. This optimized trajectory satisfies the path constraint condition and reflects the response ability of the control input at different time nodes, which is an important reference for evaluating the effect of the current control law.The trajectory error analysis is performed on the predicted trajectory, the deviations between the actual system operation and the theoretical path are calculated by comparing the current trajectory state with the expected ideal path point by point, and the distribution law and change trend of the deviations in time are analyzed, and a set of state correction terms for correcting the control law are constructed. The correction term can readjust the direction or strength of the feedback gain inside the control law, which is particularly important when the nonlinear deviation is serious or the environmental disturbance is significant, and can improve the adaptability and error recovery ability of the controller. The modified state sequence is substituted into the state feedback control law again to perform the control input calculation process, and based on the new control input sequence, the state evolution prediction is performed again, and in this process, the control strategy and state feedback structure are iteratively optimized, until the change amplitude between the two consecutive control input sequences is less than the preset convergence threshold, at this time, the control system outputs a control instruction sequence with local optimality, path stability and error recovery ability, which is used to guide the five-axis robot to perform the dynamic path tracking task.

[0034] The above describes the mechanical hand dispensing path optimization method based on 3D vision in the embodiment of the application, and the following describes the mechanical hand dispensing path optimization device based on 3D vision in the embodiment of the application, please refer to Figure 2 An embodiment of the mechanical hand dispensing path optimization device based on 3D vision in the embodiment of the application includes: The scanning module 201 is configured to perform multi-angle three-dimensional vision scanning on the dispensing workpiece to obtain point cloud data of the dispensing workpiece. The semantic segmentation module 202 is configured to perform semantic segmentation on the point cloud data to decompose the dispensing workpiece into a plurality of functional components. The identification module 203 is configured to identify target regions to be dispensed and dispensing path constraint conditions according to semantic labels and surface features of the functional components. The generation module 204 is configured to generate an initial dispensing path based on the target regions to be dispensed and the dispensing path constraint conditions. The adjustment control module 205 is configured to perform iterative linear quadratic adjustment control on the initial dispensing path to generate a motion control instruction.

[0035] Through the cooperation of the above-mentioned components, the precise reconstruction and semantic recognition of the workpiece are realized through multi-angle three-dimensional visual scanning and semantic segmentation, the problem of lacking workpiece semantic information in traditional 3D reconstruction is solved, and more accurate workpiece models and semantic features are provided for the dispensing path planning. The dispensing path planning method based on the workpiece semantic information and the surface features realizes the high-precision dispensing area recognition and constraint condition generation of the complex curved surface workpiece, compared with the traditional method, the geometric changes of the workpiece surface can be more accurately adapted, and the dispensing quality and material utilization rate are improved. The composite control strategy combining iterative linear quadratic regulation control and adaptive sliding mode control effectively suppresses the model uncertainty and external disturbance in the dispensing process of the five-axis robot, and significantly enhances the robustness and path tracking accuracy of the system. Through the real-time 3D visual feedback mechanism, the workpiece position and posture changes and surface deformation can be dynamically perceived, the adaptive optimization adjustment of the dispensing path is realized, the problem that the traditional dispensing system cannot cope with the actual state changes of the workpiece is solved, and the adaptability and stability of the dispensing system are greatly improved.

[0036] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, system and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0037] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the whole or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0038] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A 3D vision-based robot dispensing path optimization method, characterized in that: include: Performing a multi-angle three-dimensional visual scan on the dispensing workpiece to obtain point cloud data of the dispensing workpiece; Performing semantic segmentation on the point cloud data to decompose the dispensing workpiece into multiple functional components; Identifying target areas to be dispensed and dispensing path constraints based on semantic identification and surface features of the functional components; generating an initial dispensing path based on the target area to be dispensed and the dispensing path constraint conditions; Iterative linear quadratic regulation control is performed on the initial dispensing path to generate motion control instructions.

2. The 3D vision-based robot dispensing path optimization method according to claim 1 is characterized in that: The multi-angle three-dimensional visual scanning of the dispensing workpiece to obtain point cloud data of the dispensing workpiece includes: Multiple industrial cameras are evenly distributed around the five-axis robot worktable, and the multiple industrial cameras are used to synchronously capture images of the dispensing workpiece from different angles to obtain multi-view image data; Performing Gaussian filtering, adaptive histogram equalization, and edge enhancement processing on the multi-view image data to obtain a pre-processed multi-view image; Inputting the preprocessed multi-view images into a stereo matching algorithm to perform disparity calculation to obtain a target disparity map; According to the camera calibration parameters and the target disparity map, coordinate transformation is performed through the triangulation principle to obtain initial point cloud data; The initial point cloud data is subjected to noise filtering and hole filling processing to obtain dense and complete point cloud data, and the dense and complete point cloud data is subjected to normal vector calculation and curvature analysis to obtain point cloud data of the dispensing workpiece.

3. The 3D vision-based robot dispensing path optimization method according to claim 1 is characterized in that: The step of performing semantic segmentation on the point cloud data and decomposing the dispensing workpiece into multiple functional components includes: Inputting the point cloud data of the dispensing workpiece into a multi-layer convolutional neural network for feature extraction to obtain deep features of the point cloud; Performing spatial pyramid pooling on the deep features of the point cloud to obtain multi-scale feature representation; Inputting the multi-scale feature representation into a fully connected layer for classification processing to obtain an initial semantic label of the point cloud data; Constructing a graph structure based on the initial semantic labels, and performing edge weight calculation and label smoothing based on the graph structure to obtain labels with optimized semantic consistency; Dividing the point cloud data into regions with clear boundaries according to the semantically consistent optimized labels to obtain candidate components of the dispensing workpiece; Topological relationship analysis and boundary refinement processing are performed on the candidate components to obtain multiple functional components with clear semantic identification.

4. The 3D vision-based robot dispensing path optimization method according to claim 1, characterized in that: The step of identifying the target area to be dispensed and the dispensing path constraint conditions based on the semantic identification and surface features of the functional component includes: Calculating Gaussian curvature and mean curvature for each point in the functional component to obtain a surface curvature distribution map; Configuring weights for surface features according to the semantic identifiers of the functional components to obtain surface feature weights, and performing region division according to the surface feature weights and a preset dispensing adaptability threshold to obtain candidate regions to be dispensed; Performing normal vector continuity detection and edge feature extraction on the candidate area to obtain area boundary and feature line information, and performing area merging and segmentation processing based on the area boundary and the feature line information to obtain the target area to be dispensed; Performing reachability analysis and collision detection on the target area to be dispensed, determining dispensing distance constraints and posture constraints, and obtaining a safe dispensing area; Based on the surface characteristics and process requirements of the safe dispensing area, glue quantity control parameters and movement speed parameters are set to obtain dispensing path constraint conditions.

5. The 3D vision-based robot dispensing path optimization method according to claim 1 is characterized in that: The generating of the initial dispensing path based on the target area to be dispensed and the dispensing path constraint condition includes: Performing equidistant sampling on the target area to be glued to obtain candidate glue position points, and screening the candidate glue position points according to the glue path constraint condition to obtain a set of valid glue position points; A* algorithm is used to search the effective dispensing position set for a path, and a candidate path set that satisfies the constraint conditions is obtained; The candidate path set that meets the constraint conditions is input into the particle swarm optimization algorithm, and global optimization is performed with path length, turning smoothness, glue uniformity and surface following accuracy as optimization objectives to obtain an optimized path sequence; Constructing a quintic B-spline curve based on the optimized path sequence, and uniformly distributing and locally adjusting control points based on the quintic B-spline curve to obtain a smooth and continuous dispensing curve; Acceleration and deceleration planning and posture planning are performed on the smooth and continuous dispensing curve, and speed parameters and tool posture parameters are assigned to each path point to obtain an initial dispensing path.

6. The 3D vision-based robot dispensing path optimization method according to claim 1, characterized in that: The performing iterative linear quadratic adjustment control on the initial dispensing path to generate motion control instructions includes: M1: Convert the initial dispensing path into the joint space trajectory of the five-axis robot and construct a robot dynamics model including joint angles, angular velocities, and dispensing nozzle position and posture; M2: Linearize the robot dynamics model at the current working point and use the Lagrangian algorithm to construct a linear time-varying system matrix including the inertia matrix, Coriolis torque, gravity torque, and friction torque; M3: Based on the linear time-varying system matrix, the position tracking error, velocity error and control input are used as penalty terms of the quadratic cost function and a weight matrix is ​​set to obtain the optimal control objective function; M4: Based on the optimal control objective function, the control gain matrix is ​​iteratively calculated by forward calculation of the state equation and backward recursive Riccati equation to obtain a local optimal control sequence; M5: Apply the local optimal control sequence to the robot dynamics model to predict the system state, then update the linearization parameters based on the prediction error, and repeat steps M2 to M4 until the control sequence converges to obtain the global optimal control strategy; M6: Based on the global optimal control strategy, a composite control law including nominal control terms and robust control terms is constructed, and the torque command and speed command are transmitted to the robot drive system through a real-time communication protocol to output motion control commands.

7. The 3D vision-based robot dispensing path optimization method according to claim 6, characterized in that: The method of iteratively calculating the control gain matrix based on the optimal control objective function by forward calculating the state equation and backward recursively calculating the Riccati equation to obtain a local optimal control sequence includes: Setting a terminal state cost matrix and an intermediate state cost matrix according to the optimal control objective function, and combining the terminal state cost matrix and the intermediate state cost matrix with a linear time-varying system matrix to obtain an augmented matrix system; Constructing a Riccati recursion equation based on the augmented matrix system, setting a terminal Riccati matrix value, and solving the Riccati recursion equation by a reverse time recursion method to obtain a time-varying Riccati matrix sequence; Calculating a feedback control gain matrix and a feedforward control term based on the time-varying Riccati matrix sequence and the linear time-varying system matrix to obtain a state feedback control law; Inputting the linear time-varying system matrix and the current system state into the state feedback control law, executing the forward recursion of the state equation, and obtaining the optimized trajectory state sequence; Performing trajectory error analysis on the optimized trajectory state sequence, calculating the difference between the trajectory deviation and the expected trajectory, and constructing a state correction term based on the difference to obtain a corrected state sequence; The modified state sequence is substituted into the state feedback control law, and the control input sequence is recalculated and iteratively optimized to obtain a local optimal control sequence.

8. A 3D vision-based robot dispensing path optimization device, characterized in that: Used to execute the 3D vision-based robot dispensing path optimization method according to any one of claims 1 to 7, the 3D vision-based robot dispensing path optimization device comprising: A scanning module is used to perform multi-angle three-dimensional visual scanning on the dispensing workpiece to obtain point cloud data of the dispensing workpiece; A semantic segmentation module, configured to perform semantic segmentation on the point cloud data and decompose the dispensing workpiece into multiple functional components; An identification module, configured to identify a target area to be dispensed and dispensing path constraints based on the semantic identification and surface features of the functional component; A generation module, configured to generate an initial dispensing path based on the target area to be dispensed and the dispensing path constraint conditions; The regulation control module is used to perform iterative linear quadratic regulation control on the initial dispensing path to generate motion control instructions.

Images