Multi-robot collaborative measurement path planning method based on reinforcement learning

Abstract

The application belongs to the technical field of path planning, and particularly relates to a multi-robot cooperative measurement path planning method based on reinforcement learning, which comprises the following steps: obtaining a model and point cloud data of a large and complex component, and performing pretreatment; constructing a three-dimensional environment model based on the pretreated point cloud data, and defining a state space, an executable action space and a reward function of a reinforcement learning process of the reinforcement learning environment; assigning a measurement task to multiple robots based on the constructed three-dimensional environment model, generating an initial path covering the surface of the large and complex component, and optimizing the initial path by using a reinforcement learning method to generate an optimized path; sharing the optimized paths and states of the multiple robots in real time, predicting future potential conflicts, and dynamically cooperatively adjusting the paths and states of the multiple robots based on the prediction results to generate a final path and a control instruction, so as to realize multi-robot cooperative measurement path planning.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of path planning, and particularly relates to a multi-robot cooperative measurement path planning method based on reinforcement learning. BACKGROUND

[0002] Under the background of rapid development of high-end equipment manufacturing industries such as aerospace, shipbuilding and new energy, high-precision three-dimensional measurement of large and complex components such as overall fuselage of aircraft, wind turbine blades and ship sections has become a core demand in the field of intelligent manufacturing. Such components have the characteristics of complex geometric surfaces (high curvature, multiple feature fusion), large spatial scale (length often reaches tens to hundreds of meters), and strict measurement accuracy requirements (sub-millimeter error tolerance). To avoid the problems of efficiency bottleneck, coverage blind area and insufficient adaptability to dynamic environment in the traditional single-machine measurement mode, multi-machine cooperative measurement technology is currently used to realize global optimization of multi-agent path planning, while meeting the measurement accuracy and motion safety under complex surface constraints. Existing measurement path planning methods for large components include offline planning based on geometric decomposition, random sampling algorithms (such as RRT*), and swarm intelligence optimization (such as ant colony algorithm). Among them, the offline planning method based on geometric decomposition is difficult to adapt to surface mutations and dynamic obstacles by manually dividing the measurement area and presetting the scanning trajectory, which may lead to a high rate of repeated measurement of more than 30%; the random sampling algorithm can guarantee probabilistic completeness, but the generated path is highly tortuous and requires additional smoothing, which may lead to path conflicts in multi-machine cooperative scenarios; the swarm intelligence optimization method can rely on heuristic rules, but the computational complexity increases exponentially with the increase of component size, which cannot meet the real-time requirements.

[0003] However, the introduction of reinforcement learning (RL) method has shown certain potential in robot path planning, but still has certain limitations in the large component measurement scene, which are reflected in the following aspects: insufficient state space modeling due to insufficient fusion of component geometric features and multi-machine cooperative constraints, low path adherence due to the difficulty of traditional Q-learning method in handling continuous control problems, and lack of cooperative mechanism due to the lack of multi-agent competition-cooperation balance strategy in existing research, which leads to unbalanced allocation of measurement resources. SUMMARY

[0004] The application provides a multi-robot cooperative measurement path planning method based on reinforcement learning, which can realize efficient cooperative work of multiple robots through the construction of a three-layer cooperative architecture of global task allocation, local path optimization and real-time conflict resolution; meanwhile, the paths and states of the robots are detected and dynamically adjusted in real time, so that the robots can work cooperatively without conflict and with high precision in the measurement process, thereby ensuring the measurement accuracy of complex surfaces and efficiently covering the surface of large complex components, reducing the rate of repeated measurement, and realizing the global optimization of multi-agent path planning.

[0005] A multi-robot cooperative measurement path planning method based on reinforcement learning, comprising:

[0006] Obtain the model and point cloud data of the large complex component and preprocess them;

[0007] Construct a three-dimensional environment model based on the preprocessed point cloud data, and define the state space, executable action space and reward function of the reinforcement learning process of the reinforcement learning environment;

[0008] Based on the constructed three-dimensional environment model, assign the measurement task to multiple robots, generate an initial path covering the surface of the large complex component, and optimize the initial path using a reinforcement learning method to generate an optimized path;

[0009] Real-time sharing of the optimized paths and states of multiple robots, prediction of future potential conflicts, and dynamic cooperative adjustment of the paths and states of multiple robots based on the prediction results to generate a final path and control instruction, so as to realize multi-robot cooperative measurement path planning.

[0010] Through the construction of a three-layer cooperative architecture of global task allocation, local path optimization and real-time conflict resolution, efficient cooperative work of multiple robots can be realized; meanwhile, the paths and states of the robots are detected and dynamically adjusted in real time, so that the robots can work cooperatively without conflict and with high precision in the measurement process, thereby ensuring the measurement accuracy of complex surfaces and efficiently covering the surface of large complex components, reducing the rate of repeated measurement, and realizing the global optimization of multi-agent path planning.

[0011] Further, the obtaining of the model and point cloud data of the large complex component and the preprocessing thereof comprises:

[0012] Directly obtaining the CAD model of the large complex component and converting it into a first triangular mesh model;

[0013] Using a laser scanner to scan the large complex component to obtain point cloud data of the surface of the large complex component, and performing denoising and downsampling processing to generate downsampled point cloud data;

[0014] Based on the first triangular mesh model and the down-sampling point cloud data, an iterative closest point algorithm is used for alignment operation.

[0015] Further, the pre-processed point cloud data is used to construct a three-dimensional environment model, and the state space, executable action space and reward function of the reinforcement learning process of the reinforcement learning environment are defined, including:

[0016] Based on the down-sampling point cloud data, a second triangular mesh model is generated by using a Poisson surface reconstruction algorithm, and the surface information of the large and complex component is obtained in combination with the first triangular mesh model;

[0017] Based on the surface information of the large and complex component, the surface of the large and complex component is divided into a plurality of manageable sub-regions by using the surface geometric feature information and the spatial position information, and a three-dimensional environment model is constructed;

[0018] Based on the three-dimensional environment model, the geometric representation of the large and complex component is obtained by sampling, and the state space of the reinforcement learning environment is defined in combination with the states of the plurality of robots and the environment state;

[0019] Based on the single robot action and the collaborative action of the plurality of robots, the executable action space is defined;

[0020] Based on the coverage, efficiency, conflict avoidance and quality of the large and complex component measurement, the reward function of the reinforcement learning process is defined.

[0021] By embedding the geometric representation of the large and complex component into the state space of the reinforcement learning, and using the surface geometric feature information and the spatial position information for spatial physical feasibility constraints, the problem of easy deviation and difficult convergence of traditional path planning methods on complex surfaces can be solved, and the fitting degree and precision of the measurement path are improved.

[0022] Further, based on the surface information of the large and complex component, the surface of the large and complex component is divided into a plurality of manageable sub-regions by using the surface geometric feature information and the spatial position information, and a three-dimensional environment model is constructed, including:

[0023] Based on the surface information of the large and complex component, the surface of the large and complex component is divided into a plurality of initial sub-regions by using the surface geometric feature information and the spatial feature information;

[0024] The plurality of initial sub-regions are clustered by using the similarity matrix and the Laplacian matrix to form a plurality of manageable sub-regions, and a three-dimensional environment model is constructed.

[0025] Further, when defining the executable action space, the collaborative action of the plurality of robots includes task reassignment and robot priority matrix, which is used for preferentially assigning high-curvature sub-regions and preferentially performing conflict active avoidance for low-priority robots.

[0026] Further, based on the constructed three-dimensional environment model, the measurement tasks are assigned to multiple robots, an initial path covering the surface of the large and complex component is generated, and a reinforcement learning method is used to optimize the initial path to generate an optimized path, comprising:

[0027] Based on the division of multiple sub-regions of the large and complex component, the complexity of each sub-region is quantified;

[0028] Based on the maximum moving speed, measurement accuracy, endurance and field of view range of the robot, the measurement performance of each robot is evaluated;

[0029] Based on the measurement performance of each robot, a particle swarm optimization algorithm is used to assign the measurement sub-region to each robot;

[0030] Based on the sub-region assigned to each robot, a UV parameterization function is used to map a two-dimensional zigzag path to a three-dimensional curved surface, calculate the three-dimensional coordinates and path direction of the path points, and globally connect each path point, combine real-time collision detection between path points, adjust the measurement path of each robot and the corresponding time-speed sequence to form an initial path covering the surface of the large and complex component;

[0031] Based on the initial path, a trained strategy network and evaluation network are used to predict the action of each robot, calculate the continuity of the path points, and combine the real-time detection of path conflicts between path points to adjust and generate an optimized path covering the surface of the large and complex component.

[0032] Through the construction of a three-layer collaborative architecture of global task allocation, local path optimization and real-time conflict resolution, the advantages of swarm intelligence of particle swarm optimization and autonomous decision-making of multi-agent reinforcement learning are combined, breaking through the limitations of single algorithm in scalability and dynamic adaptability, and realizing efficient collaborative work of multiple robots.

[0033] Further, based on the sub-region assigned to each robot, a UV parameterization function is used to map a two-dimensional zigzag path to a three-dimensional curved surface, calculate the three-dimensional coordinates and path direction of the path points, and globally connect each path point, combine real-time collision detection between path points, adjust the measurement path of each robot and the corresponding time-speed sequence to form an initial path covering the surface of the large and complex component, comprising:

[0034] Based on the sub-region assigned to each robot, a UV parameterization function is used to map a two-dimensional zigzag path to a three-dimensional curved surface, obtain a three-dimensional path, and determine the three-dimensional coordinates and path direction of each path point;

[0035] Based on the path point sequence, a path smoothing is performed on a path connection segment formed by connecting the path points using a piecewise cubic spline interpolation algorithm, and a global connection of the path points is performed; the global connection of the path points includes assigning weights to the path connection segment and the corresponding sub-region, and using an optimization algorithm to solve an optimal path connection segment connection sequence;

[0036] Real-time collision detection is performed based on the path point spacing of each robot, and path correction is performed based on the real-time collision detection result to generate an initial path point set of each robot and a corresponding time-speed sequence, forming an initial path covering the surface of a large complex component;

[0037] The initial path point set of each robot is represented as:

[0038] ;

[0039] In the formula, represents the initial path point set of the i-th robot; represents the i-th path point of the j-th robot;

[0040] The time-speed sequence of each robot is represented as:

[0041] ;

[0042] In the formula, represents the time-speed sequence of the i-th robot; represents the i-th path point of the j-th robot; represents the i-th path point of the j-th robot.

[0043] Further, based on the initial path, a trained strategy network and an evaluation network are used to predict the action of each robot, calculate continuous path points, and combine real-time detection of path point conflicts between robots to adjust and generate an optimized path covering the surface of a large complex component, including:

[0044] The initial path is taken as a starting point, and a strategy network and an evaluation network are designed to output the action that each robot should perform in the current state, and to evaluate the expected cumulative reward under the given state and all robot actions;

[0045] Based on the triangular mesh model, a priority replay strategy is used to focus on key sub-regions, add surface constraints to limit the action range of the robot, and use a minimum time difference error method to update the evaluation network and maximize the reward value to update the strategy network parameters, obtaining a trained strategy network; ​​​​​

[0046] Based on the trained policy network, the robot predicts the actions it should perform and generates continuous path points. It then combines the continuity constraints between adjacent path points and the surface fit constraints between path points and large, complex components to construct a spatiotemporal joint optimization objective function, eliminate path jitter, and obtain an optimized continuous path.

[0047] Based on the distance between the path points of the robots, the robot can detect the path conflicts in real time and make directional adjustments based on the results of the path conflicts to generate an optimized path covering the surface of large and complex components.

[0048] The optimized path for covering the surface of a large complex component is expressed as:

[0049] ;

[0050] Where, Represents the optimized path covering the surface of a large and complex component; After optimization The three-dimensional coordinates of the path point at that moment; After optimization the speed of the moment; After optimization The surface direction vector at the moment; , Indicates the total number of moments.

[0051] Furthermore, the real-time sharing of optimized paths and states of multiple robots, prediction of future potential conflicts, and dynamic collaborative adjustment of the paths and states of multiple robots based on the prediction results to generate the final path and control instructions to achieve multi-robot collaborative measurement path planning include:

[0052] Share the optimized paths and states of multiple robots in real time; the robot states include current position, speed, posture, remaining energy, and measurement status;

[0053] Based on the optimized path and speed of each robot, the future position of the robot is predicted, and future position conflict determination between the robots is performed; the future position conflict determination between the robots includes triggering conflict adjustment when the distance between the path points of the two robots at a future time is lower than a preset distance threshold;

[0054] Based on the future position conflict judgment results, the paths and states of each robot are dynamically and collaboratively adjusted to generate the final path and control instructions to realize multi-robot collaborative measurement path planning.

[0055] Furthermore, based on the future position conflict judgment results, the paths and states of each robot are dynamically and collaboratively adjusted to generate the final path and control instructions to achieve multi-robot collaborative measurement path planning, including:

[0056] When a conflict between the future positions of two robots is predicted, the robot priority matrix is ​​used to adjust the conflict weight, giving priority to adjusting the path of the robot with the lower priority, and the adjusted robot path is used as the final path;

[0057] The conflict weight adjustment is expressed as:

[0058] ;

[0059] Where, Indicates the adjusted The robot is relative to the The priority of each robot; Indicates the unadjusted The robot is relative to the The priority of each robot; Indicates the A robot in The location of the waypoint at the moment; Indicates the A robot in The location of the waypoint at the moment;

[0060] The path adjustment of the robot is expressed as:

[0061] ;

[0062] Where, Indicates the A robot in The position of the path point is adjusted at all times, and the adjusted path point position fits the triangular mesh model, that is, , Represents a point in a triangular mesh model, represents a triangular mesh model, A function that represents the value of the independent variable that minimizes the objective function; , that is, The robot has higher priority than the A robot;

[0063] When it is predicted that the measurement progress of two robots is different, the speed of the robot with the lagging measurement progress is adjusted first;

[0064] The speed adjustment of the robot is expressed as:

[0065] ;

[0066] Where, Indicates the The robot accelerates to the adjusted speed; represents the speed of the first robot without adjustment; represents the coverage point ratio of the first robot relative to the point number of the divided sub-area; represents the coverage point ratio of the first robot relative to the point number of the divided sub-area; , that is, the first robot lags behind the first robot in measurement progress; In combination with the measurement accuracy of the robot, the posture of the robot is dynamically adjusted;

[0067] The robot posture adjustment is represented as:

[0068] The robot posture adjustment is represented as:

[0069] ;

[0070] In the formula, represents the posture of the first robot after acceleration adjustment; represents the posture of the first robot without adjustment; represents the posture of the first robot without adjustment; represents the angle between the sensor field of view of the first robot and the normal vector of the curved surface

[0071] In combination with the robot priority matrix, the adjusted robot speed and the robot posture, the robot is used as a control instruction, and the final path is combined to realize the multi-robot collaborative measurement path planning.

[0072] The beneficial effects of the present application are:

[0073] The present application can realize efficient collaborative work of multiple robots by constructing a three-layer collaborative architecture of global task allocation, local path optimization and real-time conflict resolution. At the same time, the path and state of each robot are detected and dynamically adjusted in real time, so that each robot collaborates in the measurement process without conflict and high precision, thereby ensuring the measurement accuracy of the complex curved surface while efficiently covering the surface of the large and complex component, reducing the rate of repeated measurement, and realizing the global optimization of multi-agent path planning. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 is a flowchart of the present application;

[0075] Figure 2 is a system structure schematic diagram of the present application;

[0076] Figure 3 is a structure schematic diagram of a computer device. DETAILED DESCRIPTION​​​​​​​

[0077] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0078] It is to be appreciated that various aspects of the embodiments within the scope of the appended claims are described below. It is to be understood that the aspects described herein can be implemented in a wide variety of forms and that any particular structure and / or function described herein is merely illustrative. Based on the disclosure provided, one skilled in the art should appreciate that an aspect described herein can be implemented independent of any other aspects, and that two or more of these aspects can be combined in any suitable manner. For example, an apparatus can be implemented and / or a method can be practiced using any number of the aspects set forth herein. In addition, such an apparatus can be implemented and / or such a method can be practiced using other structure and / or functionality in addition to or other than one or more of the aspects set forth herein.

[0079] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. For a person of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0080] Embodiment 1

[0081] Figure 1 The method is a multi-robot cooperative measurement path planning method based on reinforcement learning. By constructing a three-layer cooperative architecture of global task allocation, local path optimization and real-time conflict resolution, efficient cooperative work of multiple robots can be achieved. At the same time, the paths and states of each robot are detected and dynamically adjusted in real time, so that each robot cooperates in the measurement process without conflict and with high precision, thereby ensuring efficient coverage of the surface of a large complex component while ensuring the measurement accuracy of a complex surface, reducing the rate of repeated measurement, and achieving global optimization of multi-agent path planning. Specifically, the method comprises the following steps:

[0082] S1: Obtain the model and point cloud data of a large complex component and perform preprocessing;

[0083] S11: Directly obtain the CAD model of a large complex component and convert it into a first triangular mesh model;

[0084] In this embodiment, the STL and STEP format large complex component model is converted into a first triangular mesh model , which is represented as:

[0085] ;

[0086] In the formula, denotes the first triangular mesh model ; denotes the vertex set of the first triangular mesh; denotes the face sheet set of the first triangular mesh;

[0087] S12: Scanning the large complex component to obtain point cloud data of the surface of the large complex component by using a laser scanner, performing denoising and downsampling processing, and generating down-sampled point cloud data;

[0088] In the embodiment, the point cloud data of the surface of the large complex component is scanned by using a laser scanner, and the obtained point cloud data is in PCD or PLY format.

[0089] The denoising and downsampling processing to generate the down-sampled point cloud data includes:

[0090] S121: Based on the point cloud data, a statistical outlier removal algorithm is used for denoising processing to obtain denoised point cloud data ;

[0091] In the embodiment, the statistical outlier removal algorithm is to calculate the average distance of the k-nearest neighbors of each point in the point cloud data , and when the average distance satisfies the removal condition, the current point cloud data is removed as an outlier.

[0092] The expression of the average distance is:

[0093] ;

[0094] In the formula, denotes the average distance of the k-nearest neighbors of the i-th point, ; denotes the i-th point cloud data; denotes the neighbor point of the point ; denotes the Euclidean distance between the point and the neighbor point ; The removal condition is:

[0095]

[0096] ;

[0097] In the formula, denotes the average value of the average distances of the k-nearest neighbors of all points, that is​​​ ; denotes the standard deviation of the mean distance of the k-nearest neighbors of all points, i.e. ; ;

[0098] S122: based on the denoised point cloud , a voxel grid filtering method is used for downsampling processing to generate down-sampled point cloud data ;

[0099] In this embodiment, the voxel grid size is set to 0.01m.

[0100] S13: based on the first triangular mesh model and the down-sampled point cloud data, an iterative closest point algorithm is used for alignment operation;

[0101] wherein the objective function for the alignment operation using the iterative closest point algorithm is:

[0102] ;

[0103] wherein, denotes the objective function of the iterative closest point algorithm; denotes the rotation matrix of the first triangular mesh model or the down-sampled point cloud data; denotes the translation vector of the triangular mesh model or the down-sampled point cloud data; denotes the point in the first triangular mesh model corresponding to the point in the down-sampled point cloud data

[0104] S2: based on the pre-processed point cloud data, a three-dimensional environment model is constructed, and the state space, the executable action space and the reward function of the reinforcement learning process of the reinforcement learning environment are defined;

[0105] S21: based on the down-sampled point cloud data, a second triangular mesh model is generated using a Poisson surface reconstruction algorithm, and combined with the first triangular mesh model, the surface information of the large and complex component is obtained;

[0106] In this embodiment, for each point in the down-sampled point cloud data , the search radius is set to 0.05m, and the covariance matrix is calculated based on the k-nearest neighbor method to estimate the normal vector.

[0107] wherein the covariance matrix is represented as:

[0108] ;

[0109] wherein, denotes the covariance matrix; denotes the number of the nearest neighbor points, ; denotes the centroid of the k-nearest neighbors of the point ; denotes the nearest neighbor points of the point ; denotes the transpose matrix;

[0110] wherein the expression of the normal vector is:

[0111] ;

[0112] wherein, denotes the normal vector at the point , used to represent the orientation of the local surface where the point is located; denotes the three-dimensional coordinates of the normal vector of the point ;

[0113] An equation is constructed by using the Poisson reconstruction method, and the equation is solved to generate a second triangular mesh model . Wherein the constructed equation is expressed as:

[0114] ;

[0115] wherein, denotes the Laplacian operator; denotes the scalar function; denotes the divergence operator; denotes the normal vector field of all down-sampled point cloud data ;

[0116] In the embodiment, the second triangular mesh model has a depth of 8 and a smoothing factor of 0.5.

[0117] In combination with the first triangular mesh model and the second triangular mesh model , a triangular mesh model is formed, and the surface information of the large and complex component is obtained.

[0118] S22: Based on the surface information of the large and complex component, the surface of the large and complex component is divided into a plurality of manageable sub-regions by using the surface geometric feature information and the spatial position information, and a three-dimensional environment model is constructed;

[0119] S221: Based on the surface information of the large and complex component, the surface of the large and complex component is divided into a plurality of initial sub-regions by using the surface geometric feature information and the spatial feature information;

[0120] Wherein the surface geometric feature information is the curvature, and the calculation expression is:

[0121] ;

[0122] ;

[0123] wherein, denotes the curvature of the point ; denotes the determinant value of the first-order matrix; denotes the determinant value of the second-order matrix; denotes the curvature of the normalized point ; denotes the minimum value among all curvatures; denotes the maximum value among all curvatures;

[0124] S222: clustering the plurality of initial sub-regions divided by using the similarity matrix and the Laplacian matrix to form a plurality of manageable sub-regions, and constructing a three-dimensional environment model;

[0125] S221: constructing a similarity matrix based on the curvatures of the points ;

[0126] wherein, each element in the similarity matrix is expressed as:

[0127] ;

[0128] wherein, denotes the element of the point and the point in the similarity matrix , and is used to represent the similarity of the point and the point in the spatial position and the curvature, that is, the greater the value, the higher the similarity of the two points; denotes the spatial position vector of the point ; denotes the spatial position vector of the point ; denotes the spatial position weight; denotes the curvature of the point ; denotes the curvature of the point ; denotes the curvature weight;

[0129] S222: clustering the initial sub-regions by using the Laplacian matrix based on the similarity matrix to form a plurality of manageable sub-regions, and constructing a three-dimensional environment model;

[0130] wherein, the expression of the Laplacian matrix is:

[0131] ;

[0132] wherein, represents a Laplacian matrix; represents a diagonal matrix;

[0133] In the present embodiment, the set of sub-regions formed by clustering is denoted as:

[0134] ;

[0135] wherein, represents the set of sub-regions; represents the number of clusters, i.e. in the present embodiment, ;

[0136] S223: Constructing a three-dimensional space model in combination with the triangular mesh model, the set of sub-regions, and the obstacle boundary.

[0137] wherein, the three-dimensional environment model is denoted as:

[0138] ;

[0139] wherein, represents the three-dimensional environment model; represents the triangular mesh model; represents the sub-regions after clustering; represents the obstacle boundary.

[0140] S23: Based on the three-dimensional environment model, sampling to obtain the geometric representation of the large and complex component, and in combination with multiple robot states and environment states, defining the state space of the reinforcement learning environment;

[0141] wherein, the state space of the reinforcement learning environment is defined as:

[0142] ;

[0143] wherein, represents the state space of the reinforcement learning environment; represents the geometric representation of the large and complex component, which is sampled from the triangular mesh model , the state of each sampling point is denoted as: , , , represents the three-dimensional coordinates of the point , represents the identifier of the robot;​ represents a robot, i.e. each robot state represents: , , , represents the three-dimensional coordinates of the th robot, , , represents the three attitude angles of the th robot, represents the movement speed of the robot; represents the environmental parameters; represents the field of view range of the robot; represents the action parameters of the robot; represents the environmental state, i.e. represents: , , , represents the obstacle coordinates in the environment, represents the width of the obstacle, represents the length of the obstacle, represents the specific coordinates of the obstacle;

[0144] Dimension reduction is performed using a variational autoencoder, and the loss function of the variational autoencoder is:

[0145] ;

[0146] wherein, represents the loss function of the variational autoencoder; represents the current state; represents the next state; represents a hyperparameter, i.e. ; represents the Kullback-Leibler divergence function; represents the variational posterior distribution; represents the prior distribution; represents the distribution variable;

[0147] In this embodiment, the state space is normalized and updated at a frequency of 10Hz through robot sensor data and communication (ROS2, MQTT).

[0148] S24: defining an executable action space based on single-robot single-action and multi-robot cooperative action;

[0149] wherein the executable action space is defined as:

[0150] ;

[0151] wherein, represents the executable action space; represents the single-machine action of the i-th robot, i.e., the single-machine state of the i-th robot is represented as: , , 、 represents the displacement movement amount of the i-th robot in the three-dimensional space, i.e., , represents the radian information in the pose of the robot, i.e., ; represents the movement speed of the robot, i.e., , represents whether the robot starts measurement, i.e., represents that measurement has not started, represents that measurement has started; represents the cooperative action of the plurality of robots, i.e., , represents task reassignment, i.e., , represents the priority matrix, i.e. ; , represents the total number of robots;

[0152] wherein, in task allocation, the high-curvature sub-region is preferentially allocated, i.e., is represented as:

[0153] ;

[0154] wherein, represents that the task of measuring the i-th sub-region is allocated to the j-th robot; represents the robot number that minimizes the value; represents the endurance of the i-th robot; represents the weight of the task of measuring the i-th sub-region; At the same time, in order to ensure that the movement of the robot always fits the surface of the triangular mesh model, and avoids deviating from the surface of the large and complex component, i.e., the executable action is constrained as: ;

[0155]

[0156] ;

[0157] wherein,​​​​​​​ represents the position of the first robot after moving; represents the current position of the first robot; represents a point in the triangular mesh model;

[0158] S25: Defining the reward function of the reinforcement learning process based on the coverage, efficiency, collision avoidance and quality of large complex component measurement;

[0159] wherein the reward function is optimized by the current state , the executed action and the next state , i.e. defined as:

[0160] ;

[0161] ;

[0162] ;

[0163] ;

[0164] ;

[0165] ;

[0166] In the formula, represents the reward function; represents the coverage reward for measuring new points; represents the efficiency reward for punishing time and energy consumption, for evaluating the time and energy consumption of the measurement path; represents the collision avoidance reward for evaluating the collision risk and field of view overlap between multiple robots; represents the quality reward for evaluating the accuracy of the measurement data and the uniformity of the point cloud distribution; represents the coverage weight coefficient, i.e. ; represents the efficiency weight coefficient, i.e. ; represents the collision weight coefficient, i.e. ; represents the quality weight coefficient, i.e. ; represents the increment of the previously uncovered sub-area covered by the new measurement point; represents the total area of the large complex component; represents the indicator function; ​​represents a parameter, i.e. ; represents a parameter, i.e. ; represents a parameter, i.e. ; represents a parameter, i.e. ; represents the position difference of the measurement points of the i th robot and the j th robot; represents the intersection area of the measurement sub-regions of the i th robot and the j th robot; represents the measurement sub-region area of the i th robot; represents a parameter, i.e. ; represents a parameter, i.e. ; represents the total number of measurement points; represents the distance of the measurement point to the point in the triangular mesh model; represents whether the robot starts measurement; represents the measurement accuracy of the i th robot; represents the distance of the measurement point to the point in the triangular mesh model; represents whether the robot starts measurement; represents a parameter, i.e. ; represents a parameter, i.e. ; represents a parameter, i.e. ; represents a parameter, i.e.

[0167] S3: based on the constructed three-dimensional environment model, assigning the measurement task to multiple robots, generating an initial path covering the surface of the large and complex component, and using a reinforcement learning method to optimize the initial path to generate an optimized path;

[0168] S31: based on the multiple sub-regions divided according to the large and complex component, quantifying the complexity of each sub-region;

[0169] wherein the complexity of each sub-region is represented as:

[0170] ;

[0171] wherein, represents the complexity of the i th sub-region; represents the area of the sub-region ; represents the triangular mesh vertex of the i th sub-region; represents the curvature value of the sub-region vertex;

[0172] ​​S32: Evaluate the measurement performance of each robot based on the maximum moving speed, measurement accuracy, endurance capability and field of view range of the robot;

[0173] wherein the measurement performance of each robot is represented as:

[0174] ;

[0175] wherein, represents the measurement performance of the i-th robot; represents the maximum moving speed of the i-th robot; represents the measurement accuracy of the i-th robot; represents the endurance capability of the i-th robot; represents the field of view range of the i-th robot;

[0176] S33: Assign a sub-region for measurement to each robot based on the measurement performance of each robot using a particle swarm optimization algorithm;

[0177] wherein the sub-region assignment using the particle swarm optimization algorithm constructs an objective function using load variance and total time minimization, represented as:

[0178] ;

[0179] wherein, represents the construction of the objective function using load variance and total time minimization; represents the variance function; represents the assignment of the i-th sub-region to the i-th robot; The update of the particle position is represented as:

[0180] ;

[0181] wherein, represents the velocity of the particle at time t;

[0182] represents the velocity of the particle at time t; , represents a random number used to adjust the degree of learning of the particle towards its own historical optimal position and global optimal position; represents the historical optimal position of the particle itself; represents the position of the particle at time t; ​​​​​​​​​​a global optimal position of the entire particle group;

[0183] In this embodiment, high-curvature sub-regions (Ri) are preferentially assigned to robots with high measurement accuracy (Ai).

[0184] S34: Based on the sub-regions assigned to each robot, a two-dimensional zigzag path is mapped to a three-dimensional curved surface using a UV parameterization function, the three-dimensional coordinates and the path direction of each path point are calculated, and each path point is globally connected. In combination with real-time collision detection between path points, the measurement path of each robot and the corresponding time-velocity sequence are adjusted to form an initial path covering the surface of a large and complex component;

[0185] S341: Based on the sub-regions assigned to each robot, a two-dimensional zigzag path is mapped to a three-dimensional curved surface using a UV parameterization function, the three-dimensional coordinates and the path direction of each path point are calculated, and each path point is globally connected. In combination with real-time collision detection between path points, the measurement path of each robot and the corresponding time-velocity sequence are adjusted to form an initial path covering the surface of a large and complex component;

[0186] In this embodiment, the spacing of the two-dimensional zigzag path is 0.01 m, and when mapped to a three-dimensional curved surface, the coordinates of the path points are:

[0187] ;

[0188] In the formula, represents the coordinates of the path points on the three-dimensional curved surface; represents the UV parameterization function; represents the two-dimensional zigzag path coordinates, ;

[0189] wherein the principal curvature direction field determines the path direction, and the eigenvalues of the surface Hessian matrix are solved to obtain, i.e. the path direction is represented as:

[0190] ;

[0191] In the formula, represents the path direction; represents the parameter that makes the function take the maximum value; represents the eigenvalues of the Hessian matrix; represents the transposed matrix;

[0192] S342: Based on the path point sequence, a path smoothing algorithm is used to smooth the path connection segment formed by connecting the path points, and the global connection of each path point is performed;

[0193] wherein the path smoothing transition curve obtained by path smoothing is represented as:

[0194] ;

[0195] ​​In the formula, represents a smooth transition curve between path points, ; , , , , , , , , , , , respectively represent the coefficients of the piecewise cubic spline interpolation curve; the boundary condition is: , ; the continuity constraint condition is: ;

[0196] When performing global connection of path points, the problem is converted into a weighted optimal access sequence problem, that is, weights are assigned to path connection segments and corresponding sub-regions, and an optimal path connection segment connection sequence is solved by using an optimization algorithm. In the optimal access sequence method, the expression of the cost function is:

[0197] ;

[0198] In the formula, represents a cost function; represents an approximate geodesic distance function calculated by the A* algorithm; respectively represent the ranges of sub-regions of the surface of a large complex component; represents the average curvature of a region;

[0199] S343: Real-time collision detection is performed based on the path point spacing of each robot, path correction is performed based on the real-time collision detection result, an initial path point set of each robot and a corresponding time-velocity sequence are generated, and an initial path covering the surface of a large complex component is formed;

[0200] wherein the real-time collision detection is realized by a distance field, when any two path points satisfy , offset correction is performed, that is, is represented as:

[0201] ;

[0202] In the formula, represents a corrected path point; represents a path point before correction; represents a normal vector at the path point , that is, , represents a function expression of a two-dimensional path; represents a normal vector the module of;

[0203] wherein the initial path point set of each robot is represented as:

[0204] ;

[0205] wherein, represents the initial path point set of the th robot; represents the th path point of the th robot;

[0206] wherein the time-speed sequence of each robot is represented as:

[0207] ;

[0208] wherein, represents the time-speed sequence of the th robot; represents the th path point of the th robot.

[0209] S351: Based on the initial path, the trained strategy network and evaluation network are used to predict the action of each robot, calculate the continuity path point, and combine the path conflict of the path point between robots for real-time detection and adjustment to generate an optimized path covering the surface of a large and complex component.

[0210] S3511: Taking the initial path as the starting point, a strategy network and an evaluation network are designed to output the action that each robot should perform under the current state and to evaluate the expected cumulative reward under the given state and all robot actions;

[0211] wherein the strategy network is designed for each robot, i.e. to output the action that each robot should perform under the current state. The strategy network includes a 4-layer network structure, wherein the number of neurons in the hidden layer is 256, 128, and 64 in turn, and residual connection is introduced to improve gradient propagation, and the hidden layer is represented as:

[0212] ;

[0213] wherein, represents the hidden layer state at the th moment; represents the hidden layer state at the th moment; represents the hidden layer state at the th moment; represents the activation function; represents the the weight matrix of the moment; representing the first the bias vector of the moment;

[0214] where is the evaluation network designed for each robot, i.e. to evaluate the expected cumulative reward under the given state and all robots performing actions. The evaluation network fuses global and local features, where the number of neurons in the hidden layers is 512, 256, 128 and 64 in turn, and a self-attention mechanism is added in the last layer. The self-attention mechanism is represented as:

[0215] ;

[0216] wherein, represents the weight of the self-attention mechanism; represents the activation function; , represents the weight factor; represents the scaling factor;

[0217] S3512: Based on the triangular mesh model, the priority replay strategy is used to focus on the key sub-area, the surface constraint is added to limit the action range of the robot, and the evaluation network is updated by using the minimum time difference error method and the strategy network parameters are updated by maximizing the reward value, to obtain the trained strategy network;

[0218] In the simulation environment, a large and complex component model is loaded, and the model is trained. The loss function of the evaluation network update process is represented as:

[0219] ;

[0220] wherein, represents the loss function of the evaluation network; represents the expectation operator; represents the immediate reward of the robot at the moment; represents the reward value prediction of the subsequent action according to the strategy; represents the reward value prediction of the action in the state ; the learning rate of the evaluation function is ;

[0221] By maximizing the reward U value, the strategy function is updated, i.e. by adjusting the parameters of the strategy function, the robot selects the action that can obtain higher cumulative reward in the current state. The strategy function update process is:

[0222] ​ ;

[0223] Where, Represents the gradient of the objective function of the policy network, which is used to update the parameters of the policy network; Indicates that the status Take action The gradient of the reward value prediction; Represents the gradient of the policy function with respect to the policy network parameters; the learning rate of the policy function is .

[0224] Due to the natural non-uniformity of measurement data of large and complex components, priority playback based on temporal differential error is used to automatically focus on key sub-areas to compensate for the distribution deviation of measurement data, which can be expressed as:

[0225] ;

[0226] Where, Indicates the Sampling priority of each sampling point; Indicates the The timing difference error of each sampling point;

[0227] At the same time, during the training process, the robot's movements should be strictly restricted within the surface space of large and complex components to ensure that the generated measurement path meets physical feasibility. Constraints are imposed by adding regularized movements with surface constraints, which can be expressed as:

[0228] ;

[0229] Where, Regularized loss function representing surface constraints; Indicates that the status Next, using the policy network Generated motion and triangular mesh models The distance between them.

[0230] S3513: Based on the trained policy network, the robot predicts the actions it should perform and generates continuous path points. It then combines the continuity constraints between adjacent path points and the surface fit constraints between path points and large, complex components to construct a spatiotemporal joint optimization objective function, eliminate path jitter, and obtain an optimized continuous path.

[0231] When performing path jitter elimination, a spatiotemporal joint optimization objective function is constructed to smooth the path sequence. The constructed objective function is expressed as:

[0232] ;

[0233] Where, Objective function representing spatio-temporal joint optimization Continuity constraint for minimizing the Euclidean distance between adjacent path points to ensure smooth robot arm motion Surface fitting constraint for penalizing the deviation of path points from the surface of large complex components

[0234] Solve the objective function by Gauss-Seidel algorithm, output path point sequence satisfying continuity condition .

[0235] S3514: Based on the path point spacing between robots, real-time path conflict detection is performed, and directional adjustment is performed based on the path conflict detection result to generate an optimized path covering the surface of large complex components

[0236] In a multi-robot collaborative scenario, real-time path conflict detection is performed, and when the path points of any two robots satisfy , the repulsive adjustment mechanism is triggered, which is represented as:

[0237] ;

[0238] ;

[0239] In the formula, represents the corrected path point of the th robot represents the uncorrected path point of the th robot represents the normal of the th robot at the corresponding point of the surface of the large complex component represents the modulus of the normal vector ; represents the corrected path point of the th robot represents the uncorrected path point of the th robot represents the normal of the th robot at the corresponding point of the surface of the large complex component represents the modulus of the normal vector ;

[0240] By adjusting the offset along the normal, it is ensured that the corrected path is still located on the surface of the large complex component, i.e. the triangular mesh model, and the conflict can be quickly eliminated through directional adjustment.

[0241] Wherein, the generated optimized path covering the surface of large complex components is represented as:

[0242] ;

[0243] Where, Represents the optimized path covering the surface of a large and complex component; After optimization The three-dimensional coordinates of the path point at that moment; After optimization the speed of the moment; After optimization The surface direction vector at the moment; , Indicates the total number of moments.

[0244] S4: Share the optimized paths and states of multiple robots in real time, predict potential future conflicts, and dynamically and collaboratively adjust the paths and states of multiple robots based on the prediction results to generate the final path and control instructions to achieve multi-robot collaborative measurement path planning;

[0245] S41: Share optimized paths and states of multiple robots in real time;

[0246] In this embodiment, each robot shares its optimized path in real time through the communication network and status, the robot's status includes the current position ,speed ,attitude , residual energy and measurement status Shared data is updated at a 10Hz frequency, with communication latency kept to under 10ms, ensuring real-time multi-machine collaboration. During actual processing, robot state information is smoothed using a Kalman filter to ensure data reliability and provide accurate input for conflict prediction and progress synchronization.

[0247] S42: Based on the optimized path and speed of each robot, predict the future position of the robot and perform future position conflict determination between the robots; the future position conflict determination between the robots includes triggering conflict adjustment when the distance between the path points of the two robots at a future moment is lower than a preset distance threshold;

[0248] In this embodiment, based on the current path of each robot and speed , and make position predictions for the next 10 steps Specifically, when any two robots meet in the future At any time, adjustments are triggered.

[0249] S43: Based on the future position conflict judgment results, the paths and states of each robot are dynamically and collaboratively adjusted to generate the final path and control instructions to achieve multi-robot collaborative measurement path planning;

[0250] S431: When it is predicted that the future positions of two robots conflict, then the conflict weight is adjusted using the robot priority matrix, the path of the robot with low priority is adjusted preferentially, and the adjusted robot path is taken as the final path;

[0251] Due to the existence of robot field of view overlap, the priority matrix is used to adjust the conflict weight, which can ensure that the high-priority robot retains the path in the conflict and the low-priority robot actively avoids. The conflict weight adjustment is represented as:

[0252] ;

[0253] In the formula, represents the priority of the adjusted i-th robot relative to the j-th robot, that is, the closer the distance, the greater the priority adjustment amplitude; represents the priority of the unadjusted i-th robot relative to the j-th robot; represents the path point position of the i-th robot at the t time; represents the path point position of the j-th robot at the t time;

[0254] wherein the path adjustment of the robot is represented as:

[0255] ;

[0256] In the formula, represents the adjusted path point position of the i-th robot at the t time, and the adjusted path point position fits the triangular mesh model, that is, , represents the point in the triangular mesh model, represents the triangular mesh model, represents the function of making the independent variable value of the target function minimum; , that is, the priority of the i-th robot is higher than that of the j-th robot;

[0257] S432: When it is predicted that the measurement progress of two robots is different, then the speed of the robot with the lagging measurement progress is adjusted preferentially;

[0258] wherein the speed adjustment of the robot is represented as:

[0259] ;​​​​​​​​​​​​

[0260] wherein, represents the adjusted speed of the i-th robot; represents the unadjusted speed of the i-th robot; represents the proportion of the covered points of the i-th robot to the points of the divided sub-region; represents the proportion of the covered points of the i-th robot to the points of the divided sub-region; , i.e., the i-th robot lags behind the measurement progress of the j-th robot;

[0261] S433: in combination with the measurement accuracy of the robot, dynamically adjust the robot pose;

[0262] wherein, the robot pose adjustment is represented as:

[0263]

[0264] wherein, represents the adjusted pose of the i-th robot; represents the unadjusted pose of the i-th robot; represents the angle between the sensor field of view of the i-th robot and the normal vector of the curved surface;

[0265] S434: in combination with the robot priority matrix, as well as the adjusted robot speed and robot pose, as control instructions, and in combination with the final path, to realize multi-robot collaborative measurement path planning.

[0266] Embodiment 2

[0267] Based on the same technical concept, as shown in Figure 2 , the embodiment also provides a multi-robot collaborative measurement path planning based on reinforcement learning, including a data acquisition and preprocessing module, a model construction module, a path generation module, a path adjustment module, and a control module.

[0268] Specifically, the data acquisition and preprocessing module is used to acquire the model and point cloud data of a large complex component and to perform preprocessing;

[0269] Specifically, the model construction module is used to construct a three-dimensional environment model based on the preprocessed point cloud data, and to define the state space of the reinforcement learning environment, the executable action space, and the reward function of the reinforcement learning process; ​​​​​​​​​​​

[0270] Specifically, the path generation module is configured to assign the measurement task to the plurality of robots based on the constructed three-dimensional environment model, generate an initial path covering the surface of the large and complex component, and optimize the initial path by using a reinforcement learning method to generate an optimized path.

[0271] Specifically, the path adjustment module is configured to share the optimized paths and states of the plurality of robots in real time, predict potential future conflicts, and dynamically adjust the paths and states of the plurality of robots based on the prediction results to generate final paths and control instructions, so as to realize multi-robot collaborative measurement path planning.

[0272] Specifically, the control module is configured to input the control instructions into the system to perform high-precision collaborative measurement operations of the plurality of robots.

[0273] Embodiment 3

[0274] Based on the same technical concept, the embodiments of the present application also provide a computer device, which comprises a memory 1 and a processor 2, as shown in Figure 3 The memory 1 stores a computer program, and the processor 2 executes the computer program to implement the method of any one of the above embodiments.

[0275] The memory 1 comprises at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 1 can be an internal storage unit of the reinforcement learning-based multi-robot collaborative measurement path planning, such as a hard disk. In other embodiments, the memory 1 can also be an external storage device of the reinforcement learning-based multi-robot collaborative measurement path planning, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 1 can include both the internal storage unit and the external storage device of the reinforcement learning-based multi-robot collaborative measurement path planning. The memory 1 can be used not only to store application software and various data of the reinforcement learning-based multi-robot collaborative measurement path planning, such as the code of the reinforcement learning-based multi-robot collaborative measurement path planning program, etc., but also to temporarily store data that has been output or will be output.

[0276] The processor 2 may, in some embodiments, be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, used to run program codes stored in the memory 1 or process data, such as executing a multi-robot cooperative measurement path planning program based on reinforcement learning.

[0277] The application discloses an embodiment of a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in the above method embodiments are performed. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0278] The application discloses an embodiment of a computer program product of a multi-robot cooperative measurement path planning method based on reinforcement learning, which includes a computer readable storage medium storing program codes. The program codes include instructions for performing the steps of the method described in the above method embodiments. For details, refer to the above method embodiments, which will not be repeated here.

[0279] The application discloses an embodiment of a computer program, which, when executed by a processor, implements any method of the above embodiments. The computer program product can be implemented by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), and the like.

[0280] It can be understood that the same or similar parts in the above embodiments can be mutually referred to, and the content not described in detail in some embodiments can be referred to the same or similar content in other embodiments.

[0281] It should be noted that, in the description of the application, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance. In addition, in the description of the application, unless otherwise specified, "a plurality of" means at least two.

[0282] Any process or method descriptions in flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for performing specified logic functions or steps, and the various embodiments of the application include additional or fewer functions performed in the same order or in a different order, and with additional functions, in which the functions can be performed at substantially the same time or in reverse order. This should be understood by those skilled in the art of the embodiments of the application.

[0283] It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, and as in another embodiment, it can be realized by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0284] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by a program instructing the relevant hardware, and the program can be stored in a computer readable storage medium, and when executed, includes one or a combination of steps of the embodiment method.

[0285] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can exist physically, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of software functional module. The integrated module, if realized in the form of software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0286] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0287] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0288] Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A multi-robot collaborative measurement path planning method based on reinforcement learning, characterized in that: include: Obtain models and point cloud data of large and complex components and perform pre-processing; Build a 3D environment model based on preprocessed point cloud data and define the state space of the reinforcement learning environment, the executable action space, and the reward function of the reinforcement learning process; Based on the constructed 3D environment model, the measurement tasks are assigned to multiple robots to generate an initial path covering the surface of large and complex components. The initial path is then optimized using reinforcement learning methods to generate an optimized path. Share the optimized paths and states of multiple robots in real time, predict potential future conflicts, and dynamically and collaboratively adjust the paths and states of multiple robots based on the prediction results to generate the final path and control instructions to achieve multi-robot collaborative measurement path planning; Based on the constructed 3D environment model, the measurement tasks are assigned to multiple robots to generate an initial path covering the surface of a large and complex component, including: Based on the division of large and complex components into multiple sub-areas, the complexity of each sub-area is quantified; Evaluate each robot's measurement performance based on its maximum speed, measurement accuracy, endurance, and field of view. Based on the measurement performance of each robot, a particle swarm optimization algorithm is used to assign measurement sub-areas to each robot, and high curvature sub-areas are preferentially assigned to robots with high measurement accuracy. Based on the sub-area assigned to each robot, the UV parameterized function is used to map the two-dimensional Z-shaped path to a three-dimensional surface. The three-dimensional coordinates and path direction of the path points are calculated, and the path points are globally connected. Combined with real-time collision detection between path points, the measurement path and corresponding time-speed sequence of each robot are adjusted and generated to form an initial path covering the surface of large and complex components.

2. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 1 is characterized in that: The acquisition of the model and point cloud data of the large and complex component and pre-processing thereof include: Directly obtain the CAD model of large and complex components and convert it into the first triangle mesh model; Use a laser scanner to scan large and complex components to obtain point cloud data on the surface of large and complex components, perform denoising and downsampling processing, and generate downsampled point cloud data; Based on the first triangular mesh model and downsampled point cloud data, the iterative closest point algorithm is used for alignment.

3. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 2 is characterized in that: The method of constructing a three-dimensional environment model based on the pre-processed point cloud data and defining the state space of the reinforcement learning environment, the executable action space, and the reward function of the reinforcement learning process includes: Based on the downsampled point cloud data, a Poisson surface reconstruction algorithm is used to generate a second triangular mesh model, which is combined with the first triangular mesh model to obtain the surface information of large and complex components. Based on the surface information of large and complex components, the surface of large and complex components is divided into multiple manageable sub-areas using surface geometric feature information and spatial position information to construct a three-dimensional environmental model; Based on the 3D environment model, the geometric representation of large and complex components is sampled and combined with multiple robot states and environment states to define the state space of the reinforcement learning environment. Define the executable action space based on the stand-alone action of a single robot and the coordinated action of multiple robots; Define the reward function for the reinforcement learning process based on coverage, efficiency, conflict avoidance, and quality measurements of large complex components.

4. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 3 is characterized in that: Based on the surface information of large and complex components, the surface is divided into multiple manageable sub-areas using surface geometric feature information and spatial position information to construct a 3D environment model, including: Based on the surface information of large and complex components, the surface of large and complex components is divided into multiple initial sub-regions using surface geometric feature information and spatial feature information; The multiple initial sub-regions are clustered using the similarity matrix and the Laplace matrix to form multiple manageable sub-regions and construct a three-dimensional environment model.

5. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 3 is characterized in that: When defining the executable action space, the coordinated actions of multiple robots include task reallocation and a robot priority matrix, which is used to prioritize the allocation of high curvature sub-areas and to actively avoid conflicts with low-priority robots.

6. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 1 is characterized in that: The method of optimizing the initial path by using the reinforcement learning method to generate the optimized path includes: Based on the initial path, the trained policy network and evaluation network are used to predict the actions of each robot, calculate the continuous path points, and combine the real-time detection of path conflicts between path points of robots to adjust and generate the optimized path covering the surface of large and complex components.

7. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 1 is characterized in that: Based on the sub-area assigned to each robot, a UV parameterized function is used to map the 2D zigzag path to a 3D surface. The 3D coordinates and path directions of the path points are calculated, and the path points are globally connected. Combined with real-time collision detection between the path points, the measurement path and corresponding time-velocity sequence of each robot are adjusted and generated to form an initial path covering the surface of large and complex components, including: Based on the sub-area assigned to each robot, the UV parameterization function is used to map the two-dimensional Z-shaped path to a three-dimensional surface to obtain a three-dimensional path, and the three-dimensional coordinates of each path point and the path direction are determined; Based on the path point sequence, a piecewise cubic spline interpolation algorithm is used to smooth the path segments formed by connecting the path points, and to perform global connection of each path point. The global connection of each path point includes assigning weights to the path segments and corresponding sub-regions, and using an optimization algorithm to solve the optimal path segment connection sequence. Real-time collision detection is performed based on the distance between the path points of each robot, and path correction is performed based on the real-time collision detection results. The initial path point set and its corresponding time-speed sequence for each robot are generated to form an initial path covering the surface of large and complex components. The initial path point set of each robot is expressed as: ； Where, Indicates the The initial path point set of each robot; Indicates the The first robot waypoints; The time-velocity sequence of each robot is expressed as: ； Where, Indicates the Time-velocity sequence of each robot; Indicates the The first robot The collection of times and speeds of the waypoints.

8. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 6 is characterized in that: Based on the initial path, the trained policy network and evaluation network are used to predict the actions of each robot, calculate continuous path points, and combine the real-time detection of path conflicts between path points of robots to adjust and generate an optimized path covering the surface of large and complex components, including: Taking the initial path as the starting point, we design a policy network and an evaluation network to output the actions that each robot should perform in its current state, and to evaluate the expected cumulative rewards for a given state and all actions performed by the robots. Based on a triangular mesh model, a priority replay strategy is used to focus on key sub-areas, surface constraints are added to limit the robot's range of motion, and the evaluation network is updated by minimizing the temporal difference error and the policy network parameters are updated by maximizing the reward value, resulting in a trained policy network. Based on the trained policy network, the robot predicts the actions it should perform and generates continuous path points. It then combines the continuity constraints between adjacent path points and the surface fit constraints between path points and large, complex components to construct a spatiotemporal joint optimization objective function, eliminate path jitter, and obtain an optimized continuous path. Based on the distance between the path points between robots, real-time path conflict detection is performed, and directional adjustments are made based on the path conflict detection results to generate an optimized path that covers the surface of large and complex components; The optimized path for covering the surface of a large complex component is expressed as: ； Where, Represents the optimized path covering the surface of a large and complex component; After optimization The three-dimensional coordinates of the path point at that moment; After optimization the speed of the moment; After optimization The surface direction vector at the moment; , Indicates the total number of moments.

9. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 1 is characterized in that: The system shares the optimized paths and states of multiple robots in real time, predicts potential future conflicts, and dynamically and collaboratively adjusts the paths and states of multiple robots based on the prediction results to generate the final path and control instructions to achieve multi-robot collaborative measurement path planning, including: Share optimized paths and states of multiple robots in real time; the robot states include current position, speed, posture, remaining energy, and measurement status; Based on the optimized path and speed of each robot, the future position of the robot is predicted, and future position conflict determination between the robots is performed; the future position conflict determination between the robots includes triggering conflict adjustment when the distance between the path points of the two robots at a future time is lower than a preset distance threshold; Based on the future position conflict judgment results, the paths and states of each robot are dynamically and collaboratively adjusted to generate the final path and control instructions to realize multi-robot collaborative measurement path planning.

10. The multi-robot collaborative measurement path planning method based on reinforcement learning according to claim 9, characterized in that: The method dynamically and collaboratively adjusts the paths and states of each robot based on the future position conflict judgment results, generates the final path and control instructions, and realizes multi-robot collaborative measurement path planning, including: When a conflict between the future positions of two robots is predicted, the robot priority matrix is ​​used to adjust the conflict weight, giving priority to adjusting the path of the robot with the lower priority, and the adjusted robot path is used as the final path; The conflict weight adjustment is expressed as: ； Where, Indicates the adjusted The robot is relative to the The priority of each robot; Indicates the unadjusted The robot is relative to the The priority of each robot; Indicates the A robot in The location of the waypoint at the moment; Indicates the A robot in The location of the waypoint at the moment; The path adjustment of the robot is expressed as: ； Where, Indicates the A robot in The position of the path point is adjusted at all times, and the adjusted path point position fits the triangular mesh model, that is, , Represents a point in a triangular mesh model, represents a triangular mesh model, A function that represents the value of the independent variable that minimizes the objective function; , that is, The robot has higher priority than the A robot; When it is predicted that the measurement progress of two robots is different, the speed of the robot with the lagging measurement progress is adjusted first; The speed adjustment of the robot is expressed as: ； Where, Indicates the The robot accelerates to the adjusted speed; Indicates the The unadjusted speed of each robot; Indicates the The percentage of coverage points of each robot relative to the number of points in the divided sub-areas; Indicates the The percentage of coverage points of each robot relative to the number of points in the divided sub-areas; , which means the The robot lags behind A robot measures progress; Combined with the robot's measurement accuracy, the robot's posture is dynamically adjusted; The robot posture adjustment is expressed as: ； Where, Indicates the The posture of each robot after acceleration adjustment; Indicates the The unadjusted posture of the robot; Indicates the Sensor field of view and surface normal vector of each robot Angle The robot priority matrix, as well as the adjusted robot speed and robot posture, are combined as control instructions and combined with the final path to achieve multi-robot collaborative measurement path planning.

Images