A control system for an automated road grouting repair robot

Abstract

This invention discloses an automatic road grouting repair robot control system, belonging to the field of robotic equipment automation control technology. The system includes: a path planning module, a work area positioning module, a crack recognition module, and a grouting robotic arm control module. The path planning module is used for path planning; the work area positioning module is used to realize the automatic movement of the robot to the work target; the crack recognition module is used to identify the crack information of the work target based on a deep learning image algorithm; and the grouting robotic arm control module is used to control the movement of the grouting robotic arm and complete the grouting operation on the work target. Based on the characteristics of road repair operation scenarios, this invention collects training images and uses a deep learning algorithm to train a model based on the image information from the training image database. Through the trained model, the automatic road grouting repair robot can accurately and automatically repair road repair targets, especially those in complex situations.

Description

Technical Field

[0001] This invention relates to the field of automated control technology for robotic equipment, and in particular to an automated road surface grouting repair robot control system based on smart roads. Background Technology

[0002] The main research goals in the field of road repair today are to improve repair efficiency, reduce labor costs, and improve road quality. Currently, traditional road repair operations often rely on manual operation, which has limited efficiency and may be inaccurate. Smart road technology is gradually being widely applied in the field of road repair.

[0003] However, for road damage image recognition technology, a large amount of labeled training data is required to build an accurate model, and the acquisition and processing of labeled data can be a time-consuming and tedious process. At the same time, the working environment of road repair is usually complex and variable, including different types of road surfaces, traffic conditions, and weather conditions. The robot's perception and decision-making capabilities in such complex environments still face challenges, and its automated control capabilities are not ideal. Summary of the Invention

[0004] This invention provides an automatic road grouting repair robot control system to solve the technical problems of limited efficiency in traditional manual road repair operations and limited perception and decision-making capabilities of existing robot-based road repair operations in complex environments.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] On the one hand, the present invention provides an automatic road grouting repair robot control system, including: a path planning module, a work area positioning module, a crack recognition module, and a grouting robotic arm control module;

[0007] The path planning module is used to plan the path of the automatic road grouting repair robot based on the artificial potential field method according to the road repair operation scenario and operation requirements. It also introduces the random fast search tree algorithm to adjust the gravitational constant and repulsive constant of the artificial potential field method, and completes the heuristic search for path planning.

[0008] The work area positioning module is used to establish a work area positioning model based on the FastSLAM algorithm. Based on the path planning results of the path planning module, spatial recognition is performed by the real-time video stream of the work space collected by the built-in camera of the automatic road grouting repair robot. The motion trajectory of the automatic road grouting repair robot is estimated by the posterior probability distribution to complete the automatic movement of the robot to the work target.

[0009] The crack identification module is used to identify crack information of the target object based on deep learning image algorithms;

[0010] The grouting robotic arm control module is used to control the movement of the grouting robotic arm of the automatic road grouting repair robot and complete the grouting operation on the target based on the recognition result of the crack recognition module.

[0011] Furthermore, the automatic road grouting repair robot control system also includes an abnormal situation handling module;

[0012] The abnormal situation handling module is used to issue an alarm and switch to manual control when an abnormality is being repaired.

[0013] Furthermore, the automatic road grouting repair robot control system also includes a feedback module;

[0014] The feedback module is used to determine the repair effect level after the grouting operation is completed, and to provide feedback control to the automatic road grouting repair robot based on the determination result of the repair effect level.

[0015] Furthermore, the automatic road grouting repair robot control system also includes a post-processing module;

[0016] The post-processing module is used to control the robot to automatically clean up the crushed stone dust after the repair work is completed, and to store the repair process data into the repair work database to record the repair process and continue to optimize the model.

[0017] Furthermore, based on the road repair operation scenario and requirements, path planning is performed on the automatic road grouting repair robot using the artificial potential field method. A random fast search tree algorithm is introduced to adjust the gravitational and repulsive constants of the artificial potential field method, completing a heuristic search for path planning, including:

[0018] When performing path planning for an automated road grouting repair robot based on the artificial potential field method, the gravitational potential energy function U is used. att The calculation formula is:

[0019]

[0020] Where ε is the gravitational constant; p is the robot's current position; p goal ρ(p,p) represents the target point of the robot. goal () is a vector pointing from the robot's position to the target point, and its magnitude is the Euclidean distance between the robot's position and the target point;

[0021] Gravitational function F att The calculation formula is:

[0022] F att=-ερ(p,p goal )

[0023] Repulsive potential energy function U rep The calculation formula is:

[0024]

[0025] Where γ is the repulsive constant; p represents the radius of the repulsive potential field and the maximum distance at which the obstacle exerts its effect on the robot. bar Let ρ(p,p) be the location of the obstacle. bar ) is a vector pointing from the obstacle's position to the robot's position, and its magnitude is the Euclidean distance between the robot's position and the obstacle;

[0026] Repulsive force function F rep The calculation formula is:

[0027]

[0028] in, This represents the unit vector pointing from the obstacle to the robot; This represents the unit vector pointing from the robot to the target point;

[0029] The total potential energy U of the robot used t Total Resultant Force F t The calculation formula is as follows:

[0030] U t =U att +U rep

[0031] F t =F att +F rep

[0032] By introducing a random fast search tree algorithm to adjust the gravitational constant ε and repulsive constant γ in the artificial potential field method, a heuristic search is performed, at which point the resultant force F is obtained. t ′ otal for:

[0033]

[0034] Where N is the number of obstacles; Let ε represent the repulsive force of the i-th obstacle. By changing the gravitational constant ε and the repulsive constant γ, the weights of the gravitational and repulsive forces are adjusted. The larger the repulsive constant γ, the farther the generated path will be from the obstacle. In this case, the main function of the artificial potential field method is to guide the robot to avoid the obstacle. The larger the gravitational constant ε, the faster the generated path will bring the robot closer to the target point. In this case, the main function of the artificial potential field method is to guide the robot to search towards the target point, thereby completing the heuristic search for path planning. The algorithm implementation process is as follows:

[0035] Step 1: Initialize the algorithm and set the starting point X. init Target point X goal And step size s, randomly sampling state points X from the environment. rand ;

[0036] Step 2: Begin the random tree growth process. First, divide the working area into sections. In each section, introduce the gravitational constant ε and repulsive constant γ from the artificial potential field method. Adjust these constants to make the random tree growth path farther from obstacles and closer to the target point faster, with X... init and X rand The connection direction is used as the growth direction of the tree. A step size s is set in this direction as the step size of one growth cycle of the tree. A new node X is generated at the end of the growth cycle. new Determine from X init To X rand Whether the obstacle was passed through; if not, then X. rand A node is added to the tree. If it is traversed, the current new node is abandoned and a new direction is chosen for growth.

[0037] Step 3: Repeat the tree growth process until the distance from the newly generated node to the target point is less than one step size s. Then, terminate the tree growth, directly connect the current new node to the target point, and the algorithm ends.

[0038] Furthermore, the work area localization model established based on the FastSLAM algorithm, and the spatial recognition based on the path planning results of the path planning module, using the real-time video stream of the workspace collected by the built-in camera of the automatic road grouting repair robot, and estimating the motion trajectory of the automatic road grouting repair robot using the posterior probability distribution, completes the automatic movement of the robot to the work target, including:

[0039] The motion model and observation model of the FastSLAM algorithm are represented as follows:

[0040] P(X r (k)|u(k),X r (k-1))=f(u(k),X r (k-1)+ω(k)

[0041] P(Z(k)|X r (k),X n (k),n(k))=h(X r (k),X n (k))+v(k)

[0042] Among them, X r (k) represents the robot pose at time k; X r (k-1) represents the robot pose at time k-1; Z(k) represents the observed position of the robot at time k; u(k) represents the motion value of the robot at time k; n(k) represents the landmark mark; X n (k) represents the information of the landmarks of n(k); f and h are nonlinear functions; ω(k) is Gaussian white noise for the motion model; v(k) is Gaussian white noise for the observation model; P(.|.) is the robot trajectory model represented by the posterior probability distribution estimate;

[0043] The robot's motion trajectory is determined based on the FastSLAM algorithm. and Environmental Modeling X n (k) can be expressed using posterior probabilities as follows:

[0044]

[0045] Where, n k =n(1),…,n(k) are road sign markings; Z k =Z(1),…,Z(k) are the observed values ​​of the robot's position; u k =u(1),…,u(k) are the motion values ​​of the robot's position; N is the number of obstacles;

[0046] The process of establishing environmental information involves the following steps:

[0047] Step 1: Obtain a set of particles describing the robot's prior pose by sampling from the robot's kinematic model. The robot's kinematic model is represented by Cartesian coordinates. Transform the Frenet coordinate system to the Cartesian coordinate system. In the Frenet coordinate system, the vehicle state is represented as:

[0048]

[0049] Where s is the longitudinal displacement; Longitudinal velocity; d is the longitudinal acceleration; d is the lateral displacement; For lateral velocity; d is the lateral acceleration; d′ is the first derivative of d with respect to the arc length; d″ is the second derivative of d with respect to the arc length;

[0050] The lateral displacement of the mobile robot is represented in Cartesian coordinates as follows:

[0051]

[0052] in, To repair the robot's trajectory curve; The position vector along the reference line of the robot's current position; This is the orthogonal unit normal vector along the reference line direction of the robot's current position;

[0053] The lateral movement speed is:

[0054]

[0055] in, This is the orthogonal unit normal vector along the reference line direction of the robot's current position; v is the orthogonal unit tangent vector along the reference line direction of the robot's current position; x Let be the velocity of the moving robot in the Cartesian coordinate system; Δθ is the angle between the robot's current position and the direction of the reference line in the Cartesian coordinate system.

[0056] The robot's velocity in the Frenet coordinate system is expressed as:

[0057]

[0058] Where, τ r The curvature of the mobile robot's current position; To Find the time derivative; from this, we get:

[0059]

[0060]

[0061] The robot's velocity and acceleration are:

[0062]

[0063]

[0064] in, To Find the time derivative;

[0065] Step 2: Adjust the weights of the particles in the particle set describing the robot's prior pose. After calculating the particle weights, calculate the posterior probability distribution of the robot's state based on the weights and their corresponding states. The particle weights are represented as:

[0066]

[0067] in, Z represents the motion trajectory of the robot in state k-1; k-1 The observed value of the robot's position in state k-1; u k-1 n represents the motion value of the robot's position in state k-1; k-1 Mark the k-1 state landmark;

[0068] Step 3: Resample the particle set using a hierarchical resampling algorithm, which divides the sampling region into continuous, non-overlapping intervals and performs independent, identically distributed sampling on each sub-interval to obtain robot position deviation data for correction control.

[0069] Furthermore, the method for identifying crack information in the target area based on deep learning image algorithms includes:

[0070] Crack images are collected from the work scene and digitized. Based on a deep neural network framework, the digitized images are divided into training and testing sets. To avoid overfitting, the number of images collected per unit time is increased and multiple iterative calculations are performed.

[0071] The trained network model is loaded into the camera processing system carried by the automatic road grouting repair robot, and the crack images it collects are processed to identify crack information.

[0072] Furthermore, the method for digitizing the crack images is as follows:

[0073] All crack images are normalized to obtain an image information database with consistent size and pixel dimensions. All image information is then gridded to obtain 256*256 pixel squares, i.e., 256 pixels horizontally and 256 pixels vertically. Each pixel is used as the data basis for image training.

[0074] Furthermore, the deep neural network framework is a convolutional neural network;

[0075] The method for identifying crack information in the target area based on deep learning image algorithms includes:

[0076] Using a spatial feature-based image recognition algorithm, spatial feature extraction is based on the ResNeXt network architecture. An improved ResNeXt network is used to perform a secondary partitioning of the feature map groups, resulting in a total of:

[0077] G = KR

[0078] Where K represents the cardinality hyperparameter for the first partition; R represents the cardinality hyperparameter for further partitioning within the feature map group;

[0079] Summing element-wise across multiple components allows us to construct a combined representation from several radix arrays:

[0080]

[0081] in, H, W, and C represent the shapes of the block output feature maps, and R... k This refers to the k-th component after the first partitioning; U j For the j-th base array; through the spatial dimension s k ∈R C / K Global average pooling is used to aggregate global context information with embedded channel statistics. The c-th component of the k-th radix array is calculated as follows:

[0082]

[0083] in, For the c-th component, it is the set of base arrays;

[0084] Weighted aggregation is performed based on channel attention, where each feature map channel is generated by a weighted combination. The weighted fusion representation of the c-th channel is as follows:

[0085]

[0086] in, The base array is divided into two sets after the feature map group is defined. Indicates the weight allocation:

[0087]

[0088] in, s k Determine the weight of each component in the c-th channel;

[0089] The improved ResNeXt network refers to replacing the standard 3×3 convolution in the ResNeXt network with involution. In the involution operation of the improved ResNeXt network, the shape of the involution kernel is generated by the shape of the feature map of the input data and is spatially aligned with the size of the feature map. The involution kernel generation process is represented as follows:

[0090]

[0091]

[0092] Where, φ i,j Let (i,j) be the set of pixels in the neighborhood of coordinate (i,j). ξ represents a single pixel at coordinates (i,j) in the feature map; ξ is the non-linear activation function of the processing layer. and For the linear transformation matrix, this invention uses a 3×3 standard convolution with r set to 3;

[0093] The Mixup data augmentation strategy is used to generate random images from the training data and perform weighted combinations.

[0094] Furthermore, the grouting robotic arm is a six-degree-of-freedom robotic arm;

[0095] The movement of the grouting robotic arm of the automatic road grouting repair robot includes:

[0096] Using a six-degree-of-freedom robotic arm model, m path points are preset during the robotic arm's operation. Given the path point sequence at the end effector of the six-degree-of-freedom robotic arm, and after inverse kinematics calculations, the angular displacement sequence Q = [Q1, Q2, ..., Q...] of the six rotary joints is obtained. m ] T Where, the joint angular displacement vector Q of the i-th path point i =[Q i1 Q i2 Q i3 Q i4 Q i5 Q i6 ], i=1,…,m, j=1,2,…,6, Q ij Let be the joint angular displacement vector at the i-th path point in the j-th degree of freedom. The six dimensions of this vector represent the six rotational joints of the robot. The ultimate goal is to obtain the motion trajectories of each joint at the start and end path points where both joint angular velocity and joint angular acceleration are zero, while ensuring the continuity of the joint angular acceleration curves at each path point. The motion trajectory parameters of each joint are transformed into variables that depend only on time t. The angular displacements of each joint of the six-degree-of-freedom robotic arm are expressed as:

[0097] q(t)=[q1(t),q2(t),q3(t),q4(t),q5(t),q6(t)] T

[0098] Where, q j (t) represents the joint angular displacement in the j-th degree of freedom;

[0099] The angular velocities of each joint of a six-degree-of-freedom robotic arm are expressed as follows:

[0100]

[0101] Where, ω j (t) represents the joint angular velocity in the j-th degree of freedom;

[0102] The angular accelerations of each joint of a six-degree-of-freedom robotic arm are expressed as follows:

[0103]

[0104] Among them, a j (t) represents the joint angular acceleration in the j-th degree of freedom;

[0105] The joint angular pulsation characteristics of a six-DOF robotic arm are expressed as follows:

[0106]

[0107] Here, jert(t) is defined as the pulsation function of the robotic arm's joints, mathematically defined as the joint angular jerk, used to measure the smoothness of the robotic arm's trajectory. The smaller the value, the smoother the robotic arm's joint trajectory and the more stable the robotic arm. j (t) represents the joint angle pulsation characteristic in the j-th degree of freedom;

[0108] The pulsation characteristics of the robotic arm are defined as follows:

[0109]

[0110] In the formula, S represents the average pulsation of the robotic arm joints. t represents the number of joints; T represents the total operating time of the robotic arm; t represents the time.

[0111] In the trajectory planning of the robotic arm, the movement process is divided into three stages. The first stage is the transition of the robotic arm from a folded state to an unfolded state, and the trajectory of the robotic arm in this stage is represented by a fifth-degree polynomial. The second stage is the movement of the robotic arm from the unfolded state to the starting point of the task target, and the trajectory of the robotic arm in this stage is represented by a seventh-degree polynomial. The third stage is the movement of the robotic arm from the starting point of the task target to the ending point of the task target, and the trajectory of the robotic arm in this stage is represented by a seventh-degree polynomial. The specific form is as follows:

[0112] The general functional relationship between the fifth-degree polynomial and time in the first stage of the robotic arm's motion is as follows:

[0113] q(t) = a0 + a1t + a2t 2 +a3t 3 +a4t 4 +a5t 5

[0114] Set the joint initiation angle position, joint termination angle position, joint initiation angular velocity, joint termination angular velocity, joint initiation angular acceleration, and joint termination angular acceleration as constraints:

[0115]

[0116] Where t0 is the starting time, t m The termination time;

[0117] The angular velocity of the robotic arm joints can be expressed as follows based on the polynomial-time function relationship:

[0118]

[0119] The joint angular acceleration of the robotic arm is expressed as follows:

[0120]

[0121] Therefore, we can conclude that:

[0122]

[0123] The general functional relationship between the seventh-degree polynomials of the second and third stages of the robotic arm's motion and time is as follows:

[0124] q(t) = a0 + a1t + a2t 2 +a3t 3 +a4t 4 +a5t 5 +a6t 6 +a7t 7

[0125] Set the following constraints: joint initiation angle position, joint termination angle position, joint initiation angular velocity, joint termination angular velocity, joint initiation angular acceleration, joint termination angular acceleration, joint initiation angular jerk, and joint termination angular jerk.

[0126]

[0127] The angular velocity of the robotic arm joints can be expressed as follows based on the polynomial-time function relationship:

[0128]

[0129] The joint angular acceleration of the robotic arm is expressed as follows:

[0130]

[0131] The function for the variation of joint angle pulsation characteristics of the robotic arm is:

[0132]

[0133] A smoothing method using parametric equations to fit the curve addresses the discontinuity at the transition point between the first-stage fifth-order polynomial and the second-stage seventh-order polynomial. The path is defined as a polynomial spiral parametric curve, and the tangent direction at points on the spiral is represented by a fifth-order polynomial function of arc length. The curve is divided into n-1 segments by n points. The parametric formula is as follows:

[0134] θ(s) = a + bs + cs2 +ds 3 +es 4 +fs 5

[0135] Where θ(s) represents the tangent direction at a point on the helix, s represents the arc length along the helix, and a, b, c, d, e, and f are the coefficients of a fifth-order polynomial; the coordinates on the curve are represented as:

[0136]

[0137] Where, x i y i z i Let P be the i-th point on the curve. i The coordinate values;

[0138] The starting direction, starting curvature, derivative of starting curvature, ending direction, ending curvature, derivative of ending curvature, and the curve length between the two points are used as constraints for each segment of the fifth-degree polynomial spiral:

[0139]

[0140] Where, θ i Starting from the direction, Starting with curvature, The derivative of the curvature at the starting point, The destination direction, For the endpoint curvature, The derivative of the curvature at the endpoint, Δs i θ is the length of the curve between two points; i (0) represents the direction at the starting point of the curve; The curvature at the starting point of the curve; θ is the derivative of curvature at the starting point of the curve; i (Δs i ) represents the distance Δs from the starting point of the curve. i Location and direction; Δs is the distance from the starting point of the curve. i Curvature at location; Δs is the distance from the starting point of the curve. i Derivative of curvature at position;

[0141] The continuity of the position, direction, curvature, and derivative of curvature at the junction of the two curve segments is used as a constraint condition:

[0142]

[0143] Where, x i+1 y i+1 z i+1 Let P be the (i+1)th point on the curve. i+1The coordinate values;

[0144] For point P on the curve i (x i ,y i ,z i Apply position translation constraints:

[0145]

[0146] Where i = 1, ..., n; r represents the position coordinates of the original reference point. i This is the distance that the original reference point can be translated.

[0147] In another aspect, the present invention also provides an electronic device comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the various models and algorithms involved in the above-mentioned automatic road grouting repair robot control system.

[0148] In another aspect, the present invention also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the various models and algorithms involved in the above-described automatic road grouting repair robot control system.

[0149] The beneficial effects of the technical solution provided by this invention include at least the following:

[0150] 1. This invention improves the adaptability of the automatic road grouting repair robot system in path planning problems. By introducing a random fast search tree for heuristic search and adjusting the gravitational and repulsive constants of the artificial potential field method, the adaptability of the automatic road grouting repair robot in the path planning process is effectively improved, enabling it to better adapt to different work scenarios and requirements. It also extends the path planning process under constraints. By establishing a work area localization model based on the FastSLAM algorithm, combining it with visual sensors for localization, and using posterior probability distribution to estimate the robot's trajectory, the automatic road grouting repair robot can automatically move to the work target and dynamically plan the path, while seeking the optimal path to reduce energy consumption.

[0151] 2. This invention improves the accuracy and efficiency of image recognition. By employing deep learning image algorithms to identify crack features, it scans and digitizes the target area requiring repair, thereby achieving precise grouting operations. By mounting the grouting instrument on a powered robotic arm, a correspondence is established between the grouting volume, crack size, and repair target, enabling the robot to perform precise grouting work based on instructions provided by the image recognition system. Based on the characteristics of road repair operation scenarios, this invention collects training images and uses deep learning convolutional neural network algorithms to train a model from the training image database. Through the trained model, the automatic road grouting repair robot can perform precise and automatic repair of road repair targets, especially those in complex situations.

[0152] 3. The automatic road grouting repair robot control system of the present invention has a wide range of applications, low economic cost, and can be promoted to social applications, bringing good social and economic benefits and has significant application prospects. Attached Figure Description

[0153] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0154] Figure 1 This is a flowchart of the automatic road grouting repair robot control system provided in an embodiment of the present invention. Detailed Implementation

[0155] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0156] First Embodiment

[0157] This embodiment provides a control system for an automatic road grouting repair robot. The system's workflow is as follows: Figure 1 As shown, the intelligent road data acquisition and sensing equipment senses the road condition and detects that the road surface is cracked and needs repair. Then, it transmits the repair signal to the automatic road grouting repair robot through the intelligent road communication transmission system. The robot performs the grouting work under the control of the automatic road grouting repair robot control system.

[0158] Specifically, the automatic road grouting repair robot control system of this embodiment includes: a path planning module, a work area positioning module, a crack recognition module, a grouting robotic arm control module, an abnormal situation handling module, a feedback module, and a post-processing module. The functions and working processes of each module are as follows:

[0159] The path planning module is used to perform path planning based on the artificial potential field method according to the road repair operation scenario and operation requirements. It establishes the kinematic model of the automatic road grouting repair robot, introduces a random fast search tree for heuristic search, and compensates for the defects of the traditional artificial potential field method by adjusting the gravitational and repulsive constants of the artificial potential field method. This effectively improves the adaptability of the automatic road grouting repair robot system in the path planning problem.

[0160] The work area positioning module is used to establish environmental information and build a work area positioning model based on the FastSLAM algorithm. Based on the path planning of the automatic road grouting repair robot, positioning is performed through a visual sensor. The model is stored in the system database. Spatial recognition is performed based on the real-time video stream of the work space collected by the robot's built-in camera, and the robot's motion trajectory is estimated using the posterior probability distribution. This expands the automatic road grouting repair robot's ability to plan its path under limited conditions, enabling the robot to automatically move to the work target and perform dynamic path planning. At the same time, it seeks the optimal path to reduce energy consumption and solves the robot's path planning problem based on different locations and obstacle distributions.

[0161] The crack identification module is used to collect target data, perform image feature recognition, extract crack features using deep learning image algorithms, scan the target area to be repaired, import the model into the system, collect crack image information in the work scene and digitize it, divide the digitized images into training and testing sets based on a deep neural network framework, increase the number of images collected per unit time to avoid overfitting, perform multiple iterative calculations, identify the crack feature model and import it into the system, load the trained network model into the camera processing system carried by the automatic road grouting repair robot, digitize the collected image information, and identify the crack information.

[0162] The grouting robotic arm control module is used to establish the correspondence between grouting volume, crack size, and repair target, storing this information in the system database. This allows the image recognition system to provide command instructions, while the power unit of the robotic arm provides physical guidance for the grouting work, completing precise grouting operations. The robotic arm is equipped with the grouting instrument of the automatic road grouting repair robot and an image acquisition device. The linkage circuit and controller of the grouting system are also embedded in the robotic arm of the automatic road grouting repair robot. The system database is built using MongoDB database technology, including data input, reading, modification, saving, and feedback for the automatic road grouting repair robot system. The grouting robotic arm is a six-degree-of-freedom robotic arm. The automatic road grouting repair robot uses a robotic arm to move the grouting nozzle, achieving precise rust removal for complex-shaped targets. The grouting robotic arm is equipped with an image acquisition device that can monitor the grouting repair process and its effect in real time, and provide feedback control for the movement of the grouting robotic arm.

[0163] The abnormal situation handling module is used to establish a method for handling abnormal situations, write it into the program, load the model after entering the working mode, handle any abnormalities, integrate the circuits of various systems, realize the wide application of the target repair requirements, provide real-time data feedback based on the operation effect, and dynamically adjust the automatic road grouting repair robot to complete the industrial repair operation.

[0164] The feedback module is used to determine the repair effect level after the grouting operation is completed, and to provide feedback control to the automatic road grouting repair robot based on the determination result of the repair effect level.

[0165] The post-processing module is used to control the robot to automatically clean up the crushed stone dust after the repair work is completed, and to store the repair process data into the repair work database to record the repair process and continue to optimize the model.

[0166] Furthermore, the path planning module uses the Artificial Potential Field (APF) method for path planning, establishing a kinematic model of the automatic road grouting repair robot. The detailed process is as follows: The Artificial Potential Field (APF) method selects the target point by constructing an artificial potential field, given the starting point, ending point, and obstacle positions. The planned path is usually smooth and effectively avoids obstacles. In the application scenario of this invention, the external obstacles when the automatic road grouting repair robot is working are mostly single obstacles such as roadblocks and work vehicles. The repulsive force field of the obstacles is easy to set, which is consistent with the application scenario of the automatic road repair robot in this invention. The working principle of the APF algorithm is: the target point generates an "attraction" to the moving robot, and the obstacle generates a "repulsive force" to the moving robot. Finally, the movement of the moving robot is controlled by calculating the resultant force. The gravitational potential energy function U used in this invention is... attThe calculation formula is:

[0167]

[0168] Where ε is the gravitational constant; p(x p ,y p ) represents the robot's current position; p goal (x goal ,y goal ) represents the robot's target point; ρ(p,p) goal Let be a vector pointing from the robot's position to the target point, and let its magnitude be the Euclidean distance between the robot's position and the target point.

[0169]

[0170] Gravitational function F att The calculation formula is:

[0171] F att =-ερ(p,p goal )

[0172] The repulsive potential energy function U used in this invention rep The calculation formula is:

[0173]

[0174] Where γ is the repulsive constant; p(x) represents the radius of the repulsive potential field and the maximum distance at which the obstacle exerts an effect on the robot. p ,y p ) represents the current location; p bar (x bar ,y bar ) represents the location of the obstacle; ρ(p,p) bar Let be a vector pointing from the obstacle's position to the robot's position, with a magnitude equal to the Euclidean distance between the robot's position and the obstacle.

[0175]

[0176] In traditional APF algorithms, when obstacles exist near the target point, the gravitational force drops to zero when the robot reaches the target point, but the obstacle still exerts a repulsive force on the target point, causing the robot to oscillate near the target point and fail to reach it. This invention improves the repulsive force function by decomposing the repulsive force experienced by the robot into two components, F. rep1 The direction is from the obstacle to the robot, F rep2 The direction is from the robot pointing to the target point. The combination of these two parts yields the resultant repulsive force on the robot, and the improved repulsive force function F rep The calculation formula is:

[0177]

[0178] in, This represents the unit vector pointing from the obstacle to the robot; This represents the unit vector pointing from the robot to the target point;

[0179] The total potential energy U of the robot used in this invention t Total Resultant Force F t The calculation formula is as follows:

[0180] U t =U att +U rep

[0181] F t =F att +F rep

[0182] In the traditional APF algorithm, if the net repulsive force of an obstacle at a certain position is equal in magnitude and opposite in direction to the gravitational force at the target point, the robot experiences zero net force and cannot continue moving towards the target point. Furthermore, as the robot moves further from the target point, the gravitational force increases while the repulsive force of the obstacle decreases, potentially causing the robot to collide with obstacles. To address these shortcomings, this invention improves upon the traditional APF algorithm by introducing the Rapid-exploration Random Tree (RRT) algorithm. This algorithm adjusts the gravitational constant ε and repulsive constant γ of the artificial potential field method, performing a heuristic search. The net force at this point is:

[0183]

[0184] Where N is the number of obstacles; Let represent the repulsive force described by the i-th obstacle. By changing the gravitational constant v and the repulsive constant γ, the weights of the gravitational and repulsive values ​​are adjusted. The larger the repulsive constant γ, the farther the generated path will be from the obstacle. In this case, the main role of the artificial potential field method is to guide the robot to avoid the obstacle. The larger the gravitational constant v, the faster the generated path will allow the robot to approach the target point. In this case, the main role of the artificial potential field method is to guide the robot to search towards the target point, thereby completing the heuristic search for path planning. By introducing the RTT algorithm into the traditional APF algorithm, the problems of the robot being unable to move towards the target point and colliding with obstacles are solved. At the same time, the convergence speed of the algorithm is improved and the computation time is reduced. The implementation process of the algorithm is as follows:

[0185] Step 1: Initialize the algorithm and set the starting point X. init Target point X goal And step size s, randomly sampling state points X from the environment. rand ;

[0186] Step 2: Begin the random tree growth process. First, divide the working area into sections. In each section, introduce the gravitational constant ε and repulsive constant γ using the artificial potential field method. Adjust these constants to make the random tree growth path farther from obstacles and closer to the target point faster, with X... init and X rand The connection direction is used as the growth direction of the tree. A step size s is set in this direction as the step size of one growth cycle of the tree. A new node X is generated at the end of the growth cycle. new Determine from X init To X rand Whether the obstacle was passed through; if not, then X. rand A node is added to the tree. If it is traversed, the current new node is abandoned and a new direction is chosen for growth.

[0187] Step 3: Repeat the tree growth process until the distance from the newly generated node to the target point is less than one step size s. Then, terminate the tree growth, directly connect the current new node to the target point, and the algorithm ends.

[0188] Furthermore, the work area localization module models the surrounding environment based on the FastSLAM algorithm and estimates the robot's trajectory using a posterior probability distribution. The FastSLAM algorithm motion model and observation model used can be expressed as follows:

[0189] P(X r (k)|u(k),X r (k-1))=f(u(k),X r (k-1)+ω(k)

[0190] P(Z(k)|X r (k),X n (k),n(k))=h(X r (k),X n (k))+v(k)

[0191] Among them, X r (k) represents the robot pose at time k; X r (k-1) represents the robot pose at time k-1; Z(k) represents the observed position of the robot at time k; u(k) represents the motion value of the robot at time k; n(k) represents the landmark mark; X n (k) represents the information of the landmarks of n(k); f and h are nonlinear functions; ω(k) is Gaussian white noise for the motion model; v(k) is Gaussian white noise for the observation model; P(.|.) is the robot trajectory model represented by the posterior probability distribution estimate;

[0192] This invention uses the FastSLAM algorithm to record the robot's motion trajectory. and Environmental Modeling X n (k) can be expressed using posterior probabilities as follows:

[0193]

[0194] Where, n k =n(1),…,n(k) are road sign markings; Z k =Z(1),…,Z(k) are the observed values ​​of the robot's position; u k =u(1),…,u(k) are the motion values ​​of the robot's position; N is the number of obstacles;

[0195] The process of establishing environmental information in this invention consists of the following steps:

[0196] Step 1: A set of particles describing the robot's prior pose is obtained by sampling the robot's kinematic model. The robot's kinematic model is represented by Cartesian coordinates. Frenet coordinates are commonly used for position representation on roads. Therefore, this invention converts the Frenet coordinate system to the Cartesian coordinate system for easier system calls. In the Frenet coordinate system, the vehicle state is represented as:

[0197]

[0198] Where s is the longitudinal displacement; Longitudinal velocity; d is the longitudinal acceleration; d is the lateral displacement; For lateral velocity; d is the lateral acceleration; d′ is the first derivative of d with respect to the arc length; d″ is the second derivative of d with respect to the arc length;

[0199] The lateral displacement of the mobile robot is represented in Cartesian coordinates as follows:

[0200]

[0201] in, To repair the robot's trajectory curve; The position vector along the reference line of the robot's current position; This is the orthogonal unit normal vector along the reference line direction of the robot's current position;

[0202] The lateral movement speed is:

[0203]

[0204] in, This is the orthogonal unit normal vector along the reference line direction of the robot's current position; v is the orthogonal unit tangent vector along the reference line direction of the robot's current position; x Let be the velocity of the moving robot in the Cartesian coordinate system; Δθ is the angle between the robot's current position and the direction of the reference line in the Cartesian coordinate system.

[0205] The robot's velocity in the Frenet coordinate system is expressed as:

[0206]

[0207] Where, τ r The curvature of the mobile robot's current position; To Find the time derivative; from this, we can obtain:

[0208]

[0209]

[0210] The robot's velocity and acceleration are:

[0211]

[0212]

[0213] in, To Find the time derivative;

[0214] Step 2: Adjust the weights of the particles in the particle set describing the robot's prior pose. After calculating the particle weights, calculate the posterior probability distribution of the robot's state based on the weights and their corresponding states. The particle weights are represented as:

[0215]

[0216] in, Z represents the motion trajectory of the robot in state k-1; k-1 The observed value of the robot's position in state k-1; u k-1 n represents the motion value of the robot's position in state k-1; k-1 Mark the k-1 state landmark;

[0217] Step 3: Resample the particle set using a hierarchical resampling algorithm, which divides the sampling region into continuous, non-overlapping intervals and performs independent, identically distributed sampling on each sub-interval to obtain robot position deviation data for correction control.

[0218] Furthermore, when identifying crack information of the target task based on deep learning image algorithms, the method for acquiring crack images in the required work scene is to use one or more combinations of drones, miniature imaging robots, and manual photography to obtain images of road cracks. The crack images are digitized as follows: all crack images are normalized using a local computer to obtain an image information database with consistent size and pixel dimensions; all image information is then gridded to obtain 256*256 pixel squares, i.e., 256 pixels horizontally and 256 pixels vertically, with each pixel serving as the data basis for image training. Furthermore, the deep neural network framework is primarily a convolutional neural network.

[0219] The detailed process of identifying crack information in the target work area based on deep learning image algorithms is as follows:

[0220] Using a spatial feature-based image recognition algorithm, spatial feature extraction is based on the ResNeXt network architecture. An improved ResNeXt network is used to perform a secondary partitioning of the feature map groups, resulting in a total of:

[0221] G = KR

[0222] Where K represents the cardinality hyperparameter for the first partition; R represents the cardinality hyperparameter for further partitioning within the feature map group;

[0223] Summing element-wise across multiple components allows us to construct a combined representation from several radix arrays:

[0224]

[0225] in, H, W, and C represent the shapes of the block output feature maps, and R... k This refers to the k-th component after the first partitioning; U j For the j-th base array; it can be passed across spatial dimensions s k ∈R C / K Global average pooling is used to aggregate global context information with embedded channel statistics. The c-th component of the k-th radix array is calculated as follows:

[0226]

[0227] in, For the c-th component, it is the set of base arrays;

[0228] Weighted aggregation is performed based on channel attention, where each feature map channel is generated by a weighted combination. The weighted fusion representation of the c-th channel is as follows:

[0229]

[0230] in, The base array is divided into two sets after the feature map group is defined. Indicates the weight allocation:

[0231]

[0232] in, s k Determine the weight of each component in the c-th channel;

[0233] This invention improves the ResNeXt network by replacing the standard 3×3 convolution with involution. Involution, in the spatial dimension, generates different involution kernels to extract local features from feature maps at different locations, enabling the timely processing of road damage features. It can not only identify crack width but also roughly predict crack depth based on image features, thus providing feedback to the automated repair robot to control the amount of grout injected, avoiding excessive injection of repair material at once. Furthermore, it shares involution kernels in the channel dimension, exhibiting spatial specificity and channel invariance. In the ResNeXt network's involution operation, the shape of the involution kernel is generated from the shape of the input data's feature map and is spatially aligned with the feature map size. The involution kernel generation process is represented as follows:

[0234]

[0235] Where, φ i,j Let (i,j) be the set of pixels in the neighborhood of coordinate (i,j). ξ represents a single pixel at coordinates (i,j) in the feature map; ξ is the non-linear activation function of the processing layer. and For the linear transformation matrix, this invention uses a 3×3 standard convolution with r equal to 3. This invention employs the Mixup data augmentation strategy to generate random images from the training data and perform weighted combination. Training example... It can be represented as:

[0236]

[0237]

[0238] Among them, (x i ,y i ), (x j ,y j ) represents two given image ground truth labels, and λ ~ Beta (α = 0.2) is independently sampled from the augmented samples.

[0239] Furthermore, the powered robotic arm of the automatic road grouting repair robot is driven by a power control algorithm. The detailed process is as follows: This invention uses a six-degree-of-freedom robotic arm model. During the operation of the robotic arm, m path points are preset. Given the path point sequence at the end of the six-degree-of-freedom robotic arm actuator, and then through inverse kinematics operations, the angular displacement sequence Q = [Q1, Q2, ..., Q...] of the six rotational joints is obtained. m ] T Where, the joint angular displacement vector Q of the i-th path point i =[Q i1 Q i2 Q i3 Q i4 Q i5 Q i6 ], i=1,…,m, j=1,2,…,6, Q ij Let be the joint angular displacement vector at the i-th path point in the j-th degree of freedom; the six dimensions of this vector represent the six rotational joints of the robot. The ultimate goal is to obtain the motion trajectories of each joint at the start and end path points where both joint angular velocity and joint angular acceleration are zero, and to ensure the continuity of the joint angular acceleration curves at each path point. The motion trajectory parameters of each joint are transformed into variables that depend only on time t. The angular displacements of each joint of the six-degree-of-freedom robotic arm are expressed as:

[0240] q(t)=[q1(t),q2(t),q3(t),q4(t),q5(t),q6(t)] T

[0241] Where, q j (t) represents the joint angular displacement in the j-th degree of freedom;

[0242] The angular velocities of each joint of a six-degree-of-freedom robotic arm are expressed as follows:

[0243]

[0244] Where, ω j (t) represents the joint angular velocity in the j-th degree of freedom;

[0245] The angular accelerations of each joint of a six-degree-of-freedom robotic arm are expressed as follows:

[0246]

[0247] Among them, a j (t) represents the joint angular acceleration in the j-th degree of freedom;

[0248] The joint angular pulsation characteristics of a six-DOF robotic arm are expressed as follows:

[0249]

[0250] Here, jert(t) is defined as the pulsation function of the robotic arm's joints, mathematically defined as the joint angular jerk, used to measure the smoothness of the robotic arm's trajectory. The smaller the value, the smoother the robotic arm's joint trajectory and the more stable the robotic arm. j (t) represents the joint angle pulsation characteristic in the j-th degree of freedom;

[0251] In this invention, the pulsation characteristics of the robotic arm are defined as:

[0252]

[0253] In the formula, S represents the average pulsation of the robotic arm joints. The number of joints is used in this invention, which uses a six-joint robotic arm, i.e., N=6, where j is the joint number; T is the total operating time of the robotic arm; and t is the time.

[0254] In the robotic arm trajectory planning, this invention divides the robotic arm's movement process into three stages. The first stage is the robotic arm's transition from a folded state to an unfolded state. During this stage, the robotic arm's posture is relatively fixed, and the trajectory is represented by a fifth-order polynomial. The second stage is the robotic arm's journey from the unfolded state to the starting point of the work target. During this stage, the robotic arm needs to perform a series of operations such as obstacle avoidance. To ensure a smooth trajectory, the second-stage trajectory is represented by a seventh-order polynomial. The third stage is the robotic arm's journey from the starting point of the work target to the ending point of the work target. This stage requires precise control of the robotic arm to achieve accurate repairs; therefore, the third-stage trajectory is represented by a seventh-order polynomial, as follows:

[0255] The general functional relationship between the fifth-degree polynomial and time in the first stage of the robotic arm's motion is as follows:

[0256] q(t) = a0 + a1t + a2t 2 +a3t 3 +a4t 4 +a5t 5

[0257] Set the joint initiation angle position, joint termination angle position, joint initiation angular velocity, joint termination angular velocity, joint initiation angular acceleration, and joint termination angular acceleration as constraints:

[0258]

[0259] Where t0 is the starting time, t m The termination time;

[0260] The angular velocity of the robotic arm joints can be expressed as follows based on the polynomial-time function relationship:

[0261]

[0262] Similarly, the joint angular acceleration of the robotic arm can be expressed as follows:

[0263]

[0264] Therefore, we can conclude that:

[0265]

[0266] The general functional relationship between the seventh-degree polynomials of the second and third stages of the robotic arm's motion and time is as follows:

[0267] q(t) = a0 + a1t + a2t 2 +a3t 3 +a4t 4 +a5t 5 +a6t 6 +a7t 7

[0268] Set the following constraints: joint initiation angle position, joint termination angle position, joint initiation angular velocity, joint termination angular velocity, joint initiation angular acceleration, joint termination angular acceleration, joint initiation pulsation characteristic (joint initiation angular acceleration), and joint termination pulsation characteristic (joint termination angular acceleration).

[0269]

[0270] The angular velocity of the robotic arm joints can be expressed as follows based on the polynomial-time function relationship:

[0271]

[0272] Similarly, the joint angular acceleration of the robotic arm can be expressed as follows:

[0273]

[0274] The function for the variation of joint angle pulsation characteristics of the robotic arm is:

[0275]

[0276] This invention innovatively solves the problem of discontinuity at the switching point between the first-stage fifth-order polynomial and the second-stage seventh-order polynomial by using a smoothing method that fits the curve with parametric equations. The path is defined as a polynomial spiral parametric curve, and the tangent direction at points on the spiral is represented by a fifth-order polynomial function of arc length. There are n points P on the curve. i (x i ,y i ,z i Divide the curve into n-1 segments, and the parameterization formula is as follows:

[0277] θ(s) = a + bs + cs 2 +ds 3 +es 4 +fs 5

[0278] Where θ(s) represents the tangent direction at a point on the helix, s represents the arc length along the helix, and a, b, c, d, e, and f are the coefficients of a fifth-order polynomial; the coordinates on the curve can be represented as:

[0279]

[0280] Where, x i y i z i Let P be the i-th point on the curve. i The coordinate values;

[0281] The starting direction, starting curvature, derivative of starting curvature, ending direction, ending curvature, derivative of ending curvature, and the curve length between the two points are used as constraints for each segment of the fifth-degree polynomial spiral:

[0282]

[0283] Where, θ i Starting from the direction, Starting with curvature, θ is the derivative of the curvature at the starting point (rate of change of curvature). i+1 The destination direction, For the endpoint curvature, The endpoint curvature derivative (rate of change of curvature), Δs i θ is the length of the curve between two points; i (0) represents the direction at the starting point of the curve; The curvature at the starting point of the curve; θ is the derivative of curvature at the starting point of the curve; i (Δs i ) represents the distance Δs from the starting point of the curve. i Location and direction; Δs is the distance from the starting point of the curve. i Curvature at location; Δs is the distance from the starting point of the curve. i Derivative of curvature at position;

[0284] The continuity of the position, direction, curvature, and derivative of curvature at the junction of the two curve segments is used as a constraint condition:

[0285]

[0286] Where, x i+1 y i+1 zi+1 Let P be the (i+1)th point on the curve. i+1 The coordinate values;

[0287] For point P on the curve i (x i ,y i ,z i Apply position translation constraints:

[0288]

[0289] Where i = 1, ..., n; r represents the position coordinates of the original reference point. i This is the distance that the original reference point can be translated.

[0290] The innovative aspect of this invention in optimizing the robotic arm trajectory lies in dividing the joint motion of the robotic arm into three stages, employing fifth-order and seventh-order polynomials respectively. This addresses the issue of discontinuous pulsation characteristics while also considering the computational complexity. Furthermore, the transformation process from the first to the second stage is optimized using polynomial spiral parametric equations. This solves the problem of discontinuous pulsation characteristics between the fifth-order and seventh-order polynomials in practical work, avoids rigid impacts at the joints of the robotic arm, reduces vibrations and impacts generated during robot operation, and ensures smoother robotic arm operation.

[0291] In summary, this embodiment provides an automated road grouting repair robot control system. This system can automatically plan paths, locate work areas, identify crack features, and perform precise grouting operations. By introducing a random fast search tree for heuristic search, the adaptability of the automated road grouting repair robot in the path planning process is effectively improved. Deep learning image algorithms are used to identify crack features, scanning and digitizing the target area to be repaired, avoiding some limitations of traditional image recognition methods. Multiple iterative calculations and digitized image processing make the crack feature recognition model more accurate and reliable. By mounting the grouting instrument on a powered robotic arm and combining it with an image acquisition device, a correspondence between grouting volume, crack size, and repair target is established, enabling the robot to perform precise grouting work and complete the repair operation based on instructions provided by the image recognition system.

[0292] Second Embodiment

[0293] This embodiment provides an electronic device, which includes a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor.

[0294] The electronic device can vary considerably depending on its configuration or performance, and may include one or more processors (central processing units, CPUs) and one or more memories, wherein the memory stores at least one instruction, which is loaded and executed by the processor. This instruction relates to the various models and algorithms involved in the automatic road grouting repair robot control system of the first embodiment described above.

[0295] Third Embodiment

[0296] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor. The computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0297] The instructions stored in the computer-readable storage medium can be loaded and executed by the processor in the terminal, which are various models and algorithms involved in the automatic road grouting repair robot control system of the first embodiment described above.

[0298] Furthermore, it should be noted that the present invention can be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0299] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0300] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0301] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0302] Finally, it should be noted that the above description represents a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A control system for an automatic road grouting repair robot, characterized in that, include: Path planning module, work area positioning module, crack identification module, and grouting robotic arm control module; The path planning module is used to plan the path for the automatic road grouting repair robot based on the artificial potential field method according to the road repair operation scenario and requirements. It also introduces a random fast search tree algorithm to adjust the gravitational and repulsive constants of the artificial potential field method, completing a heuristic search for path planning, including: When performing path planning for an automated road grouting repair robot based on the artificial potential field method, the gravitational potential energy function is used. The calculation formula is: ； in, It is the gravitational constant; This represents the robot's current position. The target point for the robot; Let be a vector pointing from the robot's position to the target point, and its magnitude be the Euclidean distance between the robot's position and the target point; Gravitational function The calculation formula is: ； Repulsive potential energy function The calculation formula is: = ； in, It is the repulsive constant; The radius of the repulsive potential field is denoted by , which represents the maximum distance at which the obstacle exerts an effect on the robot. The location of the obstacle; Let be a vector pointing from the obstacle's position to the robot's position, and let its magnitude be the Euclidean distance between the robot's position and the obstacle. repulsion function The calculation formula is: ； in, This represents the unit vector pointing from the obstacle to the robot; This represents the unit vector pointing from the robot to the target point; The total potential energy of the robot used Total synergy The calculation formula is as follows: ； ； Introducing the random fast search tree algorithm to calculate the gravitational constant in the artificial potential field method. and repulsive constant Adjustments were made, and a heuristic search was performed; the combined force at this point... for: ； in, The number of obstacles; For the first i The repulsive force described by the obstacle; by changing the gravitational constant. and repulsive constant Adjust the weights of the attraction and repulsion values, and the repulsion constant. The larger the value, the farther the generated path will be from the obstacle. In this case, the main function of the artificial potential field method is to help the robot avoid the obstacle; gravitational constant The larger the potential field, the faster the generated path allows the robot to approach the target point. In this case, the main role of the artificial potential field method is to guide the robot to search towards the target point, thereby completing the heuristic search for path planning. The algorithm implementation process is as follows: Step 1: Initialize the algorithm and set the starting point. Target point and step size Randomly sample state points from the environment ; Step 2: Begin the random tree growth process. First, divide the working region into segments. Then, introduce the gravitational constant of the artificial potential field method into the segmented regions. and repulsive constant Regarding the gravitational constant and repulsive constant Adjustments were made to make the random tree growth path farther away from obstacles and closer to the target point faster. and The connection direction is used as the growth direction of the tree, and a step size is set in this direction. As the step size of a tree's growth cycle, a new node is generated at the end of the growth. , judge from arrive Whether the obstacle was passed through; if not, then... A node is added to the tree. If it is traversed, the current new node is abandoned and a new direction is chosen for growth. Step 3: Repeat the tree growth process until the distance from the newly generated node to the target point is less than one step size. If the tree growth is terminated, the current new node is directly connected to the target node, and the algorithm ends. The work area positioning module is used to establish a work area positioning model based on the FastSLAM algorithm. Based on the path planning results of the path planning module, spatial recognition is performed using the real-time video stream of the workspace collected by the built-in camera of the automatic road grouting repair robot. The motion trajectory of the automatic road grouting repair robot is estimated using the posterior probability distribution to complete the automatic movement of the robot to the work target, including: The motion model and observation model of the FastSLAM algorithm are represented as follows: ； ； in, for Robot pose at all times; for Robot pose at all times; for The observed position of the robot at any given time; for The robot's motion values ​​at any given time; Marking for road signs; for Information about road signs; and It is a nonlinear function; For the motion model, use Gaussian white noise; The observation model uses Gaussian white noise; The robot trajectory model is represented by the posterior probability distribution estimate; The robot's motion trajectory is determined based on the FastSLAM algorithm. and environmental modeling Using posterior probability, it can be represented as: ； in, Marking for road signs; These are the observed values ​​of the robot's position; The motion value of the robot's position; The number of obstacles; The process of establishing environmental information involves the following steps: Step 1: Obtain a set of particles describing the robot's prior pose by sampling from the robot's kinematic model. The robot's kinematic model is represented in Cartesian coordinates. Transform the Frenet coordinate system to the Cartesian coordinate system. In the Frenet coordinate system, the vehicle state is represented as: ； in, This represents longitudinal displacement; Longitudinal velocity; It is longitudinal acceleration; This is a lateral displacement; For lateral velocity; It is lateral acceleration; First derivative with respect to arc length; Second derivative with respect to arc length; The lateral displacement of the mobile robot is represented in Cartesian coordinates as follows: ； in, To repair the robot's trajectory curve; The position vector along the reference line of the robot's current position; This is the orthogonal unit normal vector along the reference line direction of the robot's current position; The lateral movement speed is: ； in, This is the orthogonal unit normal vector along the reference line direction of the robot's current position; This is the orthogonal unit tangent vector along the reference line direction of the robot's current position; The velocity of the moving robot in Cartesian coordinates; The angle between the robot's current position and the direction of the reference line in the Cartesian coordinate system; The robot's velocity in the Frenet coordinate system is expressed as: ； in, The curvature of the mobile robot's current position; To Find the time derivative; from this, we get: ； ； The robot's velocity and acceleration are: ； ； in, To Find the time derivative; Step 2: Adjust the weights of the particles in the particle set describing the robot's prior pose. After calculating the particle weights, calculate the posterior probability distribution of the robot's state based on the weights and their corresponding states. The particle weights are represented as: ； in, for k The motion trajectory of the robot in state -1; for k -1 observations of the robot's position; for k -1 is the motion value of the robot's position in state -1; for k -1 status marker; Step 3: Resample the particle set using a hierarchical resampling algorithm, which divides the sampling region into continuous, non-overlapping intervals and performs independent, identically distributed sampling on each sub-interval to obtain robot position deviation data for correction control. The crack identification module is used to identify crack information of the target object based on deep learning image algorithms, including: Using a spatial feature-based image recognition algorithm, spatial feature extraction is based on the ResNeXt network architecture. An improved ResNeXt network is used to perform a secondary partitioning of the feature map groups, resulting in a total of: ； in, This represents the cardinality hyperparameter for the first partitioning. The cardinality hyperparameter represents the number of partitions within a feature map group. Summing element-wise across multiple components allows us to construct a combined representation from several radix arrays: ； in, H, W, and C represent the shapes of the block output feature maps. The first division k One component; For the first j A base array; across spatial dimensions Global average pooling is used to aggregate global contextual information with embedded channel statistics. The first base array The individual components are calculated as follows: ； in, For the first A set of component base arrays; Weighted aggregation is performed based on channel attention, where each feature map channel is generated by a weighted combination. The weighted fusion representation of the c-th channel is as follows: ； in, The base array is divided into two sets after the feature map group is defined. Indicates the weight allocation: ； in, express Determine the weight of each component in the c-th channel; The improved ResNeXt network refers to replacing the standard 3×3 convolution in the ResNeXt network with involution. In the involution operation of the improved ResNeXt network, the shape of the involution kernel is generated by the shape of the feature map of the input data and is spatially aligned with the size of the feature map. The involution kernel generation process is represented as follows: ； ； in, coordinates The set of pixels in the neighborhood; Coordinates in the feature map A single pixel; For processing nonlinear activation functions; and The transformation matrix is ​​linear, and a 3×3 standard convolution is used. r Take 3; The Mixup data augmentation strategy is used to generate random images from the training data and perform weighted combinations. The grouting robotic arm control module is used to control the movement of the grouting robotic arm of the automatic road grouting repair robot and complete the grouting operation on the target based on the recognition result of the crack recognition module.

2. The automatic road grouting repair robot control system as described in claim 1, characterized in that, The automatic road grouting repair robot control system also includes an abnormal situation handling module; The abnormal situation handling module is used to issue an alarm and switch to manual control when an abnormality is being repaired.

3. The automatic road grouting repair robot control system as described in claim 1, characterized in that, The automatic road grouting repair robot control system also includes a feedback module; The feedback module is used to determine the repair effect level after the grouting operation is completed, and to provide feedback control to the automatic road grouting repair robot based on the determination result of the repair effect level.

4. The automatic road grouting repair robot control system as described in claim 1, characterized in that, The automatic road grouting repair robot control system also includes a post-processing module; The post-processing module is used to control the robot to automatically clean up the crushed stone dust after the repair work is completed, and to store the repair process data into the repair work database to record the repair process and continue to optimize the model.

5. The automatic road grouting repair robot control system as described in claim 1, characterized in that, The method for identifying crack information in the target area based on deep learning image algorithms includes: Crack images are collected from the work scene and digitized. Based on a deep neural network framework, the digitized images are divided into training and testing sets. To avoid overfitting, the number of images collected per unit time is increased and multiple iterative calculations are performed. The trained network model is loaded into the camera processing system carried by the automatic road grouting repair robot, and the crack images it collects are processed to identify crack information.

6. The automatic road grouting repair robot control system as described in claim 5, characterized in that, The method for digitizing crack images is as follows: All crack images are normalized to obtain an image database with consistent size and pixel count. All image information is then gridded to obtain 256 images. The frame consists of 256 pixels, that is, 256 pixels horizontally and 256 pixels vertically, with each pixel serving as the data basis for image training.

7. The automatic road grouting repair robot control system as described in claim 5, characterized in that, The deep neural network framework is a convolutional neural network.

8. The automatic road grouting repair robot control system as described in claim 1, characterized in that, The grouting robotic arm is a six-degree-of-freedom robotic arm; The movement of the grouting robotic arm of the automatic road grouting repair robot includes: Using a six-degree-of-freedom robotic arm model, during the robotic arm's operation, there are pre-set parameters... m Given a sequence of path points at the end effector of a six-DOF robotic arm, inverse kinematics operations are performed to obtain the angular displacement sequence of the six rotary joints. , among which, the i Joint angular displacement vectors at each path point , , j =1,2,…,6, For the first j On the degree of freedom, the ... i The joint angular displacement vectors at each path point; the six dimensions of this vector represent the six rotational joints of the robot. The ultimate goal is to obtain the motion trajectories of each joint at the start and end path points where both joint angular velocity and joint angular acceleration are zero, while ensuring the continuity of the joint angular acceleration curves at each path point. The motion trajectory parameters of each joint are transformed into variables that depend only on time t. The angular displacements of each joint of the six-DOF robotic arm are expressed as follows: ； in, For the first j Joint angular displacement in one degree of freedom; The angular velocities of each joint of a six-degree-of-freedom robotic arm are expressed as follows: ； in, For the first j Joint angular velocities in each degree of freedom; The angular accelerations of each joint of a six-degree-of-freedom robotic arm are expressed as follows: ； in, For the first j Joint angular acceleration in each degree of freedom; The joint angular pulsation characteristics of a six-DOF robotic arm are expressed as follows: ； in, Defined as the pulsation function of the joints of a robotic arm, mathematically defined as the joint angular jerk, it is used to measure the smoothness of the robotic arm's trajectory. The smaller the value, the smoother the joint trajectory of the robotic arm and the more stable the robotic arm. Indicates the first j Joint angle pulsation characteristics in one degree of freedom; The pulsation characteristics of the robotic arm are defined as follows: ； In the formula, S The average pulsation of the robotic arm joints. This refers to the number of joints. T Total operating time of the robotic arm; t For time; In the robotic arm trajectory planning, the robotic arm movement process is divided into three segments. The first segment is the transition of the robotic arm from a folded state to an unfolded state, and the trajectory for this segment is represented by a 5th-degree polynomial. The second segment is the movement of the robotic arm from the unfolded state to the starting point of the task target, and the trajectory for this segment is represented by a 7th-degree polynomial. The third segment is the movement of the robotic arm from the starting point of the task target to the ending point of the task target, and the trajectory for this segment is also represented by a 7th-degree polynomial. The specific form is as follows: The general functional relationship between the fifth-degree polynomial and time in the first stage of the robotic arm's motion is as follows: ； Set the joint initiation angle position, joint termination angle position, joint initiation angular velocity, joint termination angular velocity, joint initiation angular acceleration, and joint termination angular acceleration as constraints: ； in, At the starting time, The termination time; The angular velocity of the robotic arm joints can be expressed as follows based on the polynomial-time function relationship: ； The joint angular acceleration of the robotic arm is expressed as follows: ； Therefore, we can conclude that: ； The general functional relationship between the seventh-degree polynomials of the second and third stages of the robotic arm's motion and time is as follows: ； Set the following constraints: joint initiation angle position, joint termination angle position, joint initiation angular velocity, joint termination angular velocity, joint initiation angular acceleration, joint termination angular acceleration, joint initiation angular jerk, and joint termination angular jerk. ； The angular velocity of the robotic arm joints can be expressed as follows based on the polynomial-time function relationship: ； The joint angular acceleration of the robotic arm is expressed as follows: ； The function for the variation of joint angle pulsation characteristics of the robotic arm is: ； A smoothing method using parametric equations to fit the curve addresses the discontinuity at the transition point between the first-stage fifth-order polynomial and the second-stage seventh-order polynomial. The path is defined as a polynomial spiral parametric curve, and the tangent direction at points on the spiral is represented by a fifth-order polynomial function of arc length. The curve is divided into n-1 segments by n points. The parametric formula is as follows: ； in, Indicates the direction of the tangent at a point on the spiral. Indicates the arc length along the spiral. a , b , c , d , e , f The coefficients are those of a fifth-degree polynomial; the coordinates on the curve are represented as follows: ； in, , , For the first curve i Points The coordinate values; The starting direction, starting curvature, derivative of starting curvature, ending direction, ending curvature, derivative of ending curvature, and the curve length between the two points are used as constraints for each segment of the fifth-degree polynomial spiral: ； in, Starting from the direction, Starting with curvature, The derivative of the curvature at the starting point, The destination direction, For the endpoint curvature, The derivative of the curvature at the endpoint, The length of the curve between the two points; The direction at the starting point of the curve; The curvature at the starting point of the curve; The derivative of curvature at the starting point of the curve; Distance from the starting point of the curve Location and direction; Distance from the starting point of the curve Curvature at location; Distance from the starting point of the curve Derivative of curvature at position; The continuity of the position, direction, curvature, and derivative of curvature at the junction of the two curve segments is used as a constraint condition: ； in, , , For the first curve i+ 1 point The coordinate values; For points on the curve Apply position translation constraints: ； in, ; The original reference point's position coordinates, This is the distance that the original reference point can be translated.

Images