Automatic control method and controller of mobile robot

Abstract

The present invention belongs to the field of robot control, and in particular relates to an automatic control method and controller for a mobile robot. The method integrates a SLAM optimization model with the robot's visual coordinate space to build and update a visual real-time 3D work map in real time, accurately mark dynamic and static obstacles, and secondly, dynamically generate an optimal feasible path by combining task status and work environment space monitoring data, and train a reinforced control model to adjust the robot's operation strategy in real time. Thirdly, through continuous monitoring and data feedback, the work map, path planning and operation status are continuously corrected to ensure that the robot completes the task efficiently and safely in a complex dynamic environment. The controller of the present invention realizes a high degree of integration of intelligent perception, dynamic decision-making and precise execution, and significantly improves the autonomous navigation and control capabilities of the mobile robot.

Description

Technical Field

[0001] The invention belongs to the field of robot control, and in particular relates to an automatic control method and a controller of a mobile robot. Background Art

[0002] Traditional SLAM technology is easily affected by factors such as occlusion and reflection in complex environments, resulting in reduced positioning accuracy and incomplete mapping; the classic path planning algorithm has poor real-time performance when dealing with dynamic obstacles and is difficult to respond quickly to environmental changes; and traditional control strategies (such as PID control) have poor adaptability in complex dynamic environments and lack self-learning and adjustment capabilities. These deficiencies make it difficult for existing mobile robots to achieve high-precision positioning, intelligent path planning, and efficient control in complex and changing environments; and the existing technology mostly uses static coordinates when constructing a three-dimensional map for robot operation, which requires SLAM to predict both position and posture when predicting the robot's posture, increasing the difficulty of prediction.

[0003] For example, the patent with authorization announcement number CN112571409B discloses a robot control method, robot and medium based on visual SLAM, which includes the following steps: acquiring shooting data of a camera unit, and intercepting a picture frame containing an artificial landmark in the shooting data; preprocessing the picture frame to obtain a landmark area corresponding to the artificial landmark from the picture frame; acquiring contour features of the landmark graphics in the landmark area; acquiring control parameters associated with a preset landmark that matches the contour features, and controlling the robot to move according to the control parameters.

[0004] For example, the patent with authorization announcement number CN112684728B discloses a robot palletizing control method based on laser SLAM, including a master control processor, an industrial computer, a logistics line, an industrial robot and a truss robot; the industrial computer, the logistics line and the industrial robot are all communicatively connected to the master control processor, the industrial robot is communicatively connected to the truss robot, and the truss robot is arranged above the logistics line and the industrial robot; a SLAM laser radar and a control unit are arranged on the truss robot, and the SLAM laser radar is electrically connected to the control unit; a sensor and a visual camera are arranged on the logistics line, and the sensor is electrically connected to the visual camera.

[0005] The above-mentioned prior art has the following problems: the existing robot control algorithm cannot change the coordinates according to the real-time environment map in a complex environment or an environment without overall map marking planning, and most of the prior art uses a static coordinate system, which makes the coordinate transformation complicated. For this reason, the present invention provides an automatic control method and controller for a mobile robot. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes an automatic control method and controller for a mobile robot. The method integrates the SLAM optimization model and the robot's visual coordinate space to build and update a visual real-time 3D work map in real time, accurately mark dynamic and static obstacles, and secondly, combines the task status and the working environment space monitoring data to dynamically generate the optimal feasible path, and trains and strengthens the control model to adjust the robot's operation strategy in real time. Thirdly, through continuous monitoring and data feedback, the work map, path planning and operation status are continuously corrected to ensure that the robot completes the task efficiently and safely in a complex dynamic environment. The controller of the present invention realizes a high degree of integration of intelligent perception, dynamic decision-making and precise execution, which significantly improves the autonomous navigation and control capabilities of the mobile robot.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] An automatic control method for a mobile robot, comprising:

[0009] S1. Obtain the monitoring data of the robot's current position and visual working environment space, and use the robot's current position as the origin to construct the robot's visual coordinate space in the visual working environment space, and build the constructed robot's visual coordinate space into the robot control;

[0010] S2. Construct a SLAM optimization model, integrate the constructed SLAM optimization model with the visual coordinate space, input the acquired visual working environment space monitoring data into the SLAM optimization model, construct and obtain the robot's visual dynamic working map and the robot's current posture in the visual coordinate space, and mark the dynamic and static obstacle positions in the visual dynamic working map in real time to obtain a visual real-time 3D working map;

[0011] S3, obtain task status data, and generate a dynamic feasible path for the robot in real time on a visual real-time 3D work map based on the visual work environment space monitoring data through a path algorithm;

[0012] S4. According to the visual working environment space monitoring data, the configured enhanced control model is trained, the trained enhanced control model is obtained, and the trained enhanced control model is integrated into the dynamic feasible path to control the running state of the robot on the dynamic feasible path in real time;

[0013] S5. Obtain the visual working environment space monitoring data and operation status data of the robot at every moment on the dynamic feasible path in real time, and update and correct the constructed visual real-time 3D work map, dynamic feasible path and operation status in real time through the process from S1 to S4 until the robot completes the task.

[0014] Specifically, the steps of constructing the robot's visual coordinate space include:

[0015] S101, obtaining the monitoring radius parameter r of the robot sensor, and using the current position of the robot as the coordinate origin to construct a three-dimensional monitoring coordinate;

[0016] S102, obtaining a visible area with a monitoring radius of r on the xy plane of the three-dimensional monitoring coordinates according to the acquired monitoring radius parameter;

[0017] S103, embed the acquired visible area into the three-dimensional monitoring coordinates to obtain the initial visible coordinate space V, specifically:

[0018] ,in is the spatial coordinate of any point in the initial visual coordinate space, and In turn, they represent the minimum and maximum monitoring heights of the sensor in the vertical direction;

[0019] S104, based on the constructed initial visual coordinate space and the monitoring distance data of the robot during actual movement, a secondary visual coordinate space is constructed in the three-dimensional monitoring coordinates through the ellipse equation , specifically:

[0020] ,

[0021] in, It represents the shortest distance of the robot in each monitoring direction in the visual coordinate space at time t, It represents the longest distance of the robot in each monitoring direction in the visual coordinate space at time t;

[0022] S105 . When the monitoring distances of the robot in each monitoring direction are not equal, the corresponding initial visual coordinate space is converted into a secondary visual coordinate space.

[0023] Specifically, the steps of constructing a visual real-time 3D work map include:

[0024] S201, preprocessing the images, videos and IMU measurement data in the visual working environment space monitoring data through the built-in preprocessing algorithm in the sensor, and obtaining the preprocessed monitoring environment image frame sequence and IMU measurement data;

[0025] S202, inputting the acquired pre-processed monitoring environment image frame sequence into the ORB sub-model configured in the SLAM optimization model to extract key feature points, and obtaining a key feature point space;

[0026] S203, inputting the acquired pre-processed monitoring environment image frame sequence into the pre-trained YOLOv8 sub-model, performing dynamic detection on dynamic targets, potential dynamic targets and static targets in the input image, and outputting the anchor frame coordinates of dynamic detection of dynamic targets, potential dynamic targets and static targets;

[0027] S204, based on the anchor frame coordinates and key feature point space of dynamic, potential dynamic, and static target dynamic detection, obtain the key feature point set in the dynamic target anchor frame , the key feature point set in the potential dynamic anchor box And static key feature point set ,in, Represents the static key feature point set corresponding to the jth image in the monitoring environment image frame sequence;

[0028] S205, and After cascading, it is input into the sparse LK optical flow algorithm to obtain the light loss corresponding to each key feature point , and at the same time Input to the LK optical flow algorithm to obtain The average value of light loss corresponding to all key feature points in ,in express and The amount of light loss corresponding to the i-th key feature point in .

[0029] Specifically, the steps of constructing a visual real-time 3D work map also include:

[0030] S206, based on the obtained light loss amount of the potential dynamic key feature point and the average light loss amount corresponding to the static key feature point, a judgment is made through a judgment formula. If the corresponding relationship of the formula is satisfied, the corresponding key feature point is a dynamic key feature point, otherwise the key feature point is a static key feature point. The judgment formula is specifically: , where l represents the streamer threshold;

[0031] S207, constructing a static key feature point space set using the acquired static key feature points and the static key feature point set, inputting the acquired static key feature point space set and the IMU measurement data into the local mapping thread in the SLAM sub-model for key frame judgment, and obtaining the key frame in the current monitoring environment image frame sequence;

[0032] S208, according to the key frames and IMU measurement data in the current monitoring environment image frame sequence, through the loop detection thread in the SLAM sub-model, obtain the visual dynamic working map corresponding to the visual working environment space monitoring data of the robot at the current moment and the current posture of the robot;

[0033] S209: Based on the obtained visual dynamic work map, the dynamic key feature points and the key feature point set in the dynamic target obtained by S206 are marked to the corresponding position points of the visual dynamic work map through a pixel matching algorithm to obtain a visual real-time 3D work map.

[0034] Specifically, the steps of constructing a visual real-time 3D work map also include:

[0035] S210, simultaneously obtaining visual working environment space monitoring data of N robots through the controller multi-interface, and obtaining the visual real-time 3D working map and current posture of the corresponding robot through S201-S209;

[0036] S211, according to the obtained visual real-time 3D work maps corresponding to the N robots, through the area overlap detection algorithm, determine and obtain the overlapping area between the visual real-time 3D work map corresponding to the nth robot and the visual real-time 3D work maps corresponding to the remaining N-1 robots;

[0037] S212, obtaining the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the kth robot to the nth robot by changing the corresponding overlapping area of ​​the nth robot and the kth robot;

[0038] S213, obtaining a position error in the overlapping area based on the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the kth robot obtained in S212 to the nth robot and the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the nth robot;

[0039] S214. Based on the process of S212-S213, obtain a position error sequence in the overlapping area between the nth robot and the remaining N-1 robots.

[0040] Specifically, the steps of constructing a visual real-time 3D work map also include:

[0041] S215, feeding back the acquired position error sequence in the overlapping area to the loop detection thread in the SLAM sub-model, performing secondary optimization on the visual real-time 3D working map corresponding to the position error in the overlapping area not equal to 0 and the key frames in the acquired current monitoring environment image frame sequence, to obtain an optimized visual real-time 3D working map;

[0042] S216. According to S211, it is determined that there is an overlapping visual real-time 3D work map. When the visual real-time 3D work maps of the nth robot and the kth robot overlap, the kth robot is regarded as a dynamic obstacle in the visual real-time 3D work map corresponding to the nth robot at the current moment, and the relative position distance between the kth robot and the nth robot at the current moment is obtained;

[0043] S217. Set a relative distance threshold. When the relative position distance between the kth robot and the nth robot at the current moment is less than the relative distance threshold, the obtained relative position distance is fed back to the enhanced control model to make real-time adjustments to the running direction of the kth robot or the nth robot at the current moment.

[0044] Specifically, the steps of generating a dynamic feasible path of the robot in S3 include:

[0045] S301, according to the obtained visual real-time 3D work map at the current moment, obtain the relative position distance between the dynamic and static key feature points marked in the visual real-time 3D work map and the robot at the current moment, and at the same time input the movement posture information of the marked dynamic key feature points into the pre-trained trajectory prediction model to obtain the movement trajectory information of each dynamic key feature point and the probability of collision between the robot and each dynamic key feature point;

[0046] S302, inputting the relative position distance obtained in S301, the current position and posture of the robot, the moving trajectory information of each dynamic key feature point, and the probability of the robot colliding with each dynamic key feature point into a trajectory generation algorithm to generate a feasible path corresponding to the current moment of the robot;

[0047] S303. Based on the movement trajectory information of each dynamic key feature point, the real-time relative distance from each dynamic key feature point to the robot is obtained. When the real-time relative distance is less than the relative distance threshold, the feasible path is adjusted to obtain a dynamic feasible path. If it is greater than or equal to the relative distance threshold, the feasible path corresponding to the current moment remains unchanged.

[0048] Specifically, the steps of building the enhanced control model in S4 include:

[0049] S401: Construct the current input state based on the current visible real-time 3D work map, relative distance, robot posture, and dynamic feasible path. :

[0050] ,

[0051] in, Represents the three-dimensional spatial coordinates of the task target point corresponding to the task status data, , and represents the three-dimensional coordinates of the nth robot on the visual real-time 3D work map, Indicates the posture angle of the nth robot at the current time t, Indicates the real-time driving speed of the nth robot corresponding to the visible real-time 3D work map at the current time t; represents the relative distance between the nth robot and the kth dynamic obstacle at the current time t, represents the probability that the nth robot collides with the kth dynamic obstacle when moving along the current feasible path at the current time t. represents the relative distance between the nth robot and the qth static obstacle at the current time t; represents the dynamic feasible path corresponding to the current time t of the nth robot; Represents the visual real-time 3D work map corresponding to the current time t of the nth robot;

[0052] S402, setting robot control trigger information according to the current input state, including:

[0053] Set the collision probability threshold ,when Greater than , triggering the first control information, that is, adjusting the posture angle of the n-th robot, and adjusting the dynamic feasible path corresponding to the n-th robot through the adjusted posture angle;

[0054] when or When it is greater than the relative distance threshold, the second control information is triggered, that is, the posture angle of the nth robot is adjusted, and the dynamic feasible path corresponding to the nth robot is adjusted through the adjusted posture angle;

[0055] Set the ambient speed threshold ,when Greater than The third control information is triggered when

[0056] S403: Based on the set control trigger information, construct the execution action corresponding to the current moment ,in, Indicates the action that triggers the first control information to be executed. Indicates the action that triggers the execution of the corresponding second control information. Indicates the action that triggers the execution of the corresponding third control information.

[0057] An automatic controller for a mobile robot, comprising: a coordinate construction module, a map construction module, a path generation module and an enhanced control module;

[0058] A coordinate construction module is used to construct the robot's visual coordinate space based on the acquired robot's current position and visual working environment space monitoring data;

[0059] The map construction module is used to obtain the visual real-time 3D working map and current posture of the robot in the visual coordinate space based on the acquired visual working environment space monitoring data and the constructed SLAM optimization model;

[0060] The path generation module is used to generate a dynamic feasible path for the robot through a path generation algorithm according to the visual real-time 3D work map and current posture of the robot;

[0061] The enhanced control module is used to control the robot's operating state on a dynamic feasible path in real time according to the visual working environment space monitoring data through the trained enhanced control model until the task is completed.

[0062] A computer-readable storage medium stores computer instructions, which execute an automatic control method for a mobile robot when the computer instructions are executed.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] In view of the deficiencies in the prior art, the present invention constructs a robot visual coordinate space and integrates it with a SLAM optimization model, thereby realizing the ability to change coordinates according to a real-time environmental map in a complex environment or an environment without overall map marking planning. Secondly, through the integration of the SLAM optimization model and the visual coordinate space, a dynamic working map is constructed in real time and dynamic and static obstacles are marked in the map to ensure that the robot can perceive environmental changes in a timely manner. In addition, a dynamic feasible path is generated in real time on the dynamic map through a path algorithm, and a reinforcement learning model is used for real-time control and correction to ensure the efficient operation of the robot in a complex environment. In summary, this process not only improves the robot's autonomous navigation capability, but also enhances its adaptability and robustness in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flow chart of an automatic control method for a mobile robot according to Embodiment 1 of the present invention;

[0066] Figure 2 This is a structural flow chart of a visual real-time 3D work map construction process in Example 1 of the present invention;

[0067] Figure 3 This is a module diagram of an automatic controller of a mobile robot according to Embodiment 2 of the present invention;

[0068] Figure 4 This is a structural diagram of the controller in Example 1 of the present invention. DETAILED DESCRIPTION

[0069] Example 1

[0070] See also Figure 1 , an embodiment of the present invention provides: an automatic control method of a mobile robot, comprising the following steps:

[0071] S1. Connect the robot to the controller interface and initialize the controller, obtain the robot's current position and visual working environment space monitoring data, and use the robot's current position as the origin to construct the robot's visual coordinate space in the visual working environment space, and build the constructed robot's visual coordinate space into the robot control; further, the visual working environment space monitoring data in this embodiment is the working environment space data obtained within the maximum monitoring radius of the robot's configured sensor; for example, the working environment corresponding to complex areas such as mountains and caves;

[0072] Furthermore, the controller in this embodiment adopts an ARM CPU, with a variety of computing power options such as 3588, 3568, and 3576. It has a low-power and high-computing power CPU, and a reserved communication interface can be used to install the user's self-developed MCU control board, which is convenient for customizing the mobile robot controller. It can cover various high, medium and low computing power requirements and control multiple robots simultaneously.

[0073] This process further improves the accuracy and flexibility of environmental perception by dynamically adjusting the visible area within the monitoring radius and introducing elliptical equations to construct a quadratic visual coordinate space. This not only improves the robot's positioning accuracy in complex environments, but also provides more comprehensive and real-time environmental information support for its path planning and control decisions.

[0074] Furthermore, the steps of constructing the robot visual coordinate space in this embodiment include:

[0075] S101, obtaining the monitoring radius parameter r of the robot sensor, and using the current position of the robot as the coordinate origin to construct a three-dimensional monitoring coordinate;

[0076] S102, obtaining a visible area with a monitoring radius of r on the xy plane of the three-dimensional monitoring coordinates according to the acquired monitoring radius parameter;

[0077] S103, embed the acquired visible area into the three-dimensional monitoring coordinates to obtain the initial visible coordinate space V, specifically:

[0078] ,in is the spatial coordinate of any point in the initial visual coordinate space, and In turn, they represent the minimum and maximum monitoring heights of the sensor in the vertical direction;

[0079] S104, based on the constructed initial visual coordinate space and the monitoring distance data of the robot during actual movement, a secondary visual coordinate space is constructed in the three-dimensional monitoring coordinates through the ellipse equation , specifically:

[0080] ,

[0081] in, It represents the shortest distance of the robot in each monitoring direction in the visual coordinate space at time t, It represents the longest distance of the robot in each monitoring direction in the visual coordinate space at time t;

[0082] S105 . When the monitoring distances of the robot in each monitoring direction are not equal, the corresponding initial visual coordinate space is converted into a secondary visual coordinate space.

[0083] S2. Construct a SLAM optimization model, integrate the constructed SLAM optimization model with the visual coordinate space, input the acquired visual working environment space monitoring data into the SLAM optimization model, construct and obtain the robot's visual dynamic working map and the robot's current posture in the visual coordinate space, and mark the dynamic and static obstacle positions in the visual dynamic working map in real time to obtain a visual real-time 3D working map;

[0084] For further information, see Figure 2 The steps of constructing a visual real-time 3D work map in this embodiment include:

[0085] S201, preprocessing the images, videos and IMU measurement data in the visual working environment space monitoring data through the built-in preprocessing algorithm in the sensor, and obtaining the preprocessed monitoring environment image frame sequence and IMU measurement data;

[0086] S202, inputting the acquired pre-processed monitoring environment image frame sequence into the ORB sub-model configured in the SLAM optimization model to extract key feature points, and obtaining a key feature point space;

[0087] S203, inputting the acquired pre-processed monitoring environment image frame sequence into the pre-trained YOLOv8 sub-model, performing dynamic detection on dynamic targets, potential dynamic targets and static targets in the input image, and outputting the anchor frame coordinates of dynamic detection of dynamic targets, potential dynamic targets and static targets;

[0088] Furthermore, in this embodiment, dynamic targets include pedestrians, moving vehicles or animals, etc., potential dynamics include parked vehicles and temporarily placed items; static targets include buildings or fixed obstacles, etc.

[0089] Furthermore, the YOLOv8 sub-model constructed in this embodiment includes a backbone layer, a bottleneck layer and an output layer; wherein the backbone layer adds a ParNetAttention after each convolution sub-layer on the basis of the original network architecture, so that the model focuses on a specific area of ​​interest in the image, while suppressing the redundant and interfering information introduced by the improved module; the bottleneck layer adds a CARAFE sub-layer after each upsampling operation on the basis of the original network, and the CARAFE sub-layer generates a feature map of the same size as the high-resolution feature map according to the position content prediction of each detection target, and uses the generated feature map of the same size as the high-resolution feature map for feature reorganization; specifically, CARAFE divides the low-resolution feature map into several small blocks, and then interacts each small block with the high-resolution feature map to obtain the context information required for reconstruction; secondly, after obtaining sufficient context information, the CARAFE operator performs a convolution operation on the interacted feature map to obtain the final up-sampled feature reconstruction result.

[0090] S204, based on the anchor frame coordinates and key feature point space of dynamic, potential dynamic, and static target dynamic detection, obtain the key feature point set in the dynamic target anchor frame , the key feature point set in the potential dynamic anchor box And static key feature point set ,in, Represents the static key feature point set corresponding to the jth image in the monitoring environment image frame sequence;

[0091] S205, and After cascading, it is input into the sparse LK optical flow algorithm to obtain the light loss corresponding to each key feature point , and at the same time Input to the LK optical flow algorithm to obtain The average value of light loss corresponding to all key feature points in ,in express and The amount of light loss corresponding to the i-th key feature point in ;

[0092] S206, based on the obtained light loss amount of the potential dynamic key feature point and the average light loss amount corresponding to the static key feature point, a judgment is made through a judgment formula. If the corresponding relationship of the formula is satisfied, the corresponding key feature point is a dynamic key feature point, otherwise the key feature point is a static key feature point. The judgment formula is specifically: , where l represents the streamer threshold;

[0093] S207, constructing a static key feature point space set using the acquired static key feature points and the static key feature point set, inputting the acquired static key feature point space set and the IMU measurement data into the local mapping thread in the SLAM sub-model for key frame judgment, and obtaining the key frame in the current monitoring environment image frame sequence;

[0094] Furthermore, the local mapping thread in this embodiment is an original local mapping thread in the SLAM algorithm.

[0095] S208, according to the key frames and IMU measurement data in the current monitoring environment image frame sequence, the visual dynamic working map corresponding to the visual working environment space monitoring data of the robot at the current moment and the current posture of the robot are obtained through the loop detection thread in the SLAM sub-model; further, the loop detection thread in this embodiment is the original loop detection thread in the SLAM algorithm;

[0096] Furthermore, in this embodiment, the current posture of the robot only includes the direction angle of the robot's current operation, which is the angle between the facing direction and the positive Y-axis in the visual coordinate space.

[0097] S209, according to the obtained visual dynamic work map, the dynamic key feature points determined in S206 and the key feature point set in the dynamic target anchor frame are marked to the corresponding position points of the visual dynamic work map through a pixel matching algorithm to obtain a visual real-time 3D work map;

[0098] Furthermore, in this embodiment, the pixel points corresponding to the dynamic key feature points in the original image are matched with the pixels at the positions corresponding to the original dynamic key feature points in the visual dynamic working map through a matching algorithm. When the matching similarity exceeds the matching threshold, the dynamic key feature points are marked at the corresponding positions of the visual dynamic working map; in this embodiment, the matching similarity threshold is 0.8.

[0099] S210, simultaneously obtaining visual working environment space monitoring data of N robots through the controller multi-interface, and obtaining the visual real-time 3D working map and current posture of the corresponding robot through S201-S209;

[0100] S211, according to the obtained visual real-time 3D work maps corresponding to the N robots, through the area overlap detection algorithm, determine and obtain the overlapping area between the visual real-time 3D work map corresponding to the nth robot and the visual real-time 3D work maps corresponding to the remaining N-1 robots;

[0101] S212, obtaining the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the kth robot to the nth robot by changing the corresponding overlapping area of ​​the nth robot and the kth robot;

[0102] S213, obtaining a position error in the overlapping area based on the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the kth robot obtained in S212 to the nth robot and the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the nth robot;

[0103] S214, based on the process of S212-S213, obtaining a position error sequence in the overlapping area between the nth robot and the remaining N-1 robots;

[0104] S215, feeding back the acquired position error sequence in the overlapping area to the loop detection thread in the SLAM sub-model, performing secondary optimization on the visual real-time 3D working map corresponding to the position error in the overlapping area not equal to 0 by acquiring the key frames in the current monitoring environment image frame sequence, and obtaining the optimized visual real-time 3D working map;

[0105] S216. According to S211, it is determined that there is an overlapping visual real-time 3D work map. When the visual real-time 3D work maps of the nth robot and the kth robot overlap, the kth robot is regarded as a dynamic obstacle in the visual real-time 3D work map corresponding to the nth robot at the current moment, and the relative position distance between the kth robot and the nth robot at the current moment is obtained;

[0106] S217. Set a relative distance threshold. When the relative position distance between the kth robot and the nth robot at the current moment is less than the relative distance threshold, the obtained relative position distance is fed back to the enhanced control model to make real-time adjustments to the running direction of the kth robot or the nth robot at the current moment.

[0107] This process obtains high-quality monitoring environment image frame sequences and IMU measurement data by preprocessing the images, videos and IMU measurement data in the visual working environment space monitoring data, extracts key feature points through the ORB sub-model, and uses the YOLOv8 sub-model to detect and classify dynamic, potentially dynamic and static targets. This not only improves the accuracy of feature point extraction, but also realizes real-time annotation of dynamic obstacles in the environment, providing important information support for subsequent path planning; secondly, by combining the detection results with the key feature point space and using the sparse LK optical flow algorithm to calculate the light loss of the key feature points, the detection results are combined with the key feature point space, and the light loss of the key feature points is calculated. It distinguishes dynamic and static feature points in one step, effectively identifies potential dynamic obstacles, and improves the robot's perception of environmental changes. The process is integrated with the visual coordinate space, so that when predicting the robot's posture, only the robot's posture information needs to be predicted, rather than the robot's current position, which further reduces the difficulty of robot control. In addition, the method also takes into account the scenario of multi-robot collaborative work, and uses the area overlap detection algorithm to determine and obtain the overlapping area of ​​the visual real-time 3D work map between multiple robots, and optimizes the position error in the overlapping area, thereby improving the accuracy and reliability of multi-robot collaborative work.

[0108] S3, obtain task status data, and generate a dynamic feasible path for the robot in real time on a visual real-time 3D work map based on the visual work environment space monitoring data through a path algorithm;

[0109] Furthermore, the step of generating a dynamic feasible path of the robot in this embodiment includes:

[0110] S301, according to the obtained visual real-time 3D work map at the current moment, obtain the relative position distance between the dynamic and static key feature points marked in the visual real-time 3D work map and the robot at the current moment, and at the same time input the movement posture information of the marked dynamic key feature points into the pre-trained trajectory prediction model to obtain the movement trajectory information of each dynamic key feature point and the probability of collision between the robot and each dynamic key feature point;

[0111] S302, inputting the relative position distance obtained in S301, the current position and posture of the robot, the moving trajectory information of each dynamic key feature point, and the probability of the robot colliding with each dynamic key feature point into a trajectory generation algorithm to generate a feasible path corresponding to the current moment of the robot;

[0112] S303. Based on the movement trajectory information of each dynamic key feature point, the real-time relative distance from each dynamic key feature point to the robot is obtained. When the real-time relative distance is less than the relative distance threshold, the feasible path is adjusted to obtain a dynamic feasible path. If it is greater than or equal to the relative distance threshold, the feasible path corresponding to the current moment remains unchanged.

[0113] S4. According to the visual working environment space monitoring data, the configured enhanced control model is trained, the trained enhanced control model is obtained, and the trained enhanced control model is integrated into the dynamic feasible path to control the running state of the robot on the dynamic feasible path in real time;

[0114] This process significantly improves the robot's autonomous navigation capability in complex dynamic environments by integrating real-time 3D map monitoring, trajectory prediction, dynamic path planning and reinforcement learning control; real-time environmental data ensures the high accuracy and adaptability of path planning, and the trajectory prediction model effectively predicts the future trajectory of dynamic obstacles, reducing the risk of collision; dynamic adjustment of path strategies ensures that the robot can respond quickly when faced with sudden obstacles and maintain safe and efficient operation; in addition, in this embodiment, the real-time position of each robot is used as the origin, which means that when constructing a feasible path, only the relative distance and collision probability between the obstacle and the origin need to be considered, without considering the specific position of the obstacle, which makes path generation simple and further simplifies the robot's control process.

[0115] Furthermore, the steps of strengthening the control model construction in this embodiment include:

[0116] S401: Construct the current input state based on the current visible real-time 3D work map, relative distance, robot posture, and dynamic feasible path. :

[0117] ,

[0118] in, Represents the three-dimensional spatial coordinates of the task target point corresponding to the task status data, , and In turn, the three-dimensional coordinates of the nth robot on the visual real-time 3D work map are represented. In this embodiment, and Real time is equal to 0; Indicates the posture angle of the nth robot at the current time t, Indicates the real-time driving speed of the nth robot corresponding to the visible real-time 3D work map at the current time t; represents the relative distance between the nth robot and the kth dynamic obstacle at the current time t, represents the probability that the nth robot collides with the kth dynamic obstacle when moving along the current feasible path at the current time t. represents the relative distance between the nth robot and the qth static obstacle at the current time t; represents the dynamic feasible path corresponding to the current time t of the nth robot; represents the visual real-time 3D work map corresponding to the current time t of the nth robot; further, this embodiment is equal to 0 or 1, when Indicates that the task status data has no task target point. Indicates that the task status data has a task target point; according to The setting of this method can cope with different execution processes, such as mine inspection without a task target endpoint or logistics handling tasks with a task target endpoint. Different task types can be switched by changing only one parameter, which further improves the adaptability of this method.

[0119] S402, setting robot control trigger information according to the current input state, including:

[0120] Set the collision probability threshold ,when Greater than , triggering the first control information, that is, adjusting the posture angle of the n-th robot, and adjusting the dynamic feasible path corresponding to the n-th robot through the adjusted posture angle;

[0121] when or When it is greater than the relative distance threshold, the second control information is triggered, that is, the posture angle of the nth robot is adjusted, and the dynamic feasible path corresponding to the nth robot is adjusted through the adjusted posture angle;

[0122] Set the ambient speed threshold ,when Greater than The third control information is triggered when the robot is in a safe range; that is, the speed is adjusted so that the robot's running speed is always kept within a safe range; the safe range is between the robot's lowest running speed and the environmental speed threshold;

[0123] Furthermore, the environmental speed threshold set in the present embodiment is the average value of the corresponding moving speeds of the robot in different moving environments; further, in the present embodiment, different moving environments include factories, mines, mine caves or indoor spaces, etc., and the specific settings are set by technical personnel in this field according to the specific operating environment.

[0124] S403: Based on the set control trigger information, construct the execution action corresponding to the current moment ,in, Indicates the action that triggers the first control information to be executed. Indicates the action that triggers the execution of the corresponding second control information. Indicates the action that triggers the execution of the corresponding third control information;

[0125] S404: construct an enhanced control model based on TD3, input the constructed current input state, control trigger information and the execution action corresponding to the current moment into the enhanced control model for training, and obtain a trained enhanced control model.

[0126] S5. Obtain the visual working environment space monitoring data and operation status data of the robot at every moment on the dynamic feasible path in real time, and update and correct the constructed visual real-time 3D work map, dynamic feasible path and operation status in real time through the process from S1 to S4 until the robot completes the task.

[0127] Example 2

[0128] See also Figure 3 ,Another embodiment provided by the present invention: An automatic controller of a mobile robot, comprising: a coordinate construction module, a map construction module, a path generation module, a reinforcement control module and an update module;

[0129] A coordinate construction module is used to construct a robot visual coordinate space based on the acquired robot current position and visual working environment space monitoring data; the coordinate construction module includes a construction unit and a conversion unit;

[0130] A construction unit, used to construct a robot visual coordinate space according to the acquired robot current position and visual working environment space monitoring data; the visual coordinate space includes an initial visual coordinate space and a secondary visual coordinate space;

[0131] A conversion unit, used to convert between the initial visual coordinate space and the secondary visual coordinate space according to the monitoring distance length in each direction; specifically, when the monitoring distance lengths of the robot in each monitoring direction are not equal, the corresponding initial visual coordinate space is converted into the secondary visual coordinate space;

[0132] The map construction module is used to obtain the visual real-time 3D working map and current posture of the robot in the visual coordinate space based on the acquired visual working environment space monitoring data and the constructed SLAM optimization model;

[0133] The path generation module is used to generate a dynamic feasible path for the robot through a path generation algorithm according to the visual real-time 3D work map and current posture of the robot;

[0134] The enhanced control module is used to control the robot's operating state on a dynamic feasible path in real time according to the visual working environment space monitoring data through the trained enhanced control model until the task is completed.

[0135] The updating module is used to update and correct the constructed visual real-time 3D work map, dynamic feasible path and running status in real time according to the real-time acquired visual working environment space monitoring data and running status data of the robot at each moment on the dynamic feasible path until the robot completes the task.

[0136] Example 3

[0137] A computer-readable storage medium stores computer instructions, which execute an automatic control method for a mobile robot when the computer instructions are executed.

[0138] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements an automatic control method for a mobile robot when executing the computer program.

[0139] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in the field may also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are within the protection of the present invention.

Claims

1. A method for automatic control of a mobile robot, characterized in that: include: S1. Obtain the monitoring data of the robot's current position and visual working environment space, and use the robot's current position as the origin to construct the robot's visual coordinate space in the visual working environment space, and build the constructed robot's visual coordinate space into the robot control; S2. Construct a SLAM optimization model, integrate the constructed SLAM optimization model with the visual coordinate space, input the acquired visual working environment space monitoring data into the SLAM optimization model, construct and obtain the robot's visual dynamic working map and the robot's current posture in the visual coordinate space, and mark the dynamic and static obstacle positions in the visual dynamic working map in real time to obtain a visual real-time 3D working map; S3, obtain task status data, and generate a dynamic feasible path for the robot in real time on a visual real-time 3D work map based on the visual work environment space monitoring data through a path algorithm; S4. According to the visual working environment space monitoring data, the configured enhanced control model is trained, the trained enhanced control model is obtained, and the trained enhanced control model is integrated into the dynamic feasible path to control the running state of the robot on the dynamic feasible path in real time; S5. Obtain the visual working environment space monitoring data and operation status data of the robot at every moment on the dynamic feasible path in real time, and update and correct the constructed visual real-time 3D work map, dynamic feasible path and operation status in real time through the process from S1 to S4 until the robot completes the task.

2. The automatic control method of a mobile robot according to claim 1, characterized in that: The steps of constructing the robot visual coordinate space include: S101, obtaining the monitoring radius parameter r of the robot sensor, and using the current position of the robot as the coordinate origin to construct a three-dimensional monitoring coordinate; S102, obtaining a visible area with a monitoring radius of r on the xy plane of the three-dimensional monitoring coordinates according to the acquired monitoring radius parameter; S103, embed the acquired visible area into the three-dimensional monitoring coordinates to obtain the initial visible coordinate space V, specifically: ,in is the spatial coordinate of any point in the initial visual coordinate space, and In turn, they represent the minimum and maximum monitoring heights of the sensor in the vertical direction; S104, based on the constructed initial visual coordinate space and the monitoring distance data of the robot during actual movement, a secondary visual coordinate space is constructed in the three-dimensional monitoring coordinates through the ellipse equation , specifically: ， in, It represents the shortest distance of the robot in each monitoring direction in the visual coordinate space at time t, It represents the longest distance of the robot in each monitoring direction in the visual coordinate space at time t; S105 . When the monitoring distances of the robot in each monitoring direction are not equal, the corresponding initial visual coordinate space is converted into a secondary visual coordinate space.

3. The automatic control method of a mobile robot according to claim 2, characterized in that: The steps of constructing the visual real-time 3D work map include: S201, preprocessing the images, videos and IMU measurement data in the visual working environment space monitoring data through the built-in preprocessing algorithm in the sensor, and obtaining the preprocessed monitoring environment image frame sequence and IMU measurement data; S202, inputting the acquired pre-processed monitoring environment image frame sequence into the ORB sub-model configured in the SLAM optimization model to extract key feature points, and obtaining a key feature point space; S203, inputting the acquired pre-processed monitoring environment image frame sequence into the pre-trained YOLOv8 sub-model, performing dynamic detection on dynamic targets, potential dynamic targets and static targets in the input image, and outputting the anchor frame coordinates of dynamic detection of dynamic targets, potential dynamic targets and static targets; S204, based on the anchor frame coordinates and key feature point space of dynamic, potential dynamic, and static target dynamic detection, obtain the key feature point set in the dynamic target anchor frame , the key feature point set in the potential dynamic anchor box And static key feature point set ,in, Represents the static key feature point set corresponding to the jth image in the monitoring environment image frame sequence; S205, will and After cascading, it is input into the sparse LK optical flow algorithm to obtain the light loss corresponding to each key feature point , and at the same time Input to the LK optical flow algorithm to obtain The average value of light loss corresponding to all key feature points in ,in express and The amount of light loss corresponding to the i-th key feature point in .

4. The automatic control method of a mobile robot according to claim 3, characterized in that: The step of constructing the visual real-time 3D work map also includes: S206, based on the obtained light loss amount of the potential dynamic key feature point and the average light loss amount corresponding to the static key feature point, a judgment is made through a judgment formula. If the corresponding relationship of the formula is satisfied, the corresponding key feature point is a dynamic key feature point, otherwise the key feature point is a static key feature point. The judgment formula is specifically: , where l represents the streamer threshold; S207, constructing a static key feature point space set using the acquired static key feature points and the static key feature point set, inputting the acquired static key feature point space set and the IMU measurement data into the local mapping thread in the SLAM sub-model for key frame judgment, and obtaining the key frame in the current monitoring environment image frame sequence; S208, according to the key frames and IMU measurement data in the current monitoring environment image frame sequence, through the loop detection thread in the SLAM sub-model, obtain the visual dynamic working map corresponding to the visual working environment space monitoring data of the robot at the current moment and the current posture of the robot; S209: Based on the obtained visual dynamic work map, the dynamic key feature points obtained by S206 and the key feature point set in the dynamic target anchor frame are marked to the corresponding position points of the visual dynamic work map through a pixel matching algorithm to obtain a visual real-time 3D work map.

5. The automatic control method of a mobile robot according to claim 4, characterized in that: The step of constructing the visual real-time 3D work map also includes: S210, simultaneously obtaining visual working environment space monitoring data of N robots through the controller multi-interface, and obtaining the visual real-time 3D working map and current posture of the corresponding robot through S201-S209; S211, according to the obtained visual real-time 3D work maps corresponding to the N robots, through the area overlap detection algorithm, determine and obtain the overlapping area between the visual real-time 3D work map corresponding to the nth robot and the visual real-time 3D work maps corresponding to the remaining N-1 robots; S212, obtaining the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the kth robot to the nth robot by changing the corresponding overlapping area of ​​the nth robot and the kth robot; S213, obtaining a position error in the overlapping area based on the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the kth robot obtained in S212 to the nth robot and the position coordinates of each target point in the overlapping area of ​​the visual real-time 3D work map corresponding to the nth robot; S214. Based on the process of S212-S213, obtain the position error sequence in the overlapping area between the nth robot and the remaining N-1 robots.

6. The automatic control method of a mobile robot according to claim 5, characterized in that: The step of constructing the visual real-time 3D work map also includes: S215, feeding back the acquired position error sequence in the overlapping area to the loop detection thread in the SLAM sub-model, performing secondary optimization on the visual real-time 3D working map corresponding to the position error in the overlapping area not equal to 0 by acquiring the key frames in the current monitoring environment image frame sequence, and obtaining the optimized visual real-time 3D working map; S216. According to S211, it is determined that there is an overlapping visual real-time 3D work map. When the visual real-time 3D work maps of the nth robot and the kth robot overlap, the kth robot is regarded as a dynamic obstacle in the visual real-time 3D work map corresponding to the nth robot at the current moment, and the relative position distance between the kth robot and the nth robot at the current moment is obtained; S217. Set a relative distance threshold. When the relative position distance between the kth robot and the nth robot at the current moment is less than the relative distance threshold, the obtained relative position distance is fed back to the enhanced control model to make real-time adjustments to the running direction of the kth robot or the nth robot at the current moment.

7. The automatic control method of a mobile robot according to claim 6, characterized in that: The step of generating a dynamic feasible path of the robot in S3 includes: S301, according to the obtained visual real-time 3D work map at the current moment, obtain the relative position distance between the dynamic and static key feature points marked in the visual real-time 3D work map and the robot at the current moment, and at the same time input the movement posture information of the marked dynamic key feature points into the pre-trained trajectory prediction model to obtain the movement trajectory information of each dynamic key feature point and the probability of collision between the robot and each dynamic key feature point; S302, inputting the relative position distance obtained in S301, the current position and posture of the robot, the moving trajectory information of each dynamic key feature point, and the probability of the robot colliding with each dynamic key feature point into a trajectory generation algorithm to generate a feasible path corresponding to the current moment of the robot; S303. Based on the movement trajectory information of each dynamic key feature point, the real-time relative distance from each dynamic key feature point to the robot is obtained. When the real-time relative distance is less than the relative distance threshold, the feasible path is adjusted to obtain a dynamic feasible path. If it is greater than or equal to the relative distance threshold, the feasible path corresponding to the current moment remains unchanged.

8. The automatic control method of a mobile robot according to claim 7, characterized in that: The steps of building the enhanced control model in S4 include: S401: Construct the current input state based on the current visible real-time 3D work map, relative distance, robot posture, and dynamic feasible path. : ， in, Represents the three-dimensional spatial coordinates of the task target point corresponding to the task status data, , and represents the three-dimensional coordinates of the nth robot on the visual real-time 3D work map, Indicates the posture angle of the nth robot at the current time t, Indicates the real-time driving speed of the nth robot corresponding to the visible real-time 3D work map at the current time t; represents the relative distance between the nth robot and the kth dynamic obstacle at the current time t, represents the probability that the nth robot collides with the kth dynamic obstacle when moving along the current feasible path at the current time t. represents the relative distance between the nth robot and the qth static obstacle at the current time t; represents the dynamic feasible path corresponding to the current time t of the nth robot; Represents the visual real-time 3D work map corresponding to the current time t of the nth robot; S402, setting robot control trigger information according to the current input state, including: Set the collision probability threshold ,when Greater than , triggering the first control information, that is, adjusting the posture angle of the n-th robot, and adjusting the dynamic feasible path corresponding to the n-th robot through the adjusted posture angle; when or When it is greater than the relative distance threshold, the second control information is triggered, that is, the posture angle of the nth robot is adjusted, and the dynamic feasible path corresponding to the nth robot is adjusted through the adjusted posture angle; Set the ambient speed threshold ,when Greater than The third control information is triggered when S403: Based on the set control trigger information, construct the execution action corresponding to the current moment ,in, Indicates the action that triggers the first control information to be executed. Indicates the action that triggers the execution of the corresponding second control information. Indicates the action that triggers the execution of the corresponding third control information.

9. An automatic controller of a mobile robot, used to implement an automatic control method of a mobile robot according to any one of claims 1 to 8, characterized in that: include: Coordinate construction module, map construction module, path generation module and reinforcement control module; The coordinate construction module is used to construct the robot's visual coordinate space based on the acquired robot's current position and visual working environment space monitoring data; The map construction module is used to obtain the visual real-time 3D working map and current posture of the robot in the visual coordinate space through the constructed SLAM optimization model according to the acquired visual working environment space monitoring data; The path generation module is used to generate a dynamic feasible path for the robot through a path generation algorithm according to the visual real-time 3D work map and current posture corresponding to the robot; The enhanced control module is used to control the running state of the robot on the dynamic feasible path in real time according to the visual working environment space monitoring data through the trained enhanced control model until the task is completed.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the automatic control method of a mobile robot described in any one of claims 1-8 is executed.

Images