Robot control method, robot and readable storage medium

Abstract

The invention relates to a robot control method, a robot and a readable storage medium. The method comprises the steps that a time sequence surround view image of the current environment where the robot is located and state information of the current robot are obtained; the time sequence look-around image comprises multiple frames of look-around images of the current environment of the robot and time information of the multiple frames of look-around images; inputting the time sequence look-around image and the state information into a pre-trained action control model to obtain action parameters of the robot; and generating a control instruction according to the action parameters, and controlling the robot to move according to the control instruction. By using the robot control method provided by the invention, the robot can be accurately controlled.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a robot control method, a robot, and a readable storage medium. Background Technology

[0002] With the increasing emphasis placed on ecological civilization construction by the nation, lawns are ubiquitous, from roadside greening to community parks, from outdoor recreation to sports and entertainment. Due to their broad market and clear demand, lawn maintenance and beautification have enormous development potential. Intelligent lawn mowing robots are widely used because they can achieve autonomous, unmanned lawn mowing outdoors, improving efficiency and saving time and labor costs.

[0003] Existing lawn mowing robots rely on a sequential path of "perception-mapping-planning-control" for lawn covering operations. However, information loss and error accumulation occur between perception, mapping, and planning, leading to inaccurate robot control. Summary of the Invention

[0004] Therefore, it is necessary to provide a robot control method, a robot, and a readable storage medium that can improve the accuracy of robot control in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a robot control method, the method comprising:

[0006] Acquire a temporal surround view image of the robot's current environment, as well as the robot's current state information; the temporal surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images;

[0007] The time-series loop image and state information are input into a pre-trained motion control model to obtain the robot's motion parameters;

[0008] Control commands are generated based on motion parameters, and the robot is moved according to these commands.

[0009] In one embodiment, the motion control model includes a feature extraction network, a feature concatenation network, a time series modeling network, and a motion output network connected in sequence. The model inputs a time-series loop image and state information into a pre-trained motion control model to obtain the robot's motion parameters, including:

[0010] Image features and temporal features are obtained by extracting features from multiple frames of panoramic images using a feature extraction network. The image features include semantic features representing boundary lines, obstacles, covered areas, and uncovered areas.

[0011] An environmental feature representation is obtained by concatenating image features, temporal features, and state information using a feature concatenation network.

[0012] By fusing information from environmental feature representations through a time series modeling network, a high-dimensional state representation is obtained.

[0013] The high-dimensional state representation is mapped to action parameters through the action output network.

[0014] In one embodiment, the method further includes:

[0015] Obtain the robot's movement path and coverage area based on control commands;

[0016] Based on reinforcement learning algorithms, the network parameters of the motion control model are updated according to the temporal loop image, state information, robot motion parameters, movement path and coverage area.

[0017] In one embodiment, based on a reinforcement learning algorithm, the network parameters of the motion control model are updated according to the temporal loop image, state information, robot motion parameters, and movement path and coverage area, including:

[0018] Obtain the robot's historical movement paths and corresponding historical coverage areas;

[0019] New training samples are constructed based on the time-series loop image, state information, robot motion parameters, movement path and coverage area, as well as historical movement paths and historical coverage areas.

[0020] The network parameters of the motion control model are updated based on the new training samples.

[0021] In one embodiment, the new training samples include the current state, action, reward value, and next state; based on the time-series loop view image, state information, robot action parameters, movement path and coverage area, and historical movement paths and historical coverage areas, new training samples are constructed, including:

[0022] Based on the time-series loop image and state information, determine the robot's current state;

[0023] Determine the robot's actions based on its motion parameters;

[0024] The reward value is calculated based on the mobile path and coverage area, as well as the historical mobile path and historical coverage area, using a preset reward function.

[0025] The robot's next state is determined based on its state after moving according to control commands.

[0026] In one embodiment, the reward function includes positive rewards and negative rewards. Positive rewards include newly added coverage areas, and negative rewards include at least one of total path length penalty, collision event penalty, boundary event penalty, and path overlap length penalty.

[0027] In one embodiment, a reward value is calculated based on a preset reward function, according to the movement path and coverage area, as well as historical movement paths and historical coverage areas, including:

[0028] The new coverage area is determined based on the existing coverage area and the historical coverage area;

[0029] Based on the movement path and historical movement paths, determine the total path length and path overlap length;

[0030] Determine collision events and boundary events based on the movement path;

[0031] The reward value is obtained by substituting the newly added coverage area, total path length, path overlap length, collision event, and boundary event into the reward function.

[0032] In one embodiment, acquiring a temporal surround view image of the robot's current environment includes:

[0033] Acquire multi-frame surround view images of the robot's current environment;

[0034] Stacking multiple frames of loop images along the time dimension yields a time-series loop image.

[0035] In one embodiment, the training process of the motion control model includes:

[0036] Obtain training samples;

[0037] The initial network model is trained based on the training samples to obtain the action control model. The initial network model includes a feature extraction network, a feature splicing network, a time series modeling network, an action output network, and a value estimation network. The feature extraction network, feature splicing network, time series modeling network, and action output network are connected in sequence, and the time series modeling network is also connected to the value estimation network.

[0038] Secondly, one embodiment of this application provides a robot control device, which includes:

[0039] The acquisition module is used to acquire the time-series surround view image of the robot's current environment, as well as the robot's current state information; the time-series surround view image includes multiple frames of surround view image of the robot's current environment and the time information of the multiple frames of surround view image;

[0040] The input module is used to input the time-series loop view image and state information into the pre-trained motion control model to obtain the robot's motion parameters;

[0041] The control module is used to generate control commands based on motion parameters and control the robot's movement according to the control commands.

[0042] Thirdly, one embodiment of this application provides a robot, including a robot body, an image acquisition component, a status acquisition component, and a control component. The image acquisition component, the status acquisition component, and the control component are disposed on the robot body, and the control component is connected to the image acquisition component and the status acquisition component.

[0043] The image acquisition component is used to acquire a time-series surround view image of the robot's current environment and transmit the time-series surround view image to the control component; the time-series surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images;

[0044] The status acquisition component is used to acquire the current status information of the robot body and transmit the status information to the control component;

[0045] The control component is used to input the time-series loop view image and state information into the pre-trained motion control model to obtain the motion parameters of the robot body; generate control commands based on the motion parameters; and control the movement of the robot body according to the control commands.

[0046] Fourthly, one embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0047] Fifthly, one embodiment of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method provided in the first aspect.

[0048] The aforementioned robot control method, robot, and readable storage medium involve acquiring a temporal surround view image of the robot's current environment and the robot's current state information. The temporal surround view image includes multiple frames of the robot's current environment and their time information. The temporal surround view image and state information are input into a pre-trained motion control model to obtain the robot's motion parameters. Control commands are generated based on the motion parameters, and the robot is moved according to these commands. In this embodiment, the motion control model directly outputs motion parameters based on the temporal surround view image and state information. This end-to-end mapping establishes a clear path from perception to control, eliminating the need for explicit mapping and path planning. This results in a shorter information transmission chain, real-time response, and improved accuracy in robot control. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of the robot's structure in one embodiment;

[0051] Figure 2 This is a flowchart illustrating the steps of a robot control method in one embodiment;

[0052] Figure 3 This is a schematic diagram of the motion control model in one embodiment;

[0053] Figure 4 This is a flowchart illustrating the steps of a robot control method in another embodiment;

[0054] Figure 5 This is a flowchart illustrating the steps of a robot control method in another embodiment;

[0055] Figure 6 This is a flowchart illustrating the steps of a robot control method in another embodiment;

[0056] Figure 7 This is a flowchart illustrating the steps of a robot control method in another embodiment;

[0057] Figure 8 This is a flowchart illustrating the steps of a robot control method in another embodiment;

[0058] Figure 9 This is a flowchart illustrating the steps of a robot control method in another embodiment;

[0059] Figure 10 This is a flowchart illustrating the steps of a robot control method in another embodiment;

[0060] Figure 11 This is a schematic diagram of the structure of the initial network model in one embodiment;

[0061] Figure 12 A flowchart illustrating the steps of the robot control method in another embodiment;

[0062] Figure 13 This is a control architecture for a robot control method in one embodiment;

[0063] Figure 14 This is a schematic diagram of the robot control device in one embodiment;

[0064] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] It should be noted that the terms "comprising" and "having" as used in this application, and any variations thereof, are intended to cover non-exclusive inclusion.

[0067] First, before introducing the technical solutions of the embodiments disclosed in this application, the background technology or technological evolution on which the embodiments of this application are based will be introduced. In the field of robotics, traditional navigation algorithms need to assume the existence of a map, which is generally obtained through Simultaneous Localization and Mapping (SLAM) algorithms, and then the obstacles on the map are expanded. The parameters for the expansion process are generally the robot radius plus an appropriate threshold. Due to mapping errors, this threshold is usually at least greater than the minimum map resolution. Because of this threshold, a certain area that could be reached may be judged as unreachable by the navigation algorithm (generally because there are narrow passages on the navigation path), thus preventing the robot from reaching a certain area. Assuming the robot is a robot vacuum cleaner, this will cause missed areas. If the robot calls the above navigation algorithm while in a trapped state, misjudgment may occur, that is, it may be determined that there is no escape path. Other navigation algorithms, such as the artificial potential field method, even if a navigation path is planned, still have the problem of too many collisions when performing actions according to the path, making it impossible to pass. In addition, the excessive number of collisions when the robot tries to escape from narrow passages or small areas significantly affects the accuracy of mapping, thus impacting map-dependent navigation algorithms.

[0068] Existing lawn mowing robots generally rely on a sequential path of "perception-mapping-planning-control" for lawn cover operations. Typical solutions include random walking, regular strip scanning, and map-based cover planning. Random walking relies on simple obstacle avoidance and random turning control. Although simple to implement, it has slow coverage convergence, a high rate of repeated mowing, and low work efficiency. Regular path methods achieve coverage through back-and-forth straight lines, spirals, or zonal scanning, requiring the assumption of regular lawn boundaries and few obstacles. In complex environments or with irregular boundaries, it is prone to missed mowing or repeated mowing. Map-based cover planning methods rely on positioning systems such as SLAM and Real-Time Kinematic-Global Positioning System (RTK-GPS) to obtain lawn maps and then generate cover paths on the maps. These solutions have long system chains, high requirements for sensors and computing power, and positioning and mapping errors will gradually propagate to the path planning stage, causing safety redundancy and reduced coverage. In other mobile robot fields, such as indoor robotic vacuum cleaners, there are already solutions that utilize Time-of-Flight (ToF) or point cloud sensors to identify narrow passages and perform path planning. While these solutions improve maneuverability, they still rely on explicit geometric modeling and rule-based path reasoning, essentially remaining a multi-stage manual design process. For lawnmower robots, existing vision applications are mostly used for obstacle recognition or boundary detection. Although information such as whether the lawn has been mowed or whether it is near a boundary can be clearly observed from images, traditional systems do not allow the robot to "understand" these visual semantics, but rather rely on thresholds and rules for processing. Therefore, they lack the ability to automatically optimize walking strategies based on experience in long-term tasks. To address the aforementioned problems in the control of existing lawnmower robots, this application provides a robot control method.

[0069] The robot control method provided in this application embodiment can be applied to, for example, Figure 1 The robot shown includes a robot body 10, an image acquisition component 11, a status acquisition component 12, and a control component 13. The image acquisition component 11, status acquisition component 12, and control component 13 are all mounted on the robot body 10. The control component 13 is communicatively connected to the image acquisition component 11 and the status acquisition component 12.

[0070] The robot body 10 includes a chassis, drive wheels, follower wheels, and working tools. The chassis serves as the carrier for all structures mounted on the robot body. The drive wheels are those that actively output power to propel the device. The follower wheels are those that receive no power input, move only with the drive wheels, and provide support and guidance. The working tools are the tools used by the robot during operation. If the robot is a sweeping robot, the working tools are cleaning components; if the robot is a lawnmower robot, the working tools are lawnmower blades.

[0071] The image acquisition component 11 can be multiple cameras deployed around the robot body 10, including at least a front-facing camera, a rear-facing camera, a left-facing camera, and a right-facing camera. The field of view of the cameras can be wide-angle or fisheye, enabling full coverage perception of the area surrounding the robot (surrounding lawns and obstacles). The image acquisition component 11 is used to acquire multiple frames of images of the robot body's current environment, that is, to continuously acquire single-frame images from various perspectives of the robot's current environment, and transmit the acquired multiple frames of images to the control component 13. The control component 13 stacks the acquired multiple frames of images in the channel dimension and the time dimension to obtain a temporal surround view image of the robot body's current environment. The temporal surround view image includes multiple frames of surround view images of the robot body's current environment, as well as the time information of the multiple frames of surround view images.

[0072] A description of the timing loop image can be found in the following embodiment.

[0073] The state acquisition component 12 can be an attitude and odometry module. The attitude and odometry module may include an inertial measurement unit (accelerometer and gyroscope), wheel speed encoder, etc. The state acquisition component 12 is used to acquire the current state information of the robot body and transmit this information to the control component 13. The robot body's state information includes the robot's pose and motion velocity (linear velocity, angular velocity, and acceleration, etc.).

[0074] The control component 13 can acquire a time-series surround view image of the robot's current environment, as well as the robot's state information. The control component 13 is used to input the time-series surround view image and state information into a pre-trained motion control model to obtain the robot's motion parameters; generate control commands based on the motion parameters; and control the robot's movement according to the control commands. A description of the steps performed by the control component 13 can be found in the following embodiment.

[0075] The technical solution of this application and how the technical solution of this application solves the technical problem are described in detail below with specific embodiments.

[0076] In one embodiment, such as Figure 2 As shown, a robot control method is provided, which can be applied to... Figure 1 The method will be illustrated using the control component as an example. In this embodiment, the method includes the following steps:

[0077] Step 200: Obtain the time-series surround view image of the robot's current environment and the robot's current state information; the time-series surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images.

[0078] A surround-view image refers to an image formed by stitching or fusing multiple images of the robot's current environment acquired by the robot's image acquisition component within the current control cycle. Multi-frame surround-view images include the surround-view image of the current control cycle, as well as surround-view images from several control cycles prior to the current control cycle. The temporal information of a multi-frame surround-view image refers to the time sequence formed by the control cycles that acquire each frame of the surround-view image. After acquiring multiple frames of images of the robot's current environment from the image acquisition component, the control component can obtain a temporal surround-view image by stacking these multiple frames in multiple dimensions.

[0079] The robot's current state information includes its pose and velocity, with velocity encompassing linear velocity, angular velocity, and acceleration. When the robot moves to or is placed in its current environment, the image acquisition component within the robot acquires multiple frames of images of the environment and transmits them to the control component. The control component then uses these frames to obtain a temporal surround view of the robot's current environment. The state acquisition component obtains the robot's current state information and transmits it to the control component.

[0080] Step 210: Input the time-series loop view image and state information into the pre-trained motion control model to obtain the robot's motion parameters.

[0081] The motion control model can be pre-trained and stored in the control component. After acquiring the timing loop view image and state information, the control component inputs both the timing loop view image and state information into the motion control model, and outputs the robot's motion parameters through the motion model. The robot's motion parameters refer to the robot's walking motion parameters within the current control cycle. Motion parameters can include the robot's walking direction and speed.

[0082] Understandably, the image acquisition components on current robots (circumferentially distributed wide-angle or fisheye cameras) can form an omnidirectional surround view, which enables the motion control model to simultaneously extract forward obstacle features, lateral boundary features, and texture features of the back-covered area (if the robot is a lawnmower, the texture features of the covered area refer to the texture features of the mowed lawn), thereby providing complete spatial context support for end-to-end decision-making.

[0083] In an optional embodiment, the robot's walking actions may include the probability distribution of each action in a discrete action set, such as turning right, moving straight, turning left, and stopping. The control component can select actions from the discrete action set according to a sampling or greedy strategy. The sampling strategy refers to randomly selecting actions according to the probability distribution, while the greedy strategy refers to selecting the action with the highest probability.

[0084] Step 220: Generate control commands based on motion parameters, and control the robot to move according to the control commands.

[0085] After receiving the motion parameters, the control component generates control commands based on the motion parameters and controls the robot's movement according to the control commands. This embodiment does not limit the specific method for generating control commands based on the motion parameters, as long as the function can be achieved.

[0086] In an optional embodiment, the control component pre-stores a mapping relationship between action parameters and control commands. After obtaining the action parameters, the control component searches for the corresponding control command in this mapping relationship.

[0087] In another optional embodiment, if the motion parameters include linear velocity parameters and angular velocity parameters, the control component directly uses the motion parameters as control commands after obtaining them.

[0088] In an optional embodiment, the control component can control the robot's tools to operate while controlling the robot's movement according to control commands. If the robot is a lawnmower robot, then while controlling the movement of the lawnmower robot, the control component can also control the operation of the lawnmower blades and maintain their working state.

[0089] The robot control method provided in this application acquires a temporal surround view image of the robot's current environment and the robot's current state information. The temporal surround view image includes multiple frames of the robot's current environment and the time information of these frames. The temporal surround view image and state information are input into a pre-trained motion control model to obtain the robot's motion parameters. Control commands are generated based on the motion parameters, and the robot is moved according to the control commands. In this embodiment, the motion control model directly outputs motion parameters based on the temporal surround view image and state information. This end-to-end mapping establishes a clear path from perception to control, eliminating the need for explicit mapping and path planning. This results in a shorter information transmission chain, real-time response, and improved accuracy in robot control. Furthermore, the motion parameters obtained in this embodiment are derived through network learning, not fixed geometric rules, allowing the robot control method to be applied to different environments and achieve accurate robot control in various environments.

[0090] In one embodiment, such as Figure 3 As shown, the motion control model includes a feature extraction network, a feature splicing network, a time series modeling network, and an motion output network connected in sequence.

[0091] The output of the feature extraction network is connected to the input of the feature splicing network, the output of the feature splicing network is connected to the input of the time series modeling network, and the output of the time series modeling network is connected to the input of the action output network.

[0092] The feature extraction network can be a convolutional neural network (CNN), and the feature concatenation network can be a CNN or a multilayer perceptron. The time series modeling network can be a Long Short-Term Memory (LSTM), a Gated Recurrent Unit (GRU), or a temporal Transformer based on a self-attention mechanism. The action output network can use fully connected layers. This embodiment does not limit the specific types of feature extraction networks, feature concatenation networks, time series modeling networks, and action output networks, as long as their functions can be achieved.

[0093] Based on this, such as Figure 4 As shown, this involves an implementation method that inputs a time-series loop view image and state information into a pre-trained motion control model to obtain the robot's motion parameters. The steps of this implementation method include:

[0094] Step 400: Extract features from multiple frames of panoramic images using a feature extraction network to obtain image features and temporal features; image features include semantic features representing boundary lines, obstacles, covered areas, and uncovered areas.

[0095] The control component inputs the temporal panoramic image into the feature extraction network. The feature extraction network extracts features from the multi-frame panoramic image to obtain image features and temporal features.

[0096] In an optional embodiment, the feature extraction network includes multiple convolutional feature extraction sub-networks. Each convolutional feature extraction sub-network extracts features from each frame of the panoramic view image to obtain the image features of that panoramic view image. If the robot is a lawnmower robot, the feature extraction network can extract the lawn texture and mowing mark texture of the robot's environment, thereby determining the semantic features representing covered and uncovered areas. Based on the movement path of the lawnmower robot, the semantic features of standard boundary lines and obstacles can be determined. Based on the temporal information of multiple frames of panoramic view images, the multiple frames of panoramic view images are stacked in the time dimension or temporal features are extracted using a temporal convolutional network.

[0097] Specifically, if the robot is a lawnmower, visually, undisturbed lawn typically appears as a tall, disordered natural lawn texture, while mowed lawn appears as a smooth surface with regular cut marks formed by the blades. Since the motion control model is trained within an end-to-end reinforcement learning framework, the feature extraction network can autonomously learn the visual differences between the lawn texture and cut marks from multiple frames of input panoramic images. Therefore, the feature extraction network can implicitly extract semantic features representing mowed (covered) and undisturbed (uncovered) lawns, as well as the semantic features of boundary lines between lawn and non-lawn areas and the semantic features of obstacles, without relying on manual feature engineering, image segmentation, or manually set thresholds. This feature extraction network significantly improves the adaptability to different lawn textures and lighting environments.

[0098] Step 410: Use a feature concatenation network to concatenate image features, temporal features, and state information to obtain an environmental feature representation.

[0099] After obtaining image and temporal features, the control component inputs these features, along with the robot's current state information, into a feature concatenation network. This network concatenates the image features, temporal features, and state information to obtain a unified environmental feature representation. Essentially, the control component concatenates the image features extracted from each frame of the surround view image along either the channel or feature dimension, and then connects the concatenated features to the state vector to obtain the environmental feature representation, supplementing it with information such as pose and motion velocity.

[0100] Step 420: Use a time series modeling network to fuse information from the environmental feature representation to obtain a high-dimensional state representation.

[0101] After obtaining the environmental feature representation, the control component inputs the environmental feature representation into the time series modeling network. The time series modeling network models the environmental changes and robot motion state changes corresponding to multiple frames of surround view images based on the environmental feature representation, and obtains a high-dimensional state representation.

[0102] Step 430: Map the high-dimensional state representation to action parameters through the action output network.

[0103] After obtaining the high-dimensional state representation, the control component inputs it into the action output network, which then maps the high-dimensional state to action parameters. Specifically, the action output network maps the high-dimensional state representation to action parameters through a fully connected layer.

[0104] In this embodiment, the feature extraction network, feature stitching network, time series modeling network, and motion output network of the motion control model are decoupled, functionally independent, and easy to train and iterate. The feature extraction network extracts image features from multiple frames of time-series panoramic images, eliminating the need for manual extraction or feature segmentation, thus improving the automation of the motion control model. The time series modeling network captures changes in the environment and the robot's motion state, ensuring that the final output motion parameters fit the current environment, thereby improving the generalization ability of the motion control model and enabling accurate control of the robot.

[0105] In one embodiment, during the current control cycle, after the control component controls the robot to move according to the control instructions, such as... Figure 5 As shown, the method further includes the following steps:

[0106] Step 500: Obtain the robot's movement path and coverage area based on control commands.

[0107] After controlling the robot to move based on control commands, the control component can obtain the robot's movement path and the coverage area during the robot's movement.

[0108] In an optional embodiment, during robot movement, images of the movement are acquired by an image acquisition component within the robot, and state information of the robot during movement is acquired by a state acquisition component. Based on the acquired images and state information, the movement path and coverage area can be determined. Assuming the robot is a lawnmower robot, the coverage area is the area of ​​grass that has already been mowed during the robot's movement.

[0109] In an optional embodiment, the control component can spatially rasterize the robot's environment, with the coverage area including the number of grids covered during the robot's movement. It is understood that spatially rasterizing the robot's environment is only for calculating the area covered during the robot's movement to determine the reinforcement learning reward value, and does not participate in the robot's forward navigation decisions or path planning.

[0110] Step 510: Based on the reinforcement learning algorithm, update the network parameters of the motion control model according to the temporal loop image, state information, robot motion parameters, and movement path and coverage area.

[0111] After acquiring the robot's movement path and coverage area based on control commands, the control component updates the network parameters of the motion control model using reinforcement learning algorithms, along with temporal look-around information of the robot's current environment, the robot's current state information, the robot's motion parameters, and the movement path and coverage area. In essence, the control component can construct new training samples based on relevant information from the robot's current real-world environment, and use these new training samples to train the motion control model, thereby updating the model's network parameters. This allows the updated motion control model to adapt to the robot's real-world working environment.

[0112] In this embodiment, the robot's movement path and coverage area based on control commands are acquired. Using a reinforcement learning algorithm, the network parameters of the motion control model are updated based on temporal surround view images, state information, the robot's motion parameters, and the movement path and coverage area. This allows the updated motion control model to adapt to the robot's real-world working environment, including noise from the image and state acquisition components on the robot, variations in the surrounding geological conditions (such as differences in lawn geology), and terrain changes. Within the robot's current scenario, the control components can continuously acquire new training samples during multiple movements and update the motion control model's network parameters based on these new samples. This allows the motion control model to gradually approach the optimal coverage strategy for that scenario, resulting in higher robustness and accuracy.

[0113] In one embodiment, such as Figure 6 As shown, this relates to an implementation method based on reinforcement learning algorithms, which updates the network parameters of a motion control model according to a temporal loop image, state information, robot motion parameters, and movement path and coverage area. The steps of this implementation method include:

[0114] Step 600: Obtain the robot's historical movement path and the corresponding historical coverage area.

[0115] Historical movement paths refer to the paths the robot has already traversed in its current environment, i.e., the movement paths controlled by the robot in multiple control cycles prior to the current control cycle. Historical coverage areas are the coverage areas corresponding to historical movement paths, i.e., the areas covered by the robot during its movement along historical movement paths. The description of historical coverage areas can be found in the detailed description of coverage areas in the above embodiments, and will not be repeated here. Historical movement paths and corresponding historical coverage areas can be all paths the robot has already traversed in its current environment and their corresponding coverage areas, or they can be the robot's movement paths and corresponding coverage areas within a preset number of control cycles.

[0116] Historical movement paths and historical coverage areas can be stored in the control component. When it is necessary to optimize the motion control model, the robot's historical movement paths and historical coverage areas can be directly obtained.

[0117] Step 610: Construct new training samples based on the time-series loop image, state information, robot motion parameters, movement path and coverage area, as well as historical movement paths and historical coverage areas.

[0118] After the control component obtains the robot's historical movement path and historical coverage area, it can construct new training samples required for training the motion control model based on the robot's current relevant information, namely, the sequential view image, state information, robot motion parameters, movement path and coverage area, as well as the robot's historical relevant information, namely the historical movement path and historical coverage area.

[0119] In one embodiment, during the training of a network model based on a reinforcement learning algorithm, several parameters are needed: state, action, reward, and next state. Therefore, new training samples include the robot's current state, action, reward value, and next state. Based on this, as follows... Figure 7 As shown, this involves an implementation method for constructing new training samples based on a time-series loop view image, state information, robot motion parameters, movement path and coverage area, as well as historical movement paths and historical coverage areas. The steps of this implementation method include:

[0120] Step 700: Determine the current state of the robot based on the time-series loop image and state information.

[0121] Based on the temporal loop image of the robot's current environment and the robot's current state information, the control component can construct a new state in the training samples, i.e., the robot's current state. In essence, the control component can fuse the temporal loop image and state information to obtain the robot's current state.

[0122] In one specific embodiment, the control component encodes the temporal loop image into a temporal visual feature vector and the state information into a state feature vector. By fusing the temporal visual feature vector and the state feature vector, the current state vector of the robot can be obtained, which is the current state of the robot.

[0123] Step 710: Determine the robot's actions based on the robot's motion parameters.

[0124] Based on the obtained robot motion parameters, the control component can determine the specific actions the robot will perform, that is, convert the robot's motion parameters into physical actions, i.e., the robot's actions.

[0125] Step 720: Calculate the reward value based on the movement path and coverage area, as well as the historical movement path and historical coverage area, using a preset reward function.

[0126] The reward function can be pre-set and stored in the control component. After determining the historical movement path and historical coverage area, the control component can calculate the various parameters in the reward function based on these parameters. Substituting these parameters into the reward function yields the reward value. It can be understood that the reward function includes the repetition rate of the robot's movement path after moving according to the motion parameters output by the motion control model, determined by the current movement path and historical movement paths; and the overlap rate of the coverage area during movement after moving according to the motion parameters output by the motion control model, determined by the coverage area and historical coverage area. This embodiment does not limit the specific process of calculating the reward value based on the pre-set reward function, as long as the function is implemented.

[0127] Step 730: Determine the next state of the robot based on the state of the robot after it moves according to the control command.

[0128] After the robot moves according to control commands, the control component can acquire a time-series surround view image of the robot after movement through an image acquisition component on the robot, and acquire the robot's state information after movement through a state acquisition component on the robot; and determine the robot's next state based on the time-series surround view image and state information. For a description of determining the robot's next state based on the time-series surround view image and state information, please refer to the specific description of determining the robot's current state based on the time-series surround view image and state information in the above embodiment, which will not be repeated here.

[0129] Step 620: Update the network parameters of the motion control model based on the new training samples.

[0130] After receiving new training samples, the control component trains the motion control model based on these new samples to update the network parameters of the motion control network model. The specific process of updating the network parameters of the motion control model based on the new training samples can be found in the detailed description of the motion control model training process in the following embodiments, and will not be repeated here.

[0131] In one specific embodiment, the control component iteratively optimizes the motion control model by maximizing the reward value in the new training samples, thereby updating the network parameters of the motion control model and obtaining an optimized motion control model.

[0132] In an optional embodiment, iteratively optimizing the motion control model to maximize the reward value in the new training samples includes: constructing a loss function based on the reward value, performing backpropagation on the loss function, calculating the gradients of all trainable parameters of the motion control model, and adjusting the network parameters of the motion control model along the gradient direction using an optimizer.

[0133] In this embodiment, the robot's historical movement paths and corresponding historical coverage areas are obtained. New training samples are constructed based on the time-series loop image, state information, robot motion parameters, movement paths, coverage areas, and historical movement paths and coverage areas. The network parameters of the motion control model are then updated based on these new training samples. This direct use of relevant parameters from the robot's current environment to update the motion control model's network parameters makes the updated model more adaptable to the robot's real-world working environment. This improves the accuracy of subsequent robot motion parameters determined based on the updated motion control model, thereby enhancing the practicality of the robot control method.

[0134] In one embodiment, the reward function includes positive rewards and negative rewards. Positive rewards include newly added coverage areas, and negative rewards include at least one of total path length penalty, collision event penalty, boundary event penalty, and path overlap length penalty.

[0135] Positive rewards refer to the reward given when a robot, based on the motion parameters output by the motion control model, produces behavior that aligns with the task objective. Negative rewards, also known as penalties, refer to the reward given when a robot, based on the motion parameters output by the motion control model, produces behavior that deviates from the task objective, is meaningless, or even harmful. Positive rewards include newly added coverage areas. This can be understood as the robot's coverage area during the current control cycle being greater than its historical coverage area. If the robot's environment is rasterized, the newly added coverage area can be represented as the number of newly added grid cells. Total path length penalty is imposed when the total path traveled by the robot exceeds a reasonable threshold, constraining inefficient and meaningless long-distance movement. Collision event penalty is imposed when the robot collides with an obstacle during movement, constraining dangerous and non-compliant behavior. Boundary event penalty is imposed when the robot exceeds the pre-defined working boundary during movement, constraining behavior exceeding the boundary. Path overlap penalty is imposed when the robot's movement path overlaps with historical movement paths, constraining repetitive movement and path redundancy.

[0136] In an optional embodiment, weights can be set for positive and negative rewards in the reward function, and the reward value can be obtained by weighted summation of positive and negative rewards.

[0137] In this embodiment, the reward function may include penalties for adding coverage area, total path length, collision event, boundary, and path overlap length. This allows the motion control model to be optimized for the task objective without relying on manually set coverage patterns, thereby improving the applicability of the motion control model and thus improving the accuracy of robot control.

[0138] In one embodiment, such as Figure 8 As shown, this involves an implementation method for calculating a reward value based on a preset reward function, using the mobile path and coverage area, as well as historical mobile paths and historical coverage areas. The steps of this implementation method include:

[0139] Step 800: Determine the new coverage area based on the existing coverage area and historical coverage area.

[0140] The control component determines the new coverage area by comparing the robot's current coverage area with the historical coverage area. In other words, the control component can calculate the difference between the current coverage area and the historical coverage area, and determine the new coverage area based on the difference.

[0141] Step 810: Determine the total path length and path overlap length based on the movement path and historical movement paths.

[0142] The control component can calculate the total length of the robot's path based on the current movement path and historical movement paths. Specifically, the control component calculates the total path length by summing the current movement path and historical movement paths.

[0143] The control component determines the path overlap length by comparing the current movement path with the historical movement path. In other words, the control component can calculate the union of the current movement path and the historical movement path, and determine the path overlap length based on the union.

[0144] Step 820: Determine collision events and boundary events based on the movement path.

[0145] The control component determines whether collision and boundary events occur during the robot's movement based on its current path. Specifically, the control component determines whether a collision event has occurred by checking for sudden stops, abrupt changes in trajectory, or abnormal position reversals along the robot's current path. It determines whether a boundary event has occurred by checking whether the robot's path exceeds or approaches the boundary of the work area. The control component quantifies the determined collision and boundary events; for example, "0" indicates no collision and "1" indicates a collision. Similarly, "0" indicates no boundary event and "1" indicates a boundary event.

[0146] Step 830: Substitute the newly added coverage area, total path length, path overlap length, collision event, and boundary event into the reward function to obtain the reward value.

[0147] After determining the new coverage area, total path length, path overlap length, collision events, and boundary events, the control component substitutes these into a pre-set reward function to calculate the reward value.

[0148] In this embodiment, a new coverage area is determined based on the existing coverage area and the historical coverage area; the total path length and path overlap length are determined based on the movement path and the historical movement path; collision events and boundary events are determined based on the movement path; the new coverage area, total path length, path overlap length, collision events, and boundary events are substituted into the reward function to obtain a reward value. The reward value calculated in this way can simultaneously improve the coverage efficiency, path redundancy, and safety during the robot's movement, thereby improving the robustness and accuracy of the motion control model trained based on the reward value.

[0149] In some embodiments, the preset reward function is calculated by weighted summation of various indicators of positive and negative rewards. The control component assigns positive weight coefficients to positive rewards (such as newly added coverage area, i.e., the number of newly covered grids); and assigns corresponding negative weight coefficients (i.e., penalty coefficients) to negative rewards (including total path length, path overlap length, number or identifier of collision events, number or identifier of boundary events, etc.). Subsequently, the values ​​of each indicator in the reward function are weighted and summed with their corresponding weight coefficients to obtain the final reward value of the current action. Through the weighted summation reward mechanism with "coverage area" and "path repetition" as core indicators, the end-to-end network can directly perform joint optimization for the task objectives of maximizing coverage and minimizing repetition, instead of relying on manually preset fixed geometric paths or coverage patterns.

[0150] In one embodiment, such as Figure 9 As shown, one implementation method involves acquiring a temporal loop image of the robot's current environment. The steps of this implementation method include:

[0151] Step 900: Obtain multi-frame surround view images of the robot's current environment.

[0152] The image acquisition components installed on the robot can continuously acquire multiple images of the surrounding area of ​​the robot's current location, i.e., single-frame images from various perspectives of the robot. After obtaining the single-frame images from various perspectives, the control component stacks these single-frame images along the channel dimension to obtain a panoramic image. By continuously acquiring multiple images, a multi-frame panoramic image can be obtained.

[0153] In an optional embodiment, the robot is equipped with a front-facing camera, a rear-facing camera, a left-facing camera, and a right-facing camera. The four cameras capture single-frame images from four perspectives of the robot. By stacking the single-frame images from the four perspectives in the channel dimension, a panoramic image can be obtained.

[0154] Step 910: Stack the multiple frames of the loop image in the time dimension to obtain a time-series loop image.

[0155] After obtaining multiple frames of surround view images, the control component stacks the acquired multiple frames of surround view images in the time dimension to obtain a time-series surround view image.

[0156] In this embodiment, by acquiring multiple frames of surround view images of the robot's current environment and stacking these multiple frames of surround view images in the time dimension, a temporal surround view image is obtained. This method of determining the temporal surround view image is quick and easy to implement.

[0157] In one embodiment, another implementation of acquiring a temporal surround view image of the robot's current environment includes the following steps: acquiring multiple frames of images from various perspectives of the robot; stacking the multiple frames of images in the time dimension to obtain a temporal image from a single perspective; and stacking the temporal images from multiple perspectives in the channel dimension to obtain a temporal surround view image.

[0158] Understandably, the control component converts the acquired single-frame images from each viewpoint into standard three-dimensional tensors; based on the temporal information of the multi-frame images, it stacks the acquired multi-frame standard three-dimensional tensors in the time dimension to form a single-view four-dimensional temporal tensor; and it stacks the four-dimensional temporal tensors of each viewpoint in the channel dimension to obtain a temporal loop image.

[0159] The above embodiments provide two implementation methods for acquiring a time-series view of the robot's current environment. Users can choose according to their actual application, which can improve the practicality of the robot control method.

[0160] In one embodiment, such as Figure 10 As shown, the training process involving the motion control model includes:

[0161] Step 1001: Obtain training samples.

[0162] The control component includes an experience buffer pool, which stores time-series experience samples generated by the robot's interaction with the environment. Specifically, the experience buffer pool can include the robot's state-action-reward-next state. The time-series experience samples stored in the experience buffer pool can be generated by the robot's interaction with the environment in other scenarios, or by the robot's interaction with the environment in a simulation environment.

[0163] In one specific embodiment, for a lawnmower robot, various lawn scenarios are constructed in a simulation platform to simulate different lawn shapes and obstacle layouts, as well as the dynamics and sensor characteristics of the lawnmower robot. A random action strategy or preset simple geometric rules guide the lawnmower robot to perform multiple rounds of mowing tasks in the simulation environment. Temporal experience samples from these multiple rounds are collected and stored in an experience buffer pool to obtain training samples, thus completing the cold start of the initial network model.

[0164] When training the motion control model, the control components can obtain training samples from the experience buffer pool.

[0165] Step 1002: Train the initial network model based on the training samples to obtain the action control model; the initial network model includes a feature extraction network, a feature splicing network, a time series modeling network, an action output network, and a value estimation network; the feature extraction network, the feature splicing network, the time series modeling network, and the action output network are connected in sequence, and the time series modeling network is also connected to the value estimation network.

[0166] In some possible implementations, the initial network model is trained using an Actor-Critic-based deep reinforcement learning framework. The feature extraction network, feature concatenation network, time series modeling network, and action output network together constitute the Actor (policy network), which directly outputs the walking action parameters or action probability distribution in the current state; the value estimation network is the Critic (value network). During training, the high-dimensional state representation output by the time series modeling network is input into the value estimation network, which outputs a long-term reward estimate (i.e., state value) for the current state. Based on the state value output by the value estimation network and the reward value calculated using a pre-defined reward function, an advantage function is calculated to evaluate the advantage of the current action compared to the average policy, and this advantage function guides the gradient update of the Actor network parameters. By introducing a value estimation network that calculates the advantage function, the variance of the policy network gradient can be effectively reduced, thereby ensuring the stability and convergence speed of the end-to-end training process.

[0167] The structure of the initial network model is as follows Figure 11 As shown, the network includes a feature extraction network, a feature concatenation network, a time series modeling network, an action output network, and a value estimation network. The descriptions of the feature extraction network, feature concatenation network, time series modeling network, and action output network in the initial network model can be found in the detailed descriptions of the above embodiments, and will not be repeated here.

[0168] The input of the value estimation network is also connected to the output of the time series modeling network. The high-dimensional state representation is input into the value estimation network, which estimates the value of the current state. The output of the value estimation network is used for stable training and reward calculation. The process of using the reward value from the value estimation network's output for training can be found in the detailed description of updating the network parameters of the action control model in the above embodiments, and will not be repeated here.

[0169] After acquiring training samples, the control component trains the initial network model based on the training samples to obtain the action control model. The termination condition for training the initial network model can include achieving the target task, i.e., reaching a preset coverage range, reaching an iteration threshold, or reaching the task time limit. This embodiment does not limit the specific process of training the initial network model based on the training samples, as long as its functionality is achieved. The initial network model can be a network model initialized using a random strategy or a simple prior strategy.

[0170] In this embodiment, the initial network model is trained based on the acquired training samples to obtain the action control model. The initial network model includes a feature extraction network, a feature concatenation network, a time series modeling network, an action output network, and a value estimation network. During the training process, the initial network model can autonomously learn image features, enabling it to control the robot's movement more naturally without the need for manual threshold setting or segmentation. This gives the initial network model greater adaptability and self-optimization capabilities, thereby improving the applicability and robustness of the trained action control model.

[0171] In some possible implementations, the training of the motion control model employs a strategy of simulation pre-training combined with real-world fine-tuning. That is, before deploying the motion control model to a real robot, it is first cold-started and pre-trained in a simulation environment using multiple rounds of work (virtual lawn mowing) tasks. After deploying the pre-trained motion control model to the real-world work scenario, the network parameters of the pre-trained motion control model are fine-tuned online using a small learning rate, based on new training samples generated by the robot's continuous interaction with the environment. Using a small learning rate for fine-tuning allows the retrained motion control model to gradually adapt to sensor noise, real-world lawn geology differences, and subtle terrain variations in the real environment, while preventing catastrophic forgetting of the basic coverage and obstacle avoidance strategies learned in the simulation environment. Through continuous fine-tuning in the real environment, the network parameters of the retrained motion control model gradually approach the optimal coverage strategy for that real-world scenario.

[0172] Please see Figure 12One embodiment of this application provides a robot control method, the steps of which include:

[0173] Step 1201: Obtain the time-series surround view image of the robot's current environment and the robot's current state information; the time-series surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images;

[0174] Step 1202: Input the time-series loop view image and state information into the pre-trained motion control model to obtain the robot's motion parameters;

[0175] Step 1203: Generate control commands based on motion parameters, and control the robot to move according to the control commands;

[0176] Step 1204: Obtain the robot's historical movement path and corresponding historical coverage area, as well as the robot's movement path and coverage area based on control commands;

[0177] Step 1205: Construct new training samples based on the time-series loop image, state information, robot motion parameters, movement path and coverage area, as well as historical movement paths and historical coverage areas;

[0178] Step 1206: Update the network parameters of the motion control model based on the training samples.

[0179] In an optional embodiment, when the robot is a lawnmower robot, the lawnmower robot system structure block diagram is as follows: Figure 13 As shown, the system includes a surround-view perception module, an attitude / odometry module, a motion control model, an execution control module, and a reward calculation module. The description of the surround-view perception module can be found in the specific description of the image acquisition component in the above embodiments, and the description of the attitude / odometry module can be found in the description of the state acquisition component in the above embodiments. The execution control module controls the robot's movement according to control commands, and the reward calculation module calculates a reward function based on the recorded historical movement paths and corresponding historical coverage areas, as well as the movement path and coverage area. In this embodiment, integrating the surround-view perception module, attitude / odometry module, motion control model, execution control module, and reward calculation module onto the lawnmower robot enables efficient coverage tasks to be completed without relying on expensive sensors.

[0180] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0181] like Figure 1 As shown, one embodiment of this application provides a robot, including a robot body 10, an image acquisition component 11, a status acquisition component 12, and a control component 13. The image acquisition component 11, the status acquisition component 12, and the control component 13 are disposed on the robot body 10, and the control component 13 is connected to the image acquisition component 11 and the status acquisition component 12.

[0182] The image acquisition component 11 is used to acquire a time-series surround view image of the robot's current environment and transmit the time-series surround view image to the control component 13; the time-series surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images.

[0183] The status acquisition component 12 is used to acquire the current status information of the robot body and transmit the status information to the control component 13;

[0184] The control component 13 is used to input the time-series loop view image and state information into the pre-trained motion control model to obtain the motion parameters of the robot body; generate control commands based on the motion parameters; and control the movement of the robot body according to the control commands.

[0185] The descriptions of the robot body 10, image acquisition component 11, state acquisition component 12, and control component 13 can be found in the detailed descriptions of the above embodiments, and will not be repeated here. The description of the control component 13 inputting the time-series loop view image and state information into a pre-trained motion control model to obtain the motion parameters of the robot body; generating control commands based on the motion parameters; and controlling the movement of the robot body according to the control commands can be found in the detailed descriptions of the robot control methods in the above embodiments, and will not be repeated here.

[0186] In this embodiment, the control component directly outputs motion parameters based on the time-series surround view image and state information through the motion control model. This end-to-end mapping establishes a clear path from perception to control, eliminating the need for explicit mapping and path planning. The short information transmission chain and real-time response improve the accuracy of robot control. Furthermore, only simple image acquisition component 11 and state acquisition component 12 are needed to acquire the time-series surround view image and robot state information, eliminating the need for expensive sensors and high-precision maps, thus reducing hardware costs and the complexity of calibration and mapping processes.

[0187] Based on the same inventive concept, this application also provides a robot control device for implementing the robot control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more robot control device embodiments provided below can be found in the limitations of the robot control method described above, and will not be repeated here.

[0188] In one embodiment, such as Figure 14 As shown, a robot control device is provided, including: an acquisition module 21, an input module 22, and a control module 23, wherein:

[0189] The acquisition module 21 is used to acquire the time-series surround view image of the robot's current environment and the robot's current state information; the time-series surround view image includes multiple frames of surround view image of the robot's current environment and the time information of the multiple frames of surround view image;

[0190] Input module 22 is used to input the time-series loop view image and state information into the pre-trained motion control model to obtain the robot's motion parameters;

[0191] The control module 23 is used to generate control commands based on motion parameters and control the robot's movement according to the control commands.

[0192] In one embodiment, the input module 22 is specifically used to extract features from multiple frames of panoramic images through a feature extraction network to obtain image features and temporal features; the image features include semantic features representing boundary lines, obstacles, covered areas and uncovered areas; the image features, temporal features and state information are concatenated through a feature concatenation network to obtain an environmental feature representation; the environmental feature representation is fused through a time series modeling network to obtain a high-dimensional state representation; and the high-dimensional state representation is mapped to action parameters through an action output network.

[0193] In one embodiment, the robot control device further includes an update module. The update module is used to obtain the robot's movement path and coverage area based on control commands; and to update the network parameters of the motion control model based on a reinforcement learning algorithm, according to the temporal loop image, state information, robot motion parameters, and movement path and coverage area.

[0194] In one embodiment, the update module includes an acquisition unit, a construction unit, and an update unit. The acquisition unit is used to acquire the robot's historical movement paths and corresponding historical coverage areas; the construction unit is used to construct new training samples based on the time-series loop image, state information, robot motion parameters, movement paths and coverage areas, and historical movement paths and historical coverage areas; the update unit is used to update the network parameters of the motion control model based on the new training samples.

[0195] In one embodiment, the new training sample includes the current state, action, reward value, and next state. The construction unit is specifically used to determine the robot's current state based on the time-series loop image and state information; determine the robot's action based on the robot's action parameters; calculate the reward value based on a preset reward function based on the movement path and coverage area, as well as historical movement paths and historical coverage areas; and determine the robot's next state based on the state of the robot after moving according to control commands.

[0196] In one embodiment, the reward function includes positive rewards and negative rewards. Positive rewards include newly added coverage areas, and negative rewards include at least one of total path length penalty, collision event penalty, boundary event penalty, and path overlap length penalty.

[0197] In one embodiment, the acquisition module 21 is specifically used to acquire multi-frame surround view images of the robot's current environment; and stack the multi-frame surround view images in the time dimension to obtain a time-series surround view image.

[0198] In one embodiment, the robot control device further includes a training module. The training module is used to acquire training samples; train an initial network model based on the training samples to obtain an action control model; the initial network model includes a feature extraction network, a feature splicing network, a time series modeling network, an action output network, and a value estimation network; the feature extraction network, the feature splicing network, the time series modeling network, and the action output network are connected in sequence, and the time series modeling network is also connected to the value estimation network.

[0199] Each module in the aforementioned robot control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0200] In one embodiment, a computer device is provided, which may be a control component, and its internal structure diagram may be as follows: Figure 15 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a robot control method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0201] Those skilled in the art will understand that Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0202] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0203] Acquire a temporal surround view image of the robot's current environment, as well as the robot's current state information; the temporal surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images;

[0204] The time-series loop image and state information are input into a pre-trained motion control model to obtain the robot's motion parameters;

[0205] Control commands are generated based on motion parameters, and the robot is moved according to these commands.

[0206] In one embodiment, when the processor executes the computer program, it further performs the following steps: extracting features from multiple frames of panoramic images using a feature extraction network to obtain image features and temporal features; the image features include semantic features representing boundary lines, obstacles, covered areas, and uncovered areas; concatenating the image features, temporal features, and state information using a feature concatenation network to obtain an environmental feature representation; fusing the environmental feature representation with a time series modeling network to obtain a high-dimensional state representation; and mapping the high-dimensional state representation to action parameters using an action output network.

[0207] In one embodiment, when the processor executes the computer program, it also performs the following steps: obtaining the robot's movement path and coverage area based on control instructions; and updating the network parameters of the motion control model based on a reinforcement learning algorithm, according to the temporal loop image, state information, robot motion parameters, and movement path and coverage area.

[0208] In one embodiment, when the processor executes the computer program, it also performs the following steps: obtaining the robot's historical movement path and the corresponding historical coverage area; constructing new training samples based on the time-series loop image, state information, robot motion parameters, movement path and coverage area, and historical movement path and historical coverage area; and updating the network parameters of the motion control model based on the new training samples.

[0209] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the current state of the robot based on the time-series loop image and state information; determining the robot's action based on the robot's motion parameters; calculating a reward value based on a preset reward function based on the movement path and coverage area, as well as historical movement paths and historical coverage areas; and determining the robot's next state based on the state of the robot after moving according to control instructions.

[0210] In one embodiment, the reward function includes positive rewards and negative rewards. Positive rewards include newly added coverage areas, and negative rewards include at least one of total path length penalty, collision event penalty, boundary penalty, and path overlap length penalty.

[0211] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining a new coverage area based on the existing coverage area and historical coverage areas; determining the total path length and path overlap length based on the movement path and historical movement paths; determining collision events and boundary events based on the movement path; and substituting the new coverage area, total path length, path overlap length, collision events, and boundary events into a reward function to obtain a reward value.

[0212] In one embodiment, when the processor executes the computer program, it also performs the following steps: acquiring a multi-frame surround view image of the robot's current environment; stacking the multi-frame surround view images in the time dimension to obtain a time-series surround view image.

[0213] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring training samples; training an initial network model based on the training samples to obtain an action control model; the initial network model includes a feature extraction network, a feature splicing network, a time series modeling network, an action output network, and a value estimation network; the feature extraction network, the feature splicing network, the time series modeling network, and the action output network are connected in sequence, and the time series modeling network is also connected to the value estimation network.

[0214] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0215] Acquire a temporal surround view image of the robot's current environment, as well as the robot's current state information; the temporal surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images;

[0216] The time-series loop image and state information are input into a pre-trained motion control model to obtain the robot's motion parameters;

[0217] Control commands are generated based on motion parameters, and the robot is moved according to these commands.

[0218] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: extracting features from multiple frames of panoramic images using a feature extraction network to obtain image features and temporal features; the image features include semantic features representing boundary lines, obstacles, covered areas, and uncovered areas; concatenating the image features, temporal features, and state information using a feature concatenation network to obtain an environmental feature representation; fusing the environmental feature representation with a time series modeling network to obtain a high-dimensional state representation; and mapping the high-dimensional state representation to action parameters using an action output network.

[0219] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the robot's movement path and coverage area based on control commands; and updating the network parameters of the motion control model based on a reinforcement learning algorithm, according to the temporal loop image, state information, robot motion parameters, and movement path and coverage area.

[0220] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the robot's historical movement path and the corresponding historical coverage area; constructing new training samples based on the time-series loop image, state information, robot motion parameters, movement path and coverage area, and historical movement path and historical coverage area; and updating the network parameters of the motion control model based on the new training samples.

[0221] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the current state of the robot based on the time-series loop image and state information; determining the robot's action based on the robot's motion parameters; calculating a reward value based on a preset reward function based on the movement path and coverage area, as well as historical movement paths and historical coverage areas; and determining the robot's next state based on the state of the robot after moving according to control instructions.

[0222] In one embodiment, the reward function includes positive rewards and negative rewards. Positive rewards include newly added coverage areas, and negative rewards include at least one of total path length penalty, collision event penalty, boundary penalty, and path overlap length penalty.

[0223] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining a new coverage area based on the existing coverage area and historical coverage areas; determining the total path length and path overlap length based on the movement path and historical movement paths; determining collision events and boundary events based on the movement path; and substituting the new coverage area, total path length, path overlap length, collision events, and boundary events into a reward function to obtain a reward value.

[0224] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: acquiring a multi-frame surround view image of the robot's current environment; stacking the multi-frame surround view images in the time dimension to obtain a temporal surround view image.

[0225] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring training samples; training an initial network model based on the training samples to obtain an action control model; the initial network model includes a feature extraction network, a feature splicing network, a time series modeling network, an action output network, and a value estimation network; the feature extraction network, the feature splicing network, the time series modeling network, and the action output network are connected in sequence, and the time series modeling network is also connected to the value estimation network.

[0226] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0227] Acquire a temporal surround view image of the robot's current environment, as well as the robot's current state information; the temporal surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images;

[0228] The time-series loop image and state information are input into a pre-trained motion control model to obtain the robot's motion parameters;

[0229] Control commands are generated based on motion parameters, and the robot is moved according to these commands.

[0230] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: extracting features from multiple frames of panoramic images using a feature extraction network to obtain image features and temporal features; the image features include semantic features representing boundary lines, obstacles, covered areas, and uncovered areas; concatenating the image features, temporal features, and state information using a feature concatenation network to obtain an environmental feature representation; fusing the environmental feature representation with a time series modeling network to obtain a high-dimensional state representation; and mapping the high-dimensional state representation to action parameters using an action output network.

[0231] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the robot's movement path and coverage area based on control commands; and updating the network parameters of the motion control model based on a reinforcement learning algorithm, according to the temporal loop image, state information, robot motion parameters, and movement path and coverage area.

[0232] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the robot's historical movement path and the corresponding historical coverage area; constructing new training samples based on the time-series loop image, state information, robot motion parameters, movement path and coverage area, and historical movement path and historical coverage area; and updating the network parameters of the motion control model based on the new training samples.

[0233] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the current state of the robot based on the time-series loop image and state information; determining the robot's action based on the robot's motion parameters; calculating a reward value based on a preset reward function based on the movement path and coverage area, as well as historical movement paths and historical coverage areas; and determining the robot's next state based on the state of the robot after moving according to control instructions.

[0234] In one embodiment, the reward function includes positive rewards and negative rewards. Positive rewards include newly added coverage areas, and negative rewards include at least one of total path length penalty, collision event penalty, boundary penalty, and path overlap length penalty.

[0235] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining a new coverage area based on the existing coverage area and historical coverage areas; determining the total path length and path overlap length based on the movement path and historical movement paths; determining collision events and boundary events based on the movement path; and substituting the new coverage area, total path length, path overlap length, collision events, and boundary events into a reward function to obtain a reward value.

[0236] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: acquiring a multi-frame surround view image of the robot's current environment; stacking the multi-frame surround view images in the time dimension to obtain a temporal surround view image.

[0237] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring training samples; training an initial network model based on the training samples to obtain an action control model; the initial network model includes a feature extraction network, a feature splicing network, a time series modeling network, an action output network, and a value estimation network; the feature extraction network, the feature splicing network, the time series modeling network, and the action output network are connected in sequence, and the time series modeling network is also connected to the value estimation network.

[0238] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0239] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0240] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A robot control method, characterized in that, The method includes: The system acquires a time-series surround view image of the robot's current environment, as well as the robot's current state information; the time-series surround view image includes multiple frames of surround view images of the robot's current environment and the time information of the multiple frames of surround view images; The time-series loop image and the state information are input into a pre-trained motion control model to obtain the robot's motion parameters; Control commands are generated based on the motion parameters, and the robot is moved according to the control commands.

2. The method according to claim 1, characterized in that, The motion control model includes a feature extraction network, a feature concatenation network, a time series modeling network, and a motion output network connected in sequence. The step of inputting the time series loop image and the state information into the pre-trained motion control model to obtain the robot's motion parameters includes: The feature extraction network is used to extract features from the multi-frame panoramic images to obtain image features and temporal features; the image features include semantic features representing boundary lines, obstacles, covered areas and uncovered areas; The image features, the temporal features, and the state information are concatenated using the feature concatenation network to obtain an environmental feature representation. The environmental feature representation is fused using the time series modeling network to obtain a high-dimensional state representation. The high-dimensional state representation is mapped to the action parameters through the action output network.

3. The method according to claim 1, characterized in that, The method further includes: Obtain the robot's movement path and coverage area based on the control commands; Based on reinforcement learning algorithms, the network parameters of the motion control model are updated according to the temporal loop image, the state information, the robot's motion parameters, the movement path, and the coverage area.

4. The method according to claim 3, characterized in that, The method of updating the network parameters of the motion control model based on the reinforcement learning algorithm, according to the temporal loop image, the state information, the robot's motion parameters, the movement path, and the coverage area, includes: Obtain the robot's historical movement path and the corresponding historical coverage area; New training samples are constructed based on the time-series loop image, the state information, the robot's motion parameters, the movement path and the coverage area, as well as the historical movement path and historical coverage area. The network parameters of the motion control model are updated based on the new training samples.

5. The method according to claim 4, characterized in that, The new training samples include the current state, action, reward value, and next state. The construction of new training samples based on the time-series loop image, the state information, the robot's action parameters, the movement path and the coverage area, and the historical movement path and historical coverage area includes: The current state of the robot is determined based on the time-series loop image and the state information; The robot's actions are determined based on its motion parameters; The reward value is calculated based on the movement path and coverage area, as well as the historical movement path and historical coverage area, using a preset reward function. The next state of the robot is determined based on the state of the robot after it moves according to the control command.

6. The method according to claim 5, characterized in that, The reward function includes positive rewards and negative rewards. The positive rewards include newly added coverage areas, and the negative rewards include at least one of the following: total path length penalty, collision event penalty, boundary event penalty, and path overlap length penalty.

7. The method according to claim 6, characterized in that, The step of calculating a reward value based on a preset reward function, according to the movement path and coverage area, and the historical movement path and historical coverage area, includes: The newly added coverage area is determined based on the existing coverage area and the historical coverage area. Based on the movement path and the historical movement path, determine the total path length and the path overlap length; Collision events and boundary events are determined based on the movement path; The reward value is obtained by substituting the newly added coverage area, the total path length, the path overlap length, the collision event, and the boundary event into the reward function.

8. The method according to claim 1, characterized in that, The acquisition of the time-series surround view image of the robot's current environment includes: Obtain multi-frame surround view images of the robot's current environment; The multi-frame loop image is stacked in the time dimension to obtain the time-series loop image.

9. The method according to any one of claims 1-8, characterized in that, The training process of the motion control model includes: Obtain training samples; The initial network model is trained based on the training samples to obtain the action control model; the initial network model includes a feature extraction network, a feature concatenation network, a time series modeling network, an action output network, and a value estimation network; the feature extraction network, the feature concatenation network, the time series modeling network, and the action output network are connected in sequence, and the time series modeling network is also connected to the value estimation network.

10. A robot, characterized in that, It includes a robot body, an image acquisition component, a status acquisition component, and a control component. The image acquisition component, the status acquisition component, and the control component are disposed on the robot body, and the control component is connected to the image acquisition component and the status acquisition component. The image acquisition component is used to acquire a time-series surround view image of the environment in which the robot body is currently located, and transmit the time-series surround view image to the control component; the time-series surround view image includes multiple frames of surround view images of the environment in which the robot is currently located and the time information of the multiple frames of surround view images. The status acquisition component is used to acquire the current status information of the robot body and transmit the status information to the control component; The control component is used to input the time-series loop image and the state information into a pre-trained motion control model to obtain the motion parameters of the robot body; generate control commands based on the motion parameters; and control the robot body to move according to the control commands.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images