Robot navigation processing method and device, terminal equipment and program product

Abstract

The invention is suitable for the technical field of machine navigation, and provides a robot navigation processing method and device, terminal equipment and a program product, and the method comprises the following steps: obtaining current state data of a robot and current position relation data between the robot and a target navigation position; a target navigation model is adopted for processing according to the current state data and the current position relation data, target action data of the robot is determined, and the target navigation model is obtained by continuously updating an initial navigation model according to training state data, training position relation data and training track information; and controlling the robot to execute the target action data, so that the robot travels to the target navigation position. According to the method, the navigation process of the robot can be simplified while the burden of computing resources is reduced, and the navigation accuracy is not influenced, so that the navigation requirements of the robot in various scenes are met.

Description

Technical Field

[0001] The present application belongs to the field of machine navigation technology, and in particular relates to a robot navigation processing method, apparatus, terminal equipment and program product. Background Art

[0002] In intelligent navigation systems, it is crucial to implement efficient, robust and secure navigation algorithms, which have important application value in many fields such as industrial automation, logistics and transportation, and agricultural automation.

[0003] In recent years, existing robot navigation processing methods are usually based on deep reinforcement learning (DRL) for robot navigation. The DRL method can automatically discover effective navigation strategies from data through interactive learning with the environment, and has good adaptability and robustness.

[0004] However, when dealing with complex scenarios and unforeseen situations, existing robot navigation processing methods usually require a large amount of training data and computing resources for training, making it difficult for resource-constrained devices to achieve fast navigation requirements, seriously affecting the accuracy of navigation processing. Summary of the Invention

[0005] The embodiments of the present application provide a robot navigation processing method, apparatus, terminal device, and program product, which can simplify the navigation process while reducing the burden on computing resources, thereby achieving rapid navigation requirements in complex scenarios and unforeseen situations, while improving the accuracy of navigation processing.

[0006] In a first aspect, an embodiment of the present application provides a robot navigation processing method, comprising:

[0007] Obtain the current state data of the robot and the current position relationship data between the robot and the target navigation position;

[0008] The target navigation model is used to process the current state data and the current position relationship data to determine the target motion data of the robot. The target navigation model is obtained by continuously updating the initial navigation model based on the training state data, the training position relationship data and the training trajectory information.

[0009] Control the robot to execute the target action data so that the robot can move to the target navigation position.

[0010] In a possible implementation of the first aspect, before obtaining the current state data of the robot and the current position relationship data between the robot and the target navigation position, the method further includes:

[0011] Continuously determining training trajectory information of the robot, wherein the training trajectory information is obtained by performing navigation processing on training state data and training position relationship data according to an initial navigation model, the initial navigation model including at least a long short-term memory network module;

[0012] The initial navigation model is updated according to the training state data, the training position relationship data and the training trajectory information to obtain an updated navigation model;

[0013] The updated navigation model is used as the initial navigation model, and the step of continuously determining the training trajectory information of the robot is returned to be executed until the updated navigation model is updated a specified number of times or the updated navigation model reaches convergence;

[0014] The updated navigation model that reaches a specified number of times or reaches convergence is determined as the target navigation model.

[0015] In a possible implementation of the first aspect, continuously determining training trajectory information of the robot includes:

[0016] Obtaining training state data of the robot and training position relationship data between the robot and the training navigation position;

[0017] Using the initial navigation model to perform navigation processing on the training state data and the training position relationship data to obtain training trajectory information of the robot, where the training trajectory information at least includes training action data;

[0018] The robot is controlled to execute the training action data, and returns to execute the step of obtaining the training state data of the robot and the training position relationship data between the robot and the training navigation position until the obtained training trajectory information reaches a preset number.

[0019] In a possible implementation of the first aspect, the training trajectory information further includes a state-value function, reward data, and trajectory state data;

[0020] The initial navigation model is used to perform navigation processing on the training state data and training position relationship data to obtain the robot's training trajectory information, including:

[0021] Input the training state data and training position relationship data into the initial navigation model to obtain the robot's navigation strategy data and the robot's state value function at the current moment;

[0022] Sampling the navigation strategy data to obtain the robot's training motion data;

[0023] The reward function is used to calculate the training action data to obtain the reward data of the training action data;

[0024] Based on the training action data, it is determined whether the current navigation trajectory has ended and the trajectory status data is obtained.

[0025] In a possible implementation of the first aspect, the training state data includes training radar data and training speed data. Inputting the training state data and training position relationship data into an initial navigation model to obtain navigation strategy data for the robot includes:

[0026] Using the long short-term memory network module in the initial navigation model, the training radar data and the historical radar data are encoded to obtain first encoded data, wherein the training radar data and the historical radar data are radar data of the robot at different timestamps;

[0027] Encoding the first coded data using the first fully connected module in the initial navigation model to obtain second coded data;

[0028] Encoding the first encoded data using the second fully connected module in the initial navigation model to obtain third encoded data;

[0029] Using the third fully connected module in the initial navigation model, the training speed data and the training position relationship data are encoded to obtain fourth encoded data;

[0030] The attention mechanism is used to integrate the second encoded data, the third encoded data, and the fourth encoded data to obtain the fifth encoded data;

[0031] The first coded data and the fifth coded data are fused to obtain navigation strategy data of the robot.

[0032] In a possible implementation of the first aspect, the training radar data is collected by a single-line laser radar sensor in the robot, and the training speed data is collected by an odometer in the robot.

[0033] In a possible implementation manner of the first aspect, the reward function includes at least a distance reward function and an obstacle reward function, and the obstacle reward function is related to the speed and orientation angle of the robot.

[0034] In a second aspect, an embodiment of the present application provides a robot navigation processing device, comprising:

[0035] An acquisition module is used to obtain the current state data of the robot and the current position relationship data between the robot and the target navigation position;

[0036] A navigation processing module is used to determine the robot's target motion data by processing the current state data and the current position relationship data using a target navigation model, wherein the target navigation model is obtained by continuously updating the initial navigation model based on the training state data, the training position relationship data, and the training trajectory information;

[0037] The control module is used to control the robot to execute the target action data so that the robot can travel to the target navigation position.

[0038] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods of the first aspect when executing the computer program.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method of any one of the first aspects.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the methods in the first aspect above.

[0041] The embodiments of the present application provide a robot navigation processing method, apparatus, terminal device, and program product, the method comprising: obtaining the current state data of the robot, and the current position relationship data between the robot and the target navigation position; using a target navigation model to process the current state data and the current position relationship data to determine the target motion data of the robot, wherein the target navigation model is obtained by continuously updating the initial navigation model based on the training state data, the training position relationship data, and the training trajectory information; and controlling the robot to execute the target motion data so that the robot travels to the target navigation position. Utilizing the above technical solution, by using the target navigation model to process the current state data and the current position relationship data to directly determine the target motion data of the robot, the navigation process can be simplified while reducing the burden on computing resources, thereby achieving rapid navigation requirements in complex scenarios and unforeseen situations, while improving the accuracy of navigation processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is a flowchart of a robot navigation processing method provided by an embodiment of the present application;

[0044] Figure 2 is a flowchart of a robot navigation processing method provided by another embodiment of the present application;

[0045] Figure 3 This is a flowchart of determining a target navigation model provided by an embodiment of the present application;

[0046] Figure 4 This is a structural block diagram of a robot navigation processing device provided by an embodiment of the present application;

[0047] Figure 5 This is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0049] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0050] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0051] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0052] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0054] The robot navigation processing method provided in the embodiments of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0055] It can be argued that implementing efficient, robust, and secure navigation algorithms is crucial in multi-agent systems, with significant application value in numerous fields, including industrial automation, logistics, and agricultural automation. For example, in industrial automation, multiple robots need to work collaboratively to complete complex assembly and handling tasks. This requires them to autonomously plan paths, avoid obstacles, and avoid collisions, ensuring smooth production processes. In logistics, multiple unmanned vehicles or drones need to efficiently transport cargo in busy traffic environments. This requires them to possess excellent navigation capabilities, enabling them to make fast and accurate decisions in dynamically changing environments, improve transportation efficiency, and reduce accident risks. In agricultural automation, multiple agricultural robots need to perform tasks such as sowing, fertilizing, and harvesting across vast fields. This requires them to accurately identify obstacles and crop rows, rationally plan paths, and improve operational efficiency and quality.

[0056] However, existing multi-agent navigation technologies still face several limitations and challenges. Traditional navigation methods are primarily categorized as centralized and distributed. Centralized methods rely on a central server to calculate the trajectory of each agent. While this approach can perform path planning from a global perspective, the computational cost rises dramatically as the number of agents increases, placing high demands on computing resources. Centralized methods are also highly dependent on communication networks; if problems arise in the communication network, the performance of the entire system is severely impacted. Furthermore, when faced with large-scale scenarios, centralized methods are prone to response delays due to the need to process large amounts of data and complex computational tasks, resulting in reduced navigation efficiency.

[0057] Distributed approaches empower each agent with decision-making capabilities, enabling autonomous navigation based on its own sensory information and goals. This approach reduces computational costs and reliance on communication networks, improving system scalability and robustness. However, distributed approaches also present some challenges. First, they rely on perfect perception of the environment and the states of other agents. However, in practical applications, agents' perception capabilities are often limited by factors such as sensor performance and environmental noise, making it difficult to obtain accurate and comprehensive environmental information. Second, when dealing with complex scenarios and unforeseen situations, distributed approaches are prone to conflicts and collisions between agents, as each agent makes independent decisions and lacks an effective coordination mechanism. Furthermore, distributed approaches present difficulties in parameter adjustment. Different scenarios and task requirements require different parameter settings, and manual parameter adjustment is not only time-consuming and labor-intensive, but also difficult to find the optimal parameter combination.

[0058] In recent years, methods based on deep reinforcement learning (DRL) have been proposed for robot navigation and have made considerable progress. DRL methods, through interactive learning with the environment, can automatically discover effective navigation strategies from data, exhibiting good adaptability and robustness. However, existing DRL methods still have some shortcomings when dealing with complex scenarios and unforeseen situations. For example, some methods are overly complex in designing reward functions, causing the agent to learn suboptimal actions and fail to achieve optimal navigation results in all scenarios. Furthermore, DRL methods typically require large amounts of data and computing resources for training, which is difficult to implement on resource-constrained devices. Furthermore, successfully migrating DRL policies trained in simulated environments to the real world is a key research issue. There are certain differences between simulated environments and the real world, including sensor noise and the behavioral patterns of dynamic obstacles. These differences can lead to reduced performance of policies in the real world.

[0059] In summary, existing multi-agent navigation technology still faces many challenges in terms of efficiency, robustness, safety, and simulation and reality migration. A new robot navigation processing method is urgently needed to overcome the above problems and meet the navigation needs of multi-agent systems in complex environments.

[0060] Based on this, a robot navigation processing method provided in an embodiment of the present application is intended to solve a series of technical problems faced by multi-agent systems when navigating in complex environments. First, in a partially observable Markov decision process (POMDP), the agent cannot directly obtain the global state, resulting in information loss, making it difficult to make accurate decisions on the navigation strategy. For example, the agent may not be able to accurately judge the position and movement trend of obstacles in a narrow passage, thereby making incorrect obstacle avoidance actions, resulting in collisions or congestion. Secondly, there are many differences between the simulation environment and the real world, such as sensor noise, behavior patterns of dynamic obstacles, etc. These differences cause the strategy trained in the simulation environment to degrade in actual application performance and cannot effectively cope with complex situations in the real world. In addition, in some specific scenarios, such as when the agent needs to pass through a narrow passage or multiple agents are traveling in opposite directions, the existing robot navigation processing method will cause the agent to choose suboptimal actions due to reasons such as unreasonable reward function design. For example, when passing through a narrow passage, the agent may choose to detour instead of going directly through the passage for fear of collision, which increases navigation time and path length; when multiple agents are driving in reverse, the agent may pay too much attention to the nearest obstacle and ignore potential threats farther away, resulting in unsafe navigation behavior.

[0061] Figure 1 This is a flow chart of a robot navigation processing method provided by an embodiment of the present application. As an example and not a limitation, this method can be applied to terminal devices, such as Figure 1 As shown, the method includes:

[0062] S101. Acquire the current state data of the robot and the current position relationship data between the robot and the target navigation position.

[0063] The current state data can be used to characterize the current state of the robot. For example, the current state data can include current radar data and current speed data. The current radar data can refer to various radar data about the robot collected by a single-line lidar sensor, such as the robot's orientation, height and other data; the current speed data can refer to data related to the robot's speed, such as can be used to characterize the robot's speed and / or acceleration.

[0064] The target navigation position is the final position to which the robot will navigate. The current position relationship data can be used to represent the positional relationship between the robot and the target navigation position, such as the straight-line distance and orientation difference between the robot and the target navigation position.

[0065] S102: Using a target navigation model to process the current state data and the current position relationship data, determine the target motion data of the robot.

[0066] Among them, the target navigation model is obtained by continuously updating the initial navigation model based on the training state data, training position relationship data and training trajectory information.

[0067] The target navigation model can be understood as a pre-trained navigation model, which is configured inside the robot and can be used to directly output the robot's target motion data. The target navigation model can be obtained by continuously updating the initial navigation model based on the training state data, training position relationship data and training trajectory information. The training state data, training position relationship data and training trajectory information can be pre-collected training sets. The training state data is the state data of the robot collected during the training process. The training position relationship data is the position relationship data between the robot and the training navigation position collected during the training process. The training trajectory information is the trajectory information processed and output by the robot according to the initial navigation model during the training process. The specific content is not limited. The initial navigation model can be considered as a navigation model obtained by preliminary training of the original neural network model. It needs to be continuously updated to obtain the final target navigation model. The specific process of updating the initial navigation model is not described here.

[0068] During the actual navigation process, after obtaining the current state data and the current position relationship data, the built-in target navigation model can be used to process the current state data and the current position relationship data, and directly output the target action data of the robot, so that the robot can be controlled to execute the target action data to make the robot drive to the target navigation position. For example, this embodiment can make the robot gradually drive to the target navigation position by periodically repeating steps S101-S102. For example, when the robot is far away from the target navigation position, the current state data and the current position relationship data can be obtained in real time to process the corresponding target action data, so that the robot can slowly navigate to the target navigation position; alternatively, this embodiment can control the robot to drive to the target navigation position according to the obtained target action data by processing the current state data and the current position relationship data once when the robot is close to the target navigation position or there are no other obstacles.

[0069] S103: Control the robot to execute the target action data so that the robot travels to the target navigation position.

[0070] This embodiment provides a robot navigation processing method that obtains the robot's current state data and current position relationship data between the robot and a target navigation location; uses a target navigation model to process the current state data and current position relationship data to determine the robot's target motion data, wherein the target navigation model is obtained by continuously updating an initial navigation model based on training state data, training position relationship data, and training trajectory information; and controls the robot to execute the target motion data so that the robot travels to the target navigation location. This method, by directly determining the robot's target motion data based on the current state data and current position relationship data using the target navigation model, can reduce the burden on computing resources, simplify the robot's navigation process, and maintain navigation accuracy, thereby meeting the robot's navigation needs in various scenarios.

[0071] Figure 2 This is a flow chart of a robot navigation processing method provided by another embodiment of the present application. This embodiment further optimizes the situation before obtaining the current state data of the robot and the current position relationship data between the robot and the target navigation position as follows: continuously determining the training trajectory information of the robot, wherein the training trajectory information is obtained by navigation processing the training state data and the training position relationship data according to the initial navigation model, and the initial navigation model at least includes a long short-term memory network module; updating the initial navigation model according to the training state data, the training position relationship data and the training trajectory information to obtain an updated navigation model; using the updated navigation model as the initial navigation model, and returning to execute the step of continuously determining the training trajectory information of the robot until the number of updates of the updated navigation model reaches a specified number, or the updated navigation model reaches convergence; determining the updated navigation model that reaches the specified number or reaches convergence as the target navigation model. Figure 2 As shown, the method includes:

[0072] S201. Continue to determine the training trajectory information of the robot.

[0073] The training trajectory information is obtained by performing navigation processing on the training state data and the training position relationship data according to the initial navigation model, and the initial navigation model at least includes a long short-term memory network module.

[0074] S202: Update the initial navigation model according to the training state data, the training position relationship data, and the training trajectory information to obtain an updated navigation model.

[0075] S203: Use the updated navigation model as the initial navigation model, and return to the step of continuously determining the training trajectory information of the robot until the updated navigation model is updated a specified number of times or the updated navigation model reaches convergence.

[0076] S204: Determine the updated navigation model that has reached a specified number of times or has reached convergence as the target navigation model.

[0077] This embodiment can firstly continuously determine the training trajectory information of the robot, such as collecting the training trajectory information obtained by the robot navigating the training state data and the training position relationship data according to the initial navigation model within a preset time period, or continuously collecting the number of training trajectory information as the end node for determining the training trajectory information, or alternatively, determining the end node of the training trajectory information in other ways.

[0078] As an example, continuously determining the training trajectory information of the robot may include: obtaining training state data of the robot and training position relationship data between the robot and a training navigation position; performing navigation processing on the training state data and the training position relationship data using an initial navigation model to obtain training trajectory information of the robot, wherein the training trajectory information includes at least training action data; controlling the robot to execute the training action data and returning to the step of obtaining the training state data of the robot and the training position relationship data between the robot and the training navigation position until a preset number of training trajectory information is obtained. The preset number may be a pre-set number determined by relevant personnel based on experience.

[0079] In this embodiment, the means of determining the training trajectory information of the robot are not limited. For example, it can be differentiated according to the specific content of the training trajectory information, or it can be determined according to the specific structure of the initial navigation model. For example, the initial navigation model can include a long short-term memory network module (Long Short-Term Memory, LSTM), or it can include other processing modules, etc. For example, this embodiment can deal with the problem of information loss of partial observability by adopting LSTM. LSTM can effectively capture long-term dependencies in time series data, which is particularly important for maintaining the memory of historical information in multi-agent interaction. Specifically, this embodiment can directly use the LSTM layer to encode the training state data (such as radar data). By stacking several consecutive frames of radar observations, the LSTM module can capture the historical observation memory and integrate the continuous observation mode to obtain training trajectory information with higher accuracy, thereby improving the navigation performance of the multi-agent system in a complex dynamic environment.

[0080] The initial navigation model can then be updated to obtain an updated navigation model based on the different training state data and training position relationship data, as well as the training trajectory information continuously determined based on the different training state data and training position relationship data. Furthermore, this embodiment can also ensure the accuracy of the target navigation model by updating the initial navigation model multiple times. For example, the updated navigation model can be used as the initial navigation model again, and steps S201-S202 can be repeated until the initial navigation model is updated a specified number of times or the updated navigation model converges. At this point, the update of the initial navigation model is considered complete, and the updated navigation model that has reached the specified number of times or converged can be determined as the target navigation model.

[0081] S205: Acquire the current state data of the robot and the current position relationship data between the robot and the target navigation position.

[0082] S206: Using the target navigation model to process the current state data and the current position relationship data, determine the target motion data of the robot.

[0083] S207: Control the robot to execute the target action data so that the robot travels to the target navigation position.

[0084] This embodiment provides a robot navigation processing method, which continuously updates the initial navigation model based on continuously determined training state data, training position relationship data, and training trajectory information. At the same time, the initial navigation model includes at least a long-short-term memory network module, which can effectively utilize the robot's historical observation information, alleviate the problem of information loss, and enable the robot to better understand the dynamic changes of the environment and its own motion state, make more accurate and reliable navigation decisions in complex environments, thereby enhancing the robustness and safety of navigation.

[0085] In some embodiments, the training trajectory information further includes a state-value function, reward data, and trajectory state data;

[0086] The initial navigation model is used to perform navigation processing on the training state data and training position relationship data to obtain the robot's training trajectory information, including:

[0087] Input the training state data and training position relationship data into the initial navigation model to obtain the robot's navigation strategy data and the robot's state value function at the current moment;

[0088] Sampling the navigation strategy data to obtain the robot's training motion data;

[0089] The reward function is used to calculate the training action data to obtain the reward data of the training action data;

[0090] Based on the training action data, it is determined whether the current navigation trajectory has ended and the trajectory status data is obtained.

[0091] In a specific embodiment, the training trajectory information may include, in addition to the training action data, a state value function, reward data, and trajectory state data, wherein the state value function may be understood as the value function of the robot's current state, the reward data is used to characterize the reward of the training action data, and the trajectory state data may be used to indicate whether the current trajectory has ended; the navigation strategy data may be used to indicate the current navigation strategy information.

[0092] Specifically, by inputting the training state data and the training position relationship data into the initial navigation model, the robot's navigation strategy data and the robot's state value function at the current moment can be directly output. The specific process of obtaining the robot's navigation strategy data can be determined based on the specific structure of the initial navigation model. For example, when the training state data includes training radar data and training speed data, the long short-term memory network module in the initial navigation model can be used to encode the training radar data and historical radar data to obtain first encoded data, where the training radar data and historical radar data are radar data of the robot at different timestamps; the first encoded data can be encoded using the first fully connected module in the initial navigation model to obtain second encoded data; the first encoded data can be encoded using the second fully connected module in the initial navigation model to obtain third encoded data; the training speed data and the training position relationship data can then be encoded using the third fully connected module in the initial navigation model to obtain fourth encoded data; the second encoded data, the third encoded data, and the fourth encoded data can then be integrated using an attention mechanism to obtain fifth encoded data; and the first encoded data and the fifth encoded data can be fused to obtain the robot's navigation strategy data. On this basis, by introducing the attention mechanism, the robot can pay more attention to the information in the sensor data and prioritize the information that has the greatest impact on the current navigation strategy, dynamically balancing the importance of radar observation and speed, and enhancing the adaptability and flexibility of the strategy in multi-agent scenarios.

[0093] In some embodiments, the training radar data is collected by a single-line lidar sensor in the robot, and the training speed data is collected by an odometer in the robot.

[0094] In order to further reduce the gap between simulation and reality, this embodiment can set the sensor equipped in the robot to a single-line lidar sensor. The gap between simulation and reality of the lidar sensor is smaller than that of sensors such as cameras that require subsequent rendering and other operations.

[0095] At the same time, this embodiment also designs a multi-robot (or intelligent agent) simulation platform based on a physical simulator, which can further narrow the gap between simulation and reality, so that the obtained strategy can be directly deployed on the real machine. In most two-dimensional simulators, the actions of intelligent agents are usually perfectly executed, and their states are usually represented by x t+1 =x t +v t Δt+0.5a t Δt 2 Calculate, where x t is the displacement at time t, v t is the speed at time t, a t is the acceleration at time t, and Δt is the time interval. In a physics-based simulator, the agent state change is calculated by the physics engine, i.e., x t+1 =T(x t ,v t ,a t ,f t ,...), where T is the transfer function of the physics engine, f t is the force on the robot. The transfer function T can make the execution of the action more random and more consistent with the state of the real world. In addition, to further simulate the noise existing in the sensor itself in the real environment, this embodiment adds Gaussian noise to the sensor information in the simulation environment.

[0096] In some embodiments, the reward function includes at least a distance reward function and an obstacle reward function, and the obstacle reward function is related to the speed and orientation angle of the robot.

[0097] It can be considered that the dense reward function commonly used in the DRL-based obstacle avoidance algorithm includes distance reward, obstacle penalty and efficiency reward, that is, r total =r g +r c +r e , r g is the distance reward, calculated as the difference in distance between the robot and the goal point at the current time step and the previous time step; r c is the obstacle penalty, which is usually calculated only by the closest distance to the obstacle, r other is a reward that varies across different methods. Furthermore, collisions with other objects incur significant penalties, while reaching the endpoint yields significant rewards, concluding the collection of the current trajectory. However, a reward based solely on distance can lead the policy to take suboptimal actions in certain scenarios, such as when the agent is navigating narrow gaps or when multiple robots are traveling in opposite directions. In these scenarios, since the reward function is based solely on distance, more optimal actions or potential threats may be overlooked.

[0098] To address the problems of traditional reward functions in the aforementioned scenarios, this embodiment proposes a novel reward function. While considering common factors such as the distance between the agent and the target point and the collision penalty with obstacles, it introduces a reward function based on speed and sensor information. This allows the agent to better combine multimodal information and encourages the agent to reach the target point as quickly as possible along the shortest path while ensuring safety.

[0099] Furthermore, the reward function considers coordination between agents to avoid congestion and collisions in multi-agent scenarios. For example, when multiple agents encounter each other, the reward function will provide appropriate rewards or penalties based on their relative positions and motion states, encouraging them to avoid each other and maintain a safe distance, thus enabling multi-agent navigation.

[0100] Specifically, the reward function proposed in this embodiment may be related to the current direction and speed of the agent, that is, Where l represents the maximum length of the radar ray, represents the radar observation result of agent i at time step t, ω c is r c The weight vector that changes with angular velocity is weighted and summed with the sensor information collected by the agent to obtain the reward function related to the obstacle. Among them, σ is a hyperparameter, which can be set to 0.2; x is the current orientation of the agent; w is the current angular velocity of the agent; Δt is the time interval.

[0101] Figure 3 This is a flow chart of determining a target navigation model provided by an embodiment of the present application. Figure 3 As shown in Figure 1, the training process of the target navigation model is mainly divided into two parts: trajectory data collection and policy update. The trajectory data collection cycle can include: the agent obtains environmental information through sensors, then undergoes simple information processing (stacking), and then inputs the processed signal (observation state) into the neural network. The neural network outputs the policy and samples it to obtain the agent's action. The agent then executes the action and receives feedback from the environment (reward, next state, termination signal). The process then enters the next loop, in which all trajectories obtained by the agent are stored in the replay buffer.

[0102] Specifically, the intelligent agent (i.e., robot) in the simulation platform can obtain the radar information at the current moment through the radar The radar observation is cached to obtain multi-frame radar information; the position relationship between the agent and the target is obtained through the local coordinates communicated with the target Including the distance and direction difference between the agent and the target; the speed of the agent is collected through the odometer Specifically, it may include velocity v and angular velocity w. Then the collected information can be input into the neural network to obtain the current moment strategy π(a t |o t ), the value function V(o t ), action a can be obtained by sampling the current moment strategy t , the simulation platform can calculate the reward r of this action according to the reward function t , and judge whether the current trajectory has ended and return done t .

[0103] Then, the agent can execute the action and enter the trajectory collection loop for the next moment. The data is stored in the replay cache and waits for the policy update phase. For example, after the data collection phase has cycled a certain number of times, the policy update phase will begin. During the policy update phase, all data stored in the replay cache is retrieved and used to update the network. After that, the replay cache is cleared and the next round of data collection begins. Furthermore, the training process repeats the above data collection and policy update phases until a specific number of iterations is reached or the policy converges. On this basis, the parallelism between multiple agents can be fully utilized to improve training efficiency and effectively handle complex interaction problems in large-scale agent systems. In addition, since each agent collects trajectories separately, the policy can navigate in unknown environments with dynamic objects.

[0104] Among them, the specific navigation processing process of the neural network can include using the LSTM layer to process radar information Encode to obtain the first encoded data S; input the first encoded data S into different fully connected blocks to generate the value V (i.e., the second encoded data) and the key K vector (i.e., the third encoded data); at the same time, the speed can be and position relationship Encode to generate query vector Q (i.e., fourth encoded data). Then, by formula Integrate all relevant information into a comprehensive representation to obtain X (i.e., the fifth coded data), where d k is the size of K; the encoding obtained by the attention mechanism and LSTM is fused through an adaptive parameter α, and the final encoding is obtained through the Actor network [v t ,w t ]; Finally, by combining v std ,w std Get the current strategy π(a t |ot ), which is the navigation strategy data. In addition, The value function V(o) of the current state is obtained through the Critic network t ). In this embodiment, the fully connected block may refer to a module consisting of two fully connected layers, and the number of neurons in each fully connected layer may be 256.

[0105] Furthermore, to improve the efficiency of the data collection phase and the dynamics of the environment, this embodiment also designs a collision replay mechanism for the simulation platform. This mechanism replays the state (pose and velocity) of the agent that collided with the agent back to N time steps ago, and then continues to collect trajectories, while the remaining agents remain in their current state. This collision replay mechanism not only allows the agents to collect more collision data, but also enhances the dynamics and unpredictability of the environment because replaying can cause sudden interruptions to agents in normal navigation.

[0106] As can be seen from the above description, compared to other learning algorithms based on Proximal Policy Optimization (PPO), the robot navigation processing method of this embodiment differs in the collection of trajectories (observation state, action, reward, next observation state, termination signal). Specifically, during trajectory collection, multiple agents simultaneously move in an unknown environment, unaware of the states of other robots and the surrounding environment. They only know their own position, speed, distance and direction to the target point, and radar sensor information.

[0107] Furthermore, compared to strategies that only encode radar observations through LSTM, the attention mechanism allows the strategy to better incorporate the agent's current speed and target information into its decision-making, rather than relying solely on radar observations. This makes the generated actions more consistent with the robot's dynamics. Combining LSTM and the attention mechanism not only improves the strategy's response speed and accuracy, but also enhances the agent's obstacle avoidance capabilities in highly dynamic, multi-agent interactive environments. Furthermore, by limiting the number of model parameters to a scale that can be quickly calculated using the CPU alone, the strategy's real-time performance is ensured.

[0108] In order to verify the performance and effectiveness of the robot navigation processing method of this embodiment, this embodiment was tested in a simulated environment and in the real world.

[0109] In the simulated environment test, a physics-based simulation platform was built to simulate various complex scenarios and dynamic changes in the real world. The experiment set up different scenarios, including single-agent and multi-agent scenarios. In the single-agent scenario, the agent needed to navigate to the target point in an environment containing 15 randomly placed obstacles. The obstacles were 1m×1m×1m cubes with random positions and orientations. In the multi-agent scenario, several agents were placed in a circular formation with a random number of randomly placed obstacles. In the experiment, the distance between the agent's initial position and the target point was 10 meters in the single-agent scenario and 16 meters in the multi-agent scenario. The robustness and adaptability of the strategy were tested by varying the number and distribution of obstacles in the scene, as well as the number and initial positions of the agents.

[0110] During the experiment, the agent navigated using the lightweight end-to-end strategy proposed in this paper, which directly maps raw radar observation data into action commands. In actual application, the robot uses a single-line lidar sensor and a wheeled odometry as radar observation, target information, and velocity data acquisition methods, and directly outputs the current action data based on the built-in network model. Since the robot only requires information about the target in its local coordinate system, positioning is achieved by accumulating distance and orientation from the odometry. In the experiment, the agent's linear velocity was limited to [0,1] m / s, and its angular velocity was limited to [-π,π] rad / s; the control frequency was set to 60 Hz, and the maximum number of simulation steps was 5000.

[0111] Experimental results show that the strategy of this embodiment performs best in single-machine and multi-machine simulation environment tests when only local information is available. In the single-machine simulation environment, the success rate is 95.9%, the collision rate is 3.9%, the trapped rate is 0.2%, and the average number of steps is 1300.78; in the multi-machine simulation environment, the success rate is 94.78%, the collision rate is 4.68%, the trapped rate is 0.53%, and the average number of steps is 1805.94.

[0112] Compared to existing algorithms that use global information for planning, the robot navigation processing method of this embodiment significantly improves both the success rate and collision rate. Furthermore, during multi-agent experiments, the agents exhibited significant deceleration, which is very rare in other end-to-end strategies.

[0113] In order to further verify the application effect of the strategy in the real world, this embodiment was also tested using the Turtlebot3 robot in a real environment. Since the maximum linear velocity and angular velocity of Turtlebot3 are 0.2 m / s and 1 radian / s respectively, we only scaled the actions output by the strategy accordingly to adapt to the capabilities of the robot. The settings of other parameters are consistent with those in the simulation environment. In the experiment, the robot used its own radar for environmental perception. The radar has a field of view of 2π radians and 360 laser beams; this embodiment uses 72 laser beams with a field of view of 0.8π radians. In the single-agent scenario, the robot needs to pass through a narrow passage and avoid obstacles of unknown positions and shapes to reach the target point. The experimental results show that the robot successfully passed through the narrow passage and avoided obstacles, and finally reached the target point.

[0114] In a multi-agent scenario, robots need to avoid interference from other robots and unexpected pedestrians. In the experiment, a pedestrian appeared in the environment at 5 seconds and began to interfere with the robots. Subsequently, the pedestrian walked directly through two robots at 10 seconds and 13 seconds. Between 17 seconds and 27 seconds, the pedestrian continued to interfere with the operation of one robot. At 31 seconds, all robots successfully reached the target point and avoided collisions with other objects. These experimental results demonstrate the effectiveness and reliability of the strategy of this embodiment in the real world and its navigation capabilities in complex environments.

[0115] Through the above-mentioned simulated environment tests and real-world tests, the robot navigation processing method of this embodiment has been fully verified, and can effectively solve the navigation problem of multi-agent systems in complex environments. It has a high success rate, low collision rate and good adaptability, and can meet the needs of practical applications.

[0116] To sum up, the robot navigation processing method of this embodiment is based on the proximal policy optimization algorithm. It can accelerate the strategy convergence speed through multi-robot parallel computing, while making the training environment more dynamic, so that the intelligent agent can learn better strategies in a multi-machine navigation environment.

[0117] The robot navigation processing method of this embodiment adopts a lightweight end-to-end strategy that can directly map raw radar observation data into action commands for the intelligent agent, simplifying the navigation process, reducing the number of parameters and computing resource requirements, enabling the strategy to run on resource-constrained devices, and reducing the hardware cost and energy consumption of the system; through the novel reward function design, the computational burden of the strategy is also reduced, enabling the intelligent agent to make navigation decisions quickly and accurately in complex environments, reducing the number of steps and time required to reach the target point, and improving the success rate of navigation.

[0118] The robot navigation processing method of this embodiment can effectively utilize the historical observation information of the intelligent agent through the long short-term memory network module, alleviate the information loss problem in the POMDP, enable the intelligent agent to better understand the dynamic changes of the environment and its own motion state, reduce the risk of collision and congestion, and thus make more accurate and reliable navigation decisions in complex environments, thereby enhancing the robustness and safety of navigation.

[0119] In addition, the robot navigation processing method of this embodiment introduces an attention mechanism, which enables the strategy to pay more attention to the most important information in the current task, such as the location of the target point, the relative position of obstacles, etc., so that the intelligent agent can better handle the uncertainty and dynamic changes of the environment, further improving the decision-making performance and adaptability of the strategy in complex scenarios, and improving the synergy and coordination between multiple agents, so that the entire system can complete complex navigation tasks more efficiently.

[0120] Finally, by designing a simulation platform based on a physics engine, the gap between simulation and reality is further narrowed, and the performance loss during the migration process is reduced, so that the strategy trained by this algorithm can be directly deployed on a real robot, improving the effectiveness and reliability of the strategy in practical applications.

[0121] Therefore, the robot navigation processing method of this embodiment has broad application prospects in multiple fields. It is applicable to any scenario that requires the collaborative work of multiple intelligent agents and has important economic and social value. For example, in the field of industrial automation, this method can be used to coordinate multiple robots to collaboratively complete complex assembly and handling tasks, thereby improving production efficiency and safety. In the field of logistics and transportation, this method can effectively manage the path planning and obstacle avoidance of multiple unmanned vehicles or drones, thereby improving the efficiency and reliability of cargo transportation. In addition, in agricultural automation, this method can be applied to the field operations of multiple agricultural robots, such as sowing, fertilizing, and harvesting, to optimize the operation path and improve the operation efficiency and quality.

[0122] Corresponding to the robot navigation processing method of the above embodiment, Figure 4 This is a structural block diagram of a robot navigation processing device provided in one embodiment of the present application. For the sake of convenience, only the parts related to the embodiment of the present application are shown.

[0123] Reference Figure 4 , the device comprises:

[0124] An acquisition module 301 is used to acquire the current state data of the robot and the current position relationship data between the robot and the target navigation position;

[0125] A navigation processing module 302 is configured to process the current state data and the current position relationship data using a target navigation model to determine the target motion data of the robot. The target navigation model is obtained by continuously updating the initial navigation model based on the training state data, the training position relationship data, and the training trajectory information.

[0126] The control module 303 is used to control the robot to execute the target action data so that the robot travels to the target navigation position.

[0127] This embodiment provides a robot navigation processing device. An acquisition module acquires the robot's current state data and current position relationship data between the robot and a target navigation location. A navigation processing module processes the current state data and current position relationship data using a target navigation model to determine the robot's target motion data. The target navigation model is obtained by continuously updating an initial navigation model based on training state data, training position relationship data, and training trajectory information. A control module controls the robot to execute the target motion data, causing the robot to travel to the target navigation location. This device, by directly determining the robot's target motion data through processing using the target navigation model based on the current state data and current position relationship data, can reduce the burden on computing resources while simplifying the robot's navigation process without compromising navigation accuracy, thereby meeting the robot's navigation needs in various scenarios.

[0128] Optionally, the robot navigation processing device provided in this embodiment further includes:

[0129] a training trajectory determination module, configured to continuously determine training trajectory information of the robot before obtaining current state data of the robot and current positional relationship data between the robot and a target navigation position, wherein the training trajectory information is obtained by performing navigation processing on the training state data and the training positional relationship data according to an initial navigation model, the initial navigation model including at least a long short-term memory network module;

[0130] An updating module is used to update the initial navigation model according to the training state data, the training position relationship data and the training trajectory information before obtaining the current state data of the robot and the current position relationship data between the robot and the target navigation position to obtain an updated navigation model;

[0131] Returning the execution module, configured to use the updated navigation model as the initial navigation model before obtaining the current state data of the robot and the current position relationship data between the robot and the target navigation position, and returning to execute the step of continuously determining the training trajectory information of the robot until the updated navigation model is updated a specified number of times or the updated navigation model reaches convergence;

[0132] The model determination module is used to determine the updated navigation model that has reached a specified number of times or converged as the target navigation model before obtaining the current state data of the robot and the current position relationship data between the robot and the target navigation position.

[0133] Optionally, the training trajectory determination module includes:

[0134] an acquisition unit, configured to acquire training state data of the robot and training position relationship data between the robot and a training navigation position;

[0135] A navigation processing unit is used to perform navigation processing on the training state data and the training position relationship data using an initial navigation model to obtain training trajectory information of the robot, where the training trajectory information includes at least training motion data;

[0136] The action execution unit is used to control the robot to execute the training action data and return to execute the steps of obtaining the robot's training state data and the training position relationship data between the robot and the training navigation position until the obtained training trajectory information reaches a preset number.

[0137] Optionally, the training trajectory information further includes a state value function, reward data, and trajectory state data; and the navigation processing unit includes:

[0138] The input subunit is used to input the training state data and the training position relationship data into the initial navigation model to obtain the robot's navigation strategy data and the robot's state value function at the current moment;

[0139] The sampling subunit is used to sample the navigation strategy data to obtain the training action data of the robot;

[0140] A reward subunit, configured to calculate the training action data using a reward function to obtain reward data for the training action data;

[0141] The trajectory judgment subunit is used to judge whether the current navigation trajectory has ended based on the training action data and obtain trajectory status data.

[0142] Optionally, the training state data includes training radar data and training speed data, and the input subunit is specifically used to:

[0143] Using the long short-term memory network module in the initial navigation model, the training radar data and the historical radar data are encoded to obtain first encoded data, wherein the training radar data and the historical radar data are radar data of the robot at different timestamps;

[0144] Encoding the first coded data using the first fully connected module in the initial navigation model to obtain second coded data;

[0145] Encoding the first encoded data using the second fully connected module in the initial navigation model to obtain third encoded data;

[0146] Using the third fully connected module in the initial navigation model, the training speed data and the training position relationship data are encoded to obtain fourth encoded data;

[0147] The attention mechanism is used to integrate the second encoded data, the third encoded data, and the fourth encoded data to obtain the fifth encoded data;

[0148] The first coded data and the fifth coded data are fused to obtain navigation strategy data of the robot.

[0149] Optionally, the training radar data is collected by a single-line laser radar sensor in the robot, and the training speed data is collected by an odometer in the robot.

[0150] Optionally, the reward function includes at least a distance reward function and an obstacle reward function, and the obstacle reward function is related to the speed and orientation angle of the robot.

[0151] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0153] The embodiment of the present application also provides a terminal device, Figure 5 This is a schematic diagram of the structure of a terminal device provided by an embodiment of the present application. Figure 5As shown, the terminal device includes: at least one processor 401, a memory 402, an input device 403, an output device 404, and a computer program stored in the memory 402 and executable on at least one processor 401. When the processor 401 executes the computer program, the steps in any of the above-mentioned method embodiments are implemented.

[0154] The input device 403 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the terminal device. The output device 404 may include a display device such as a display screen.

[0155] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by the processor 401, the steps in the above-mentioned method embodiments can be implemented.

[0156] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 401, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, a computer-readable storage medium cannot be an electric carrier signal or a telecommunication signal.

[0158] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0159] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0160] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal devices and methods can be implemented in other ways. For example, the device / terminal device embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0161] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0162] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A robot navigation processing method, characterized in that: include: Acquiring current state data of the robot and current position relationship data between the robot and a target navigation position; Using a target navigation model to process the current state data and the current position relationship data to determine target motion data of the robot, wherein the target navigation model is obtained by continuously updating an initial navigation model based on training state data, training position relationship data, and training trajectory information; The robot is controlled to execute the target motion data so that the robot travels to the target navigation position.

2. The robot navigation processing method according to claim 1, wherein: Before acquiring the current state data of the robot and the current position relationship data between the robot and the target navigation position, the method further includes: Continuously determining training trajectory information of the robot, wherein the training trajectory information is obtained by performing navigation processing on the training state data and the training position relationship data according to the initial navigation model, the initial navigation model comprising at least a long short-term memory network module; updating the initial navigation model according to the training state data, the training position relationship data, and the training trajectory information to obtain an updated navigation model; Using the updated navigation model as the initial navigation model, and returning to the step of continuously determining the training trajectory information of the robot until the updated navigation model is updated a specified number of times or the updated navigation model reaches convergence; The updated navigation model that reaches a specified number of times or reaches convergence is determined as the target navigation model.

3. The robot navigation processing method according to claim 2, wherein: The continuously determining the training trajectory information of the robot includes: Acquiring training state data of the robot and training position relationship data between the robot and a training navigation position; Using the initial navigation model to perform navigation processing on the training state data and the training position relationship data to obtain training trajectory information of the robot, wherein the training trajectory information at least includes training motion data; The robot is controlled to execute the training action data, and returns to execute the step of obtaining the training state data of the robot and the training position relationship data between the robot and the training navigation position, until the obtained training trajectory information reaches a preset number.

4. The robot navigation processing method according to claim 3, wherein: The training trajectory information also includes a state value function, reward data and trajectory state data; The using the initial navigation model to perform navigation processing on the training state data and the training position relationship data to obtain the training trajectory information of the robot includes: Inputting the training state data and the training position relationship data into the initial navigation model to obtain the navigation strategy data of the robot and the state value function of the robot at the current moment; Sampling the navigation strategy data to obtain training motion data of the robot; Calculating the training action data using a reward function to obtain reward data for the training action data; Based on the training action data, it is determined whether the current navigation track has ended, and track status data is obtained.

5. The robot navigation processing method according to claim 4, wherein: The training state data includes training radar data and training speed data. Inputting the training state data and the training position relationship data into the initial navigation model to obtain the navigation strategy data of the robot includes: Using a long short-term memory network module in the initial navigation model, encoding the training radar data and the historical radar data to obtain first encoded data, wherein the training radar data and the historical radar data are radar data of the robot at different timestamps; Encoding the first coded data using a first fully connected module in the initial navigation model to obtain second coded data; encoding the first coded data using a second fully connected module in the initial navigation model to obtain third coded data; encoding the training speed data and the training position relationship data using a third fully connected module in the initial navigation model to obtain fourth encoded data; Using an attention mechanism to integrate the second encoded data, the third encoded data, and the fourth encoded data to obtain fifth encoded data; The first coded data and the fifth coded data are fused to obtain navigation strategy data of the robot.

6. The robot navigation processing method according to claim 5, wherein: The training radar data is collected by a single-line laser radar sensor in the robot, and the training speed data is collected by an odometer in the robot.

7. The robot navigation processing method according to claim 4, wherein: The reward function includes at least a distance reward function and an obstacle reward function, and the obstacle reward function is related to the speed and orientation angle of the robot.

8. A robot navigation processing device, characterized in that: include: An acquisition module is used to acquire the current state data of the robot and the current position relationship data between the robot and the target navigation position; a navigation processing module, configured to determine target motion data of the robot by processing the current state data and the current position relationship data using a target navigation model, wherein the target navigation model is obtained by continuously updating an initial navigation model based on training state data, training position relationship data, and training trajectory information; A control module is used to control the robot to execute the target action data so that the robot travels to the target navigation position.

9. A terminal device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the terminal device implements the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a terminal device, the terminal device is caused to execute the method according to any one of claims 1 to 7.

Images