A navigation decision-making method and device based on deep learning and intelligent agents

Abstract

The application provides a navigation decision method and device based on deep learning and intelligent Agent, the method comprising: collecting an RGB image and a depth image of a current environment, and obtaining device power of a robot and a task instruction input by a user; analyzing the RGB image and the depth image to determine obstacle information of the current environment, and analyzing the task instruction to obtain task associated information; generating a first candidate target point set according to the RGB image and the depth image; inputting the obstacle information, the device power, the task associated information and the first candidate target point set into a preset reinforcement learning model for comprehensive evaluation to determine an optimal target point for robot movement in the first candidate target point set. Through implementation of the application scheme, decision shock caused by single perception failure or task change can be effectively avoided, thereby improving the robustness of the navigation system in a dynamic environment.

Description

Technical Field

[0001] This application relates to the field of robot navigation technology, and in particular to a navigation decision-making method and apparatus based on deep learning and intelligent agents. Background Technology

[0002] In the field of indoor mobile robot navigation and mapping, with the development of deep learning technology, reinforcement learning has been gradually applied to navigation decision optimization. However, the current technology system still suffers from problems such as simplistic decision-making and insufficient environmental adaptability. Traditional reinforcement learning navigation methods (such as decision models based on PPO and DQN) can only output a single target point based on fixed inputs (such as occupied grid maps or RGB images), lacking the ability to comprehensively judge "dynamic environmental changes, task priority adjustments, and resource constraint adaptation." At the same time, current robot agent (intelligent software module with autonomous decision-making capabilities) technology focuses on single tasks (such as dialogue interaction and simple obstacle avoidance) and has not been deeply integrated with reinforcement learning navigation. Existing agents only act as "instruction executors" and cannot participate in the entire process of target point generation, path planning, and dynamic adjustment, resulting in insufficient robustness of the navigation system when facing complex environments (such as dense dynamic obstacles or temporary changes in task objectives). Summary of the Invention

[0003] This application provides a navigation decision-making method and apparatus based on deep learning and intelligent agents to address the problem of insufficient robustness of robot navigation systems in complex environments in related technologies.

[0004] The first aspect of this application provides a navigation decision-making method based on deep learning and intelligent agents, the navigation decision-making method based on deep learning and intelligent agents comprising: Acquire RGB and depth images of the current environment, and obtain the robot's battery level and user-inputted task commands; The environmental assessment agent analyzes the RGB image and the depth image to determine the obstacle information in the current environment, and the task analysis agent analyzes the task instructions to obtain task association information. A first set of candidate target points is generated based on the RGB image and the depth image; The obstacle information, the device battery level, the task association information, and the first candidate target point set are input into a preset reinforcement learning model for comprehensive evaluation to determine the optimal target point for the robot to move from the first candidate target point set.

[0005] Optionally, in a first implementation of the first aspect of this application, the step of analyzing the RGB image and the depth image through an environmental assessment agent to determine the obstacle information of the current environment includes: The RGB image is semantically segmented by an environmental assessment agent to generate a pixel-level semantic label map, which is labeled with at least passable areas, obstacle categories and dynamic target categories. The depth image is inversely projected using an environmental assessment agent to generate 3D point cloud data aligned with the semantic label graph space. Based on the dynamic target categories marked in the semantic tag map, a subset of the dynamic target's point cloud is extracted from the 3D point cloud data, and inter-frame difference calculation is performed on the point cloud subsets of multiple consecutive frames to generate motion vector information of each dynamic target. Based on the obstacle categories in the semantic tag map and the 3D point cloud data, a static obstacle distribution map is generated, and combined with the motion vector information, obstacle information containing the static obstacle positions and dynamic obstacle movement trends is generated.

[0006] Optionally, in the second implementation of the first aspect of this application, the step of parsing the task instructions through the task parsing agent to obtain task association information includes: The task parsing agent performs instruction type matching processing on the task instruction, matching the task instruction with a preset instruction template library to determine the task type corresponding to the task instruction; The priority level associated with the task type is queried from the preset priority mapping table, and the corresponding priority quantization coefficient is generated according to the priority level. The task type is associated with the priority quantification coefficient to generate task association information containing type identifier and priority coefficient.

[0007] Optionally, in a third implementation of the first aspect of this application, the step of generating a first candidate target point set based on the RGB image and the depth image includes: An RGB image and a depth image of the sample environment are generated using an environment simulator, and a first set of candidate target points for the sample is generated based on the RGB image and the depth image. Randomly select a sample target point from the first set of candidate target points, and control the robot to move according to the randomly selected sample target point; Obtain map data of the explored and unexplored areas after the robot moves, record the distance and time the robot travels to the target point, and obtain the area of ​​the newly added unexplored area after the move; The local map is updated based on the map data of the explored and unexplored areas, and the candidate target points in the first candidate target point set are weighted and evaluated based on the travel distance, the travel time, and the area of ​​the newly added unexplored area to generate the evaluation weight of each candidate target point. The candidate target points are sorted according to their evaluation weights to generate the first candidate target point set.

[0008] Optionally, in the fourth implementation of the first aspect of this application, the step of inputting the obstacle information, the device battery level, the task association information, and the first candidate target point set into a preset reinforcement learning model for comprehensive evaluation to determine the optimal target point for robot movement in the first candidate target point set includes: Based on the priority quantification coefficient in the task association information, the task association degree is calculated for each candidate target point in the first candidate target point set, and the target association degree coefficient between each candidate target point and the task target area is generated. Based on the dynamic obstacle movement trend information in the obstacle information, the path safety of each candidate target point is evaluated, and the path safety coefficient corresponding to each candidate target point is generated. Based on the device power consumption, path resource consumption analysis is performed on each candidate target point to generate resource consumption rate coefficients corresponding to each candidate target point. Based on the target correlation coefficient, the path safety coefficient, and the resource consumption rate coefficient, the comprehensive weight coefficient of each candidate target point is determined, and the candidate target points are sorted according to the comprehensive weight coefficient. The candidate target point with the highest comprehensive weight coefficient is determined as the optimal target point for robot movement.

[0009] Optionally, in the fifth implementation of the first aspect of this application, after the step of sorting the candidate target points according to their evaluation weights to generate a first set of candidate target points, the method further includes: The environment assessment agent receives global hotspot area information from the edge server and performs weight boosting on candidate target points located in the global hotspot area in the first candidate target point set based on the global hotspot area information to generate a second candidate target point set. The dynamic obstacle prediction range is uploaded by the receiving server of the environment assessment agent. The weights of the candidate target points in the second candidate target point set that fall within the predicted movement range are updated according to the predicted movement range to generate a third candidate target point set. Before the next optimal target point is determined, the first candidate target point set is updated based on the third candidate target point set.

[0010] A second aspect of this application provides a navigation decision-making device based on deep learning and intelligent agents. This device is used to implement a navigation decision-making method based on deep learning and intelligent agents. The navigation decision-making device includes: The acquisition module is used to acquire RGB and depth images of the current environment, as well as obtain the robot's battery level and user-inputted task commands. The analysis module is used to analyze the RGB image and the depth image through the environment assessment agent to determine the obstacle information of the current environment, and to analyze the task instructions through the task analysis agent to obtain task association information; The generation module is used to generate a first set of candidate target points based on the RGB image and the depth image; The decision module is used to input the obstacle information, the device battery level, the task association information, and the first candidate target point set into a preset reinforcement learning model for comprehensive evaluation, and to determine the optimal target point for the robot to move from the first candidate target point set.

[0011] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute a computer program stored in the memory, and when the processor executes the computer program, it implements the steps of the navigation decision-making method based on deep learning and intelligent agents provided in the first aspect of this application.

[0012] The fourth aspect of this application provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the navigation decision-making method based on deep learning and intelligent agents provided in the first aspect of this application.

[0013] Compared with the prior art, this application has the following beneficial effects: By using an environmental assessment agent to process RGB and depth images to obtain obstacle information, and a task parsing agent to parse task instructions to obtain task-related information, and combining the device battery level and candidate target points to input into a reinforcement learning model for comprehensive evaluation, multi-source heterogeneous information fusion decision-making can be achieved. This can effectively avoid decision oscillations caused by single perception failure or task changes, thereby improving the robustness of the navigation system in dynamic environments. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating the navigation decision-making method based on deep learning and intelligent agents provided in the embodiments of this application; Figure 2 A schematic diagram of the navigation point selection training process based on deep learning algorithms provided in an embodiment of this application; Figure 3 A schematic diagram of the program modules of the navigation decision-making device based on deep learning and intelligent agents provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0016] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] To address the issue of insufficient robustness of robot navigation systems in complex environments in related technologies, embodiments of this application provide a navigation decision-making method based on deep learning and intelligent agents, such as... Figure 1 This is a flowchart illustrating the navigation decision-making method based on deep learning and intelligent agents provided in this embodiment. The navigation decision-making method based on deep learning and intelligent agents includes the following steps: Step 110: Collect RGB and depth images of the current environment, and obtain the robot's battery level and user-inputted task commands.

[0018] Specifically, RGB images are used to describe visual information in the current environment, while depth images are used to describe the distance information between the robot and various spatial locations within the current environment. Together, they constitute the robot's initial visual perception of its surroundings. The device's battery level is read through the battery management system, reflecting the robot's current energy reserve status. User-inputted task commands are issued in natural language or structured instruction form, specifying the specific tasks the robot needs to perform.

[0019] Step 120: The environmental assessment agent analyzes the RGB and depth images to determine the obstacle information in the current environment, and the task analysis agent analyzes the task instructions to obtain the task association information.

[0020] Specifically, the Environment Assessment Agent is an intelligent module dedicated to visual perception and scene understanding. It integrates a lightweight semantic segmentation network capable of pixel-by-pixel classification of input RGB images, distinguishing different semantic categories such as passable areas, static obstacles, and dynamic targets. The Environment Assessment Agent also includes an inverse projection transformation module, which converts depth images into 3D point clouds and calculates motion vectors for dynamic targets using data from multiple consecutive frames. The input to the Environment Assessment Agent is an RGB image and a depth image; the output is structured obstacle information, including the spatial location of static obstacles and the motion trend of dynamic obstacles. The Task Parsing Agent is an intelligent module dedicated to semantic understanding and task quantification. It maintains a pre-defined instruction template library, matching user-input natural language instructions or structured task instructions with patterns in the library to identify the specific task type. This agent also includes a priority mapping table, associating different task types with corresponding priority levels and quantifying qualitative descriptions into computable numerical coefficients. The input to the Task Parsing Agent is the user-input task instruction; the output is task association information, including task type identifiers and priority quantification coefficients.

[0021] Step 130: Generate a first set of candidate target points based on the RGB image and the depth image.

[0022] Specifically, after completing environmental perception and task analysis, a first set of candidate target points is generated based on RGB and depth images. Based on the joint processing of the RGB and depth images, a local occupancy grid map of the current environment is constructed by fusing their complementary information. This map records three states of space—occupied, vacant, or unknown—in the form of grid cells. Based on the abrupt boundary changes between explored and unexplored areas in this map, leading boundary points are extracted. Then, discrete boundary points are spatially clustered, and the center coordinates of each cluster are calculated. Finally, the center coordinates of each cluster are used as candidate target points, forming the first set of candidate target points.

[0023] Step 140: Input obstacle information, device battery level, task association information, and the first candidate target point set into the preset reinforcement learning model for comprehensive evaluation, and determine the optimal target point for robot movement in the first candidate target point set.

[0024] Specifically, obstacle information, device battery level, task-related information, and a set of first-choice candidate target points are input into a pre-defined reinforcement learning model for comprehensive evaluation to determine the optimal target point for the robot's movement within this set of candidate target points. This reinforcement learning model is a neural network decision-making model pre-trained using deep reinforcement learning algorithms. Its input layer simultaneously receives the aforementioned four types of heterogeneous data. After multi-layer nonlinear transformations and attention mechanisms, spatial geometric features, energy constraint information, task semantic features, and the spatial distribution of candidate points are jointly encoded. Internally, the model evaluates the value of each candidate target point using a dominance function, comprehensively balancing path safety, energy consumption, and task progress efficiency, and finally outputs the candidate target point with the highest evaluation score as the optimal target point.

[0025] In one optional implementation, the step of analyzing RGB and depth images using an environment assessment agent to determine obstacle information in the current environment includes: performing semantic segmentation on the RGB image using the environment assessment agent to generate a pixel-level semantic label map, wherein the semantic label map at least labels passable areas, obstacle categories, and dynamic target categories; performing inverse projection transformation on the depth image using the environment assessment agent to generate 3D point cloud data spatially aligned with the semantic label map; extracting a subset of dynamic target point clouds from the 3D point cloud data based on the dynamic target categories labeled in the semantic label map, and performing inter-frame difference calculation on the point cloud subsets of multiple consecutive frames to generate motion vector information for each dynamic target; generating a static obstacle distribution map based on the obstacle categories in the semantic label map and the 3D point cloud data, and combining the motion vector information to generate obstacle information including the location of static obstacles and the movement trend of dynamic obstacles.

[0026] In this embodiment, after receiving the RGB and depth images collected by the robot, the environmental assessment agent needs to perform semantic segmentation on the RGB images. Semantic segmentation refers to using a deep learning network to assign a corresponding semantic category to each pixel in the image, transforming the image from simple color information into semantic information with environmental meaning. Each pixel corresponds to a unique category label, thus forming a pixel-level semantic label map. In the pixel-level semantic label map, passable areas represent locations where the robot can safely navigate, such as the ground, roads, or open areas; obstacle categories represent fixed objects that do not change position, such as walls, equipment, shelves, or pillars; and dynamic target categories represent objects that can autonomously change position, such as workers, forklifts, or transport robots. Because each pixel has a clear category identifier, the boundary range of each type of target in the image can be accurately determined.

[0027] Inverse projection transformation refers to the process of restoring the coordinates of a two-dimensional image to true three-dimensional spatial coordinates based on the depth value corresponding to each pixel in the depth image and in conjunction with the camera's internal parameters, thereby generating three-dimensional point cloud data. Three-dimensional point cloud data consists of a large number of spatial points, each with three-dimensional coordinate information, which can realistically reflect the spatial position and geometric structure of objects in the environment. Since the RGB image and the depth image originate from the same perception system, the generated three-dimensional point cloud data can maintain a spatial correspondence with the pixel-level semantic label map. That is, each pixel in the semantic label map can correspond to one or more three-dimensional spatial points in the point cloud, thus giving each spatial point both spatial coordinate attributes and semantic category attributes.

[0028] The environmental assessment agent extracts corresponding point cloud subsets from 3D point cloud data based on the dynamic target categories labeled in the semantic tag map. Then, it performs inter-frame difference calculation on the continuously acquired multi-frame point cloud subsets. Inter-frame difference calculation involves comparing the 3D coordinates of the same dynamic target acquired at adjacent times, calculating the displacement change of the target in space by calculating the position change of corresponding spatial points, and combining this with the acquisition time interval to calculate the target's motion direction and velocity, thus forming motion vector information. This motion vector information not only includes the direction of movement of the dynamic target but also reflects its motion trend over a future period. For example, in a smart warehouse navigation scenario, the environmental assessment agent identifies a forklift transporting goods. In consecutive frames of point cloud data, it can observe the forklift continuously moving forward towards the warehouse aisle. By calculating the change in the center position of the forklift's point cloud between consecutive frames, it can determine that the forklift is approaching the robot's direction of travel along the aisle, forming the corresponding motion vector information.

[0029] After completing the dynamic target analysis, the environmental assessment agent further generates a static obstacle distribution map based on the obstacle categories in the semantic label map and the corresponding 3D point cloud data. The static obstacle distribution map is a spatial distribution model constructed based on the positional relationships of fixed obstacles in 3D space. It not only records the position of each obstacle but also accurately describes the spatial connections between them. When the static obstacle distribution map is fused with the dynamic target motion vector information, complete obstacle information is generated. This obstacle information includes both the positional distribution of static obstacles and the future movement trends of dynamic obstacles. For example, in the same intelligent warehouse navigation scenario, the static obstacle distribution map shows that the shelves are located on both sides of the aisle, while the motion vector information shows that the forklift is moving forward along the aisle. Therefore, the final obstacle information not only reflects the fixed space occupied by the shelves but also the movement area that the forklift will pass through in the future. This allows subsequent navigation decisions to not only avoid fixed obstacles but also to proactively avoid locations that dynamic targets may pass through, improving the accuracy and safety of navigation path planning.

[0030] In one optional implementation, the step of parsing task instructions using a task parsing agent to obtain task association information includes: performing instruction type matching processing on the task instructions using the task parsing agent, matching the task instructions with a preset instruction template library to determine the task type corresponding to the task instructions; querying the priority level associated with the task type from a preset priority mapping table, and generating a corresponding priority quantization coefficient based on the priority level; and associating and binding the task type with the priority quantization coefficient to generate task association information containing type identifier and priority coefficient.

[0031] In this embodiment, after receiving the task instruction input by the user, the task parsing agent performs semantic parsing on the task instruction and determines the task type corresponding to the current task through an instruction type matching mechanism. Specifically, the task parsing agent pre-stores an instruction template library, which is used to save task description features, keyword information, and semantic expression rules corresponding to different task types. The task parsing agent first extracts features from the natural language task instruction input by the user, converting the key semantic information in the task instruction into a recognizable instruction feature vector. Feature extraction refers to extracting information from text information that reflects the task purpose, execution object, and execution requirements. Subsequently, the task parsing agent matches the generated instruction feature vector with the preset instruction template library, and determines the task type corresponding to the current task instruction by calculating the semantic similarity between the current task instruction and different templates.

[0032] After determining the task type, the task parsing agent further queries a pre-defined priority mapping table. This table establishes the relationship between different task types and their urgency, storing the mapping between task type, priority level, and corresponding priority quantification coefficient. The priority level indicates the importance of the task execution; for example, emergency rescue tasks correspond to a high priority level, routine inspection tasks to a medium priority level, and regular area exploration tasks to a low priority level. The priority quantification coefficient converts the task importance into a numerical parameter that can be used in calculations, allowing the subsequent reinforcement learning model to adjust its target point selection strategy according to different task requirements. For example, the highest priority level emergency rescue task might correspond to a priority quantification coefficient of 1.0, a routine inspection task to 0.5, and an area exploration task to 0.3.

[0033] After the task parsing agent obtains the priority level corresponding to the task type, it generates a priority quantification coefficient based on the correspondence in the priority mapping table. It then associates and binds the task type information with the priority quantification coefficient to form task association information. Task association information is a data structure used to describe the current task objective and the importance of the task. It includes at least a task type identifier and a priority coefficient, and may further include association information such as the task target area, task execution requirements, and task resource constraints. This task association information is transmitted to the reinforcement learning model, enabling the model to consider not only the current environmental state but also the task requirements when generating navigation targets, thus adjusting the target point selection results accordingly.

[0034] For example, in a smart warehousing scenario, when a robot is performing an equipment inspection task, the inspection type identifier and corresponding priority coefficient in the task association information will cause the reinforcement learning model to prioritize candidate target points near the area of ​​the equipment to be inspected. When a user inputs "Go to the faulty equipment area immediately for inspection," because this task is parsed as a high-priority inspection task, the task parsing agent generates a higher priority quantification coefficient, enabling the robot to reduce exploration of irrelevant areas and improve the efficiency of the robot reaching the target area and completing the task. In this way, the task parsing agent can convert the user's natural language task instructions into machine-computable task association information, realizing the information association between task requirements and the navigation decision-making process.

[0035] In one optional implementation, the step of generating a first candidate target point set based on an RGB image and a depth image includes: generating an RGB image and a depth image of a sample environment using an environment simulator, and generating a sample first candidate target point set based on the RGB image and the depth image; randomly selecting a sample target point from the sample first candidate target point set, and controlling the robot to move according to the randomly selected sample target point; acquiring map data of the explored and unexplored areas after the robot moves, recording the distance and time the robot travels to the sample target point, and acquiring the area of ​​the newly added unexplored area after the movement; updating the local map based on the map data of the explored and unexplored areas, and performing a weighted evaluation on each candidate target point in the first candidate target point set based on the distance traveled, the time traveled, and the area of ​​the newly added unexplored area to generate an evaluation weight for each candidate target point; and sorting each candidate target point according to its evaluation weight to generate the first candidate target point set.

[0036] In this embodiment, as Figure 2As shown, during the reinforcement learning model training process, the environment simulator is used to construct a virtual interactive scene corresponding to the actual robot operating environment, enabling the robot to acquire a large number of navigation decision samples without the need for continuous operation of real equipment. The environment simulator generates different spatial structures, obstacle distributions, and explorable areas based on preset scene parameters, and simultaneously outputs RGB and depth images of the corresponding scene. The RGB and depth images generated by the environment simulator are input into the navigation decision model. The image perception results determine the robot's current selectable exploration direction and generate a set of first-order candidate target points. The set of first-order candidate target points refers to the set of multiple potential target locations that the robot can reach in the current environmental state. Each candidate target point corresponds to a spatial coordinate, representing the possible location the robot might choose for its next move. For example, when an intelligent warehouse robot explores a warehouse environment, the environment simulator generates a virtual warehouse scene containing shelves, aisles, and free areas. The robot identifies the spatial structure based on the RGB images and obtains distance information for each area by combining the depth images, generating multiple candidate target points from the boundaries of the unexplored areas.

[0037] After generating a set of first-stage candidate target points, a sample target point is selected from this set as the current training interaction target, and the robot is controlled to move towards that target point. During the robot's movement, the environment simulator continuously records changes in the robot's current position and environmental state, enabling the reinforcement learning model to acquire complete navigation feedback data. After the robot reaches the sample target point, the environment simulator acquires map information before and after the robot's movement, updating the explored and unexplored areas. The explored area represents the region where the robot's sensors have acquired environmental information and completed spatial modeling, while the unexplored area represents the region where the robot has not yet acquired complete environmental information and still needs further exploration. By comparing the map data before and after the robot's movement, the newly acquired environmental range can be determined, and the change in the area of ​​the newly unexplored region can be calculated.

[0038] During robot movement, the environment simulator also records the distance traveled and the time taken for the robot to complete the current movement task. The distance traveled reflects the spatial distance the robot needs to reach the target point, the time taken reflects the time required for the robot to navigate to the target point, and the area of ​​newly added unexplored regions is used to evaluate the robot's contribution to exploring the unknown environment after movement. These parameters collectively serve as the evaluation criteria for candidate target points, enabling the reinforcement learning model to not only focus on whether the robot can reach the target location, but also to determine whether the target location can improve the overall efficiency of environmental exploration.

[0039] Based on the map data after the robot's movement, the environment simulator updates the robot's local map. The local map is a small-scale environmental model built by the robot using information acquired by its sensors, recording the spatial structure around the robot, the location of obstacles, and the extent of the explored area. The updated local map is correlated with the movement distance, movement time, and area of ​​newly explored unexplored areas corresponding to candidate target points, and a weighted evaluation is performed on each candidate target point in the first set of candidate target points. Specifically, when a candidate target point allows the robot to reach it with a shorter movement distance and less movement time, while also discovering a larger area of ​​unknown territory, the evaluation weight of that candidate target point is increased; conversely, when a candidate target point requires the robot to traverse a longer path and results in a smaller area of ​​newly explored territory, the evaluation weight of that candidate target point is decreased.

[0040] For example, in an autonomous exploration scenario for an intelligent warehouse robot, there are two candidate target points ahead of the robot. Target point A is close to the robot's current position, but moving it will only reveal a small new area. Target point B is slightly farther away, but reaching it will open up a new warehouse passage area, allowing the robot to obtain more information about the unknown environment. The environment simulator, after comprehensively evaluating the movement distance, time spent, and newly explored area, can assign a higher weight to target point B. This allows the reinforcement learning model to learn not only to choose the closest location but also to choose a location that improves overall mapping efficiency. Finally, the candidate target points are sorted according to their evaluation weights, with those having higher weights prioritized, forming an updated first set of candidate target points. This provides training data for the subsequent navigation decision model to select the optimal movement target. Through this method, the reinforcement learning model can learn target point selection patterns under different environmental states during extensive virtual environment interactions, improving the robot's autonomous exploration and navigation decision-making capabilities.

[0041] In one optional implementation, the step of inputting obstacle information, device battery level, task association information, and a first set of candidate target points into a preset reinforcement learning model for comprehensive evaluation to determine the optimal target point for robot movement within the first set of candidate target points includes: calculating the task association degree of each candidate target point in the first set of candidate target points based on the priority quantification coefficient in the task association information, generating a target association degree coefficient between each candidate target point and the task target area; evaluating the path safety of each candidate target point based on the dynamic obstacle movement trend information in the obstacle information, generating a path safety coefficient corresponding to each candidate target point; analyzing the path resource consumption of each candidate target point based on the device battery level, generating a resource consumption rate coefficient corresponding to each candidate target point; determining the comprehensive weight coefficient of each candidate target point based on the target association coefficient, the path safety coefficient, and the resource consumption rate coefficient, ranking each candidate target point according to the comprehensive weight coefficient, and determining the candidate target point with the highest comprehensive weight coefficient as the optimal target point for robot movement.

[0042] In this embodiment, when determining the robot's next navigation target, the first candidate target point set generated by the reinforcement learning model contains multiple selectable movement positions. The environment assessment agent needs to combine the current task requirements, the dynamic state of the environment, and the robot's own resource status to comprehensively evaluate each candidate target point and select the optimal target point that best matches the current task target from multiple candidate target points.

[0043] Specifically, the environmental assessment agent calculates the task relevance of each candidate target point in the first candidate target point set based on the priority quantification coefficient in the task association information. Task relevance represents the degree of matching between a candidate target point and the current task target area, reflecting the robot's contribution to completing the current task after moving to a target point. The task association information generated by the task parsing agent includes a task type identifier and a priority quantification coefficient. The environmental assessment agent determines the corresponding task target area based on the task type and calculates the positional relationship between each candidate target point and the task target area. When a candidate target point is close to the task target area and can effectively promote the execution of the current task, the target relevance coefficient corresponding to that candidate target point is high; when a candidate target point is far from the task target area or has a low degree of relevance to the current task target, the corresponding target relevance coefficient decreases. For example, when an intelligent warehouse robot performs an equipment inspection task, the target area is the warehouse area where the equipment to be inspected is located. Two candidate target points are located near the equipment area and near the ordinary storage area, respectively. Even if the ordinary storage area is closer to the robot's current position, the candidate target point near the equipment area can complete the inspection task faster, so the candidate target point will get a higher target correlation coefficient.

[0044] After calculating the task relevance, the environmental assessment agent further evaluates the path safety of each candidate target point based on the dynamic obstacle movement trend information in the obstacle information. The path safety coefficient represents the degree to which the robot is affected by dynamic environmental changes as it moves from its current position to the candidate target point. The environmental assessment agent analyzes the spatial relationship between the robot's planned path and the dynamic obstacle's movement area using the dynamic obstacle's movement direction, speed, and predicted movement range. When the path corresponding to the candidate target point does not intersect with the future movement area of ​​the dynamic obstacle, and the robot has sufficient avoidance space, the path safety coefficient of the candidate target point increases; when the candidate target point is located in a position where the dynamic obstacle may pass, or when there is a high risk of collision during the robot's movement, the path safety coefficient of the candidate target point decreases. For example, in a smart warehouse environment, the environmental assessment agent detects that a forklift is moving along a certain aisle and predicts that the forklift will pass through the area in front of the aisle in the next few seconds. If a candidate target point is within the predicted movement range of the forklift, the path safety coefficient of that candidate target point decreases, while candidate target points located in other safe aisle areas receive a higher safety rating.

[0045] Meanwhile, the environmental assessment agent analyzes the path resource consumption of each candidate target point based on the robot's current battery status. The resource consumption rate coefficient describes the relationship between the energy resources required for the robot to reach the candidate target point and its remaining battery power. The environmental assessment agent calculates the proportion of resources required to complete the movement task based on the path length, path complexity, and the robot's current battery power. A better resource consumption rate coefficient is obtained when the candidate target point is close, the path is relatively flat, and energy consumption is low; conversely, a lower resource consumption rate coefficient is obtained when the candidate target point is far away or requires traversing complex areas leading to high energy consumption. For example, when the smart warehouse robot's battery is low, even if a distant candidate target point expands the exploration range, the resource consumption evaluation for that target point will be lower because it may be unable to return to the charging area after reaching that location.

[0046] After obtaining the target relevance coefficient, path safety coefficient, and resource consumption rate coefficient, the environmental assessment agent performs a fusion calculation on these evaluation parameters to generate a comprehensive weight coefficient for each candidate target point. The comprehensive weight coefficient reflects the overall suitability of the candidate target point in terms of task completion capability, environmental safety, and resource consumption level. In the specific calculation process, the influence ratio of different evaluation parameters can be dynamically adjusted according to task priority, so that high-priority tasks focus more on task completion efficiency, low-battery states focus more on energy consumption limitations, and environments with dense dynamic obstacles focus more on path safety. Subsequently, the environmental assessment agent sorts the candidate target points in the first candidate target point set according to the comprehensive weight coefficient and selects the candidate target point with the highest comprehensive weight coefficient as the optimal target point for the robot's movement.

[0047] For example, in a warehouse inspection scenario, the robot simultaneously receives three candidate target points. One target point is the closest but close to the forklift's operating area; another target point is relatively safe but far from the equipment to be inspected; and the third target point, while at a moderate distance, is close to the equipment, has a safe path, and reasonable power consumption. After comprehensive evaluation, the third target point receives the highest overall weight coefficient, and therefore the robot selects this target point to move to. Through this method, the robot can dynamically adjust its navigation target based on task requirements, environmental changes, and its own resource status, realizing a shift from single path planning to task-driven intelligent navigation decision-making.

[0048] In one optional implementation, after the step of sorting the candidate target points according to their evaluation weights to generate a first candidate target point set, the method further includes: receiving global hotspot area information from the edge server via an environmental assessment agent; performing weight boosting on candidate target points located within the global hotspot area in the first candidate target point set based on the global hotspot area information to generate a second candidate target point set; receiving predicted movement ranges of dynamic obstacles uploaded by the edge server via an environmental assessment agent; updating the weights of candidate target points falling within the predicted movement range in the second candidate target point set based on the predicted movement range to generate a third candidate target point set; and updating the first candidate target point set based on the third candidate target point set before determining the next optimal target point.

[0049] In this embodiment, during the robot's autonomous navigation, the environmental assessment agent not only needs to select targets based on the local environmental information collected by the robot itself, but also needs to combine the global environmental information provided by the edge server and the dynamic environmental prediction information uploaded by other robots to adjust the candidate target points in real time. The environmental assessment agent first receives global hotspot area information from the edge server and performs spatial location matching on each candidate target point in the first candidate target point set. Global hotspot area information refers to the key exploration areas determined by the edge server after fusing and analyzing local map data uploaded by multiple robots. This information reflects locations in the current environment with high exploration or task value, such as areas that have not been fully explored but may contain critical equipment, target objects, or important passages. The edge server obtains the global map state by fusing environmental data collected by multiple robots and generates global hotspot area information based on the degree of regional unknowns, task relevance, and environmental value.

[0050] After receiving global hotspot region information, the environmental assessment agent matches the spatial extent of these hotspot regions with the coordinates of candidate target points in the first candidate target point set. When a candidate target point is located within a global hotspot region, it indicates that the location corresponding to that candidate target point has higher global exploration value. Therefore, the environmental assessment agent increases the weight of this candidate target point. Weight increase refers to adding a weight coefficient associated with the hotspot region to the original candidate target point evaluation weight, making the reinforcement learning model more inclined to select target locations that improve the overall environmental mapping efficiency in subsequent target selection processes. After adjusting the global hotspot region weights, a second candidate target point set containing the influence of global information is regenerated.

[0051] For example, when an intelligent warehouse robot performs an autonomous exploration task, robot A can only acquire a map of its own surrounding area. The edge server, by merging map data uploaded by robots A, B, and C, discovers a large unexplored area in the northeast of the warehouse, and identifies nearby devices to be detected. Therefore, this area is marked as a global hotspot. When robot A generates candidate target points located in the northeast region, the environmental assessment agent increases the weight of that target point, causing robot A to prioritize moving towards that area.

[0052] After generating the second set of candidate target points, the environmental assessment agent further receives the predicted movement range of dynamic obstacles uploaded by the edge server. The predicted movement range of dynamic obstacles refers to the range information obtained by the edge server based on environmental perception data and the historical movement state of the dynamic target, predicting the spatial area that the dynamic target may traverse in the future. This predicted range includes not only the current position of the dynamic target, but also its direction of movement, speed, and the area it may occupy in the future. The environmental assessment agent performs spatial relationship analysis between the predicted movement range and the candidate target points in the second set of candidate target points. When a candidate target point is within the predicted movement range of dynamic obstacles, it indicates that the robot may have a path conflict with the dynamic target during its journey to that location, so the weight of the candidate target point needs to be reduced; when the candidate target point is far from the predicted movement range of dynamic obstacles and the corresponding path has high safety, the weight of the candidate target point is maintained or increased.

[0053] For example, in a smart warehousing scenario, an edge server predicts, based on data uploaded by multiple robots, that a forklift will pass through the central aisle area of ​​the warehouse within the next few seconds. The robot's currently generated set of second candidate target points includes a target point located within this aisle area. The environmental assessment agent, based on the forklift's predicted movement range, determines that this target point poses a potential collision risk. Therefore, it reduces the weight of this target point while retaining a higher weight for candidate target points located within the lateral safety zone, thus preventing the robot from entering the area where dynamic obstacles might pass as originally planned. After adjustment based on dynamic obstacle prediction, a third set of candidate target points is formed.

[0054] After completing the fusion of global hotspot area information and adjusting dynamic obstacle prediction information, the environmental assessment agent updates the first candidate target point set based on the third candidate target point set before determining the next optimal target point. The update process does not regenerate candidate target points; instead, it incorporates edge-side global environmental information and dynamic environmental prediction results into the target weight calculation process, ensuring that the candidate target point set simultaneously reflects local perception information, global task requirements, and future environmental change trends. The updated candidate target point set is then input into the subsequent comprehensive weight calculation module to redetermine the optimal target point for robot movement.

[0055] Through this approach, robot navigation decisions are no longer limited to local environmental assessments by a single robot. Instead, they can expand the scope of environmental awareness by utilizing global hotspot information provided by edge servers and adjust navigation direction in advance by combining dynamic obstacle prediction information. For example, in the collaborative exploration of multiple robots in a smart warehouse, when an area is identified as an important equipment area by the edge server, the robot can proactively increase its priority to reach candidate target points in that area. When another robot detects the movement trend of a forklift and uploads its prediction range, other robots can reduce their probability of selecting candidate target points in that area in advance. This enables information sharing and dynamic collaboration among multiple robots, improving overall navigation efficiency and environmental adaptability.

[0056] In one optional implementation, a collaborative agent acquires the location information, battery level, and current task priority of each robot; based on the location information of each robot and the spatial distribution of each candidate target point in the first candidate target point set, the exploration sub-regions of each robot are divided to generate an initial task allocation scheme for each robot; based on the battery level and task priority of each robot, the matching degree of the candidate target points allocated to each robot in the initial task allocation scheme is evaluated to generate a matching degree coefficient between each robot and the allocated candidate target points; based on the matching degree coefficient, the task allocation scheme of each robot is iteratively adjusted until the overlap rate of target points between each robot is lower than a preset threshold, thus obtaining the final task allocation scheme.

[0057] In this embodiment, the collaborative agent is used to coordinate tasks and allocate goals among multiple robots. It acquires the status information of each robot, including its current location, remaining battery power, and the priority of its current task. It also receives status data uploaded by each robot through an inter-robot communication interface. The location information describes the robot's current spatial location, the battery power reflects the robot's resource capacity to continuously execute tasks, and the task priority indicates the importance of the robot's current task. By acquiring this information, the collaborative agent can establish a comprehensive state model that includes robot status, environmental goals, and task requirements.

[0058] After acquiring the status information of each robot, the collaborative agent divides the exploration sub-regions for multiple robots based on their current positions and the spatial distribution of candidate target points in the first set of candidate target points. Exploration sub-region division refers to assigning different candidate target points to the exploration areas of corresponding robots based on the spatial distance between the robot's current position and the candidate target points, the target point distribution density, and the area coverage, enabling different robots to perform target exploration tasks in different areas.

[0059] During the partitioning process, the collaborative agent analyzes the spatial relationship between each robot's current position and candidate target points. It prioritizes assigning candidate target points that are closer to a particular robot and not covered by other robots to that robot, thus generating an initial task allocation scheme. This initial scheme records the set of candidate target points for each robot and the area it is responsible for exploring. For example, in a smart warehouse environment, robot A is located at the warehouse entrance, and robot B is located inside the warehouse. The first set of candidate target points contains multiple target points distributed across different shelf areas. Based on the robot's position, the collaborative agent assigns target points closer to the entrance shelf area to robot A and target points closer to the internal shelf area to robot B, forming the initial task allocation scheme.

[0060] After generating the initial task allocation scheme, the collaborative agent further evaluates the matching degree between the robots and the assigned candidate target points by combining the battery power and task priority of each robot. The matching degree coefficient is used to represent the degree of adaptation between the robot's current state and the assigned target point, and it comprehensively reflects whether the robot has the ability to complete the corresponding target task. When the robot has sufficient battery power, is currently close to the target point, and the priority of the task undertaken by the robot matches the importance of the target area, the matching degree coefficient between the robot and the candidate target point is high; when the robot has low remaining battery power, is far from the target point, or the current task priority is lower than the task requirements of the target area, the matching degree coefficient decreases. For example, in the intelligent warehouse inspection scenario, robot A has 80% battery power and is currently close to the faulty equipment area, while robot B has only 20% battery power. Although robot B is close to the area, it does not have enough battery power to complete the inspection task and return to the charging area. Therefore, the collaborative agent will reduce the matching degree coefficient between robot B and the target area.

[0061] Based on the calculated matching coefficient, the collaborative agent iteratively adjusts the initial task allocation scheme. Iterative adjustment refers to continuously recalculating the matching relationship between robots and candidate target points based on the current task allocation results, and adjusting the target point affiliation to make the task allocation among multiple robots more reasonable. During the adjustment process, the collaborative agent detects the overlap of target points between different robots. When multiple robots are assigned to the same or too close candidate target points, the target point overlap rate is calculated. The target point overlap rate represents the degree of repetition between the task areas of multiple robots. When the target point overlap rate exceeds a preset threshold, the collaborative agent readjusts the affiliation of some candidate target points, reassigning duplicate target points to robots with higher matching coefficients, until the target point overlap rate among all robots is below the preset threshold, ultimately forming the task allocation scheme. Through this method, the collaborative agent can dynamically optimize task allocation based on robot status, task requirements, and target point spatial relationships, achieving collaborative navigation and rational resource utilization among multiple robots.

[0062] In one optional implementation, the collaborative agents of each robot monitor the spatial distance between the target robot's current target point and the target points of other robots in real time. When the spatial distance is less than a preset safe distance threshold, the collaborative agents obtain the target robot's current task priority and the current task priorities of other robots. The collaborative agents compare the target robot's current task priority with the current task priorities of other robots. When the target robot's current task priority is higher than the current task priorities of other robots, the local collaborative agents maintain the target robot's current target point unchanged and send a target point adjustment request to other robots. When the target robot's current task priority is lower than the current task priorities of other robots, the local collaborative agents select the highest priority unoccupied candidate target point from the first candidate target point set as the updated target point.

[0063] In this embodiment, during multi-robot cooperative navigation, the cooperative agent needs to continuously monitor the target point relationships between different robots to avoid path conflicts, resource waste, or communication congestion caused by multiple robots simultaneously heading to the same area. Specifically, each robot's cooperative agent obtains the current planned target point position of the target robot and the target point positions of other robots in real time, and calculates the spatial distance between robots based on the spatial coordinates corresponding to the target points. The spatial distance represents the interval between the expected arrival positions of different robots and reflects whether multiple robots are likely to enter the same area during subsequent movement. When the spatial distance between the target points of two robots is less than a preset safe distance threshold, it indicates that there is a potential conflict between the navigation targets of the two robots, and further task priority judgment is required.

[0064] When the distance to the target point is detected to be less than the safe distance threshold, the collaborative agent further obtains the priority information of the target robot and other robots' currently executed tasks. Task priority indicates the importance of the robot's current task and is derived from the task association information generated by the task parsing agent. For example, an emergency equipment fault detection task corresponds to a higher priority, while a normal environment exploration task corresponds to a lower priority. The collaborative agent obtains the task priorities of each robot through information exchange between robots and compares the target robot's current task priority with the current task priorities of other robots to determine the task retention strategy in the event of a target point conflict.

[0065] When a target robot's task priority is higher than other robots, the local collaboration agent keeps the target robot's current target point unchanged and sends target point adjustment requests to other robots. Target point adjustment requests are used to notify other robots to replan their movement targets, allowing high-priority tasks to be executed first. The local collaboration agent is a collaborative control module operating near a single robot, used to quickly adjust the robot's local navigation behavior based on real-time environmental changes. For example, when intelligent warehouse robots are performing inspection tasks, robot A is heading towards an area with a malfunctioning equipment, while robot B is heading towards a nearby area for general environmental exploration; their target points are too close. Since robot A is performing a fault detection task, its priority is higher than robot B's. Therefore, robot A keeps its current target point unchanged, and robot B, upon receiving the adjustment request, reselects an alternative target point, preventing both robots from entering the same area simultaneously.

[0066] When the target robot's task priority is lower than that of other robots, the local collaborative agent reselects a moving target based on the first set of candidate target points. During the selection process, the local collaborative agent first eliminates candidate target points that are already occupied by other robots or pose a risk of spatial conflict. Then, it sorts the candidate target points according to their comprehensive weight coefficients and selects the highest-priority target point from the remaining candidate target points as the updated target point. This target point satisfies the current robot's navigation needs while avoiding the task areas of higher-priority robots. Through this method, the collaborative agent can dynamically coordinate the target allocation relationship between robots based on task priority, enabling the multi-robot system to maintain task execution efficiency and navigation safety even in the event of spatial conflicts.

[0067] According to the navigation decision-making method based on deep learning and intelligent agents provided in this application, RGB and depth images of the current environment are collected, along with the robot's battery level and user-inputted task commands. An environmental assessment agent analyzes the RGB and depth images to determine obstacle information in the current environment, and a task parsing agent parses the task commands to obtain task-related information. A first set of candidate target points is generated based on the RGB and depth images. The obstacle information, battery level, task-related information, and the first set of candidate target points are input into a preset reinforcement learning model for comprehensive evaluation to determine the optimal target point for the robot to move from the first set of candidate target points. By implementing this application, the environmental assessment agent processes RGB and depth images to obtain obstacle information, the task parsing agent parses task commands to obtain task-related information, and the battery level and candidate target points are combined and input into a reinforcement learning model for comprehensive evaluation. This achieves multi-source heterogeneous information fusion decision-making, effectively avoiding decision oscillations caused by single-sensor failure or task changes, thereby improving the robustness of the navigation system in dynamic environments.

[0068] Figure 3 This application provides a navigation decision-making device based on deep learning and intelligent agents, which can be used to implement the navigation decision-making method based on deep learning and intelligent agents in the aforementioned embodiments. Figure 3 As shown, this navigation decision-making device based on deep learning and intelligent agents mainly includes: The acquisition module 10 is used to acquire RGB and depth images of the current environment, and to obtain the robot's device battery level and user-inputted task commands; The analysis module 20 is used to analyze RGB and depth images through the environment assessment agent to determine obstacle information in the current environment, and to analyze task instructions through the task analysis agent to obtain task-related information. Generation module 30 is used to generate a first set of candidate target points based on the RGB image and the depth image; The decision module 40 is used to input obstacle information, device power, task association information and the first candidate target point set into a preset reinforcement learning model for comprehensive evaluation, and determine the optimal target point for the robot to move in the first candidate target point set.

[0069] In one optional implementation, the analysis module is specifically used to: perform semantic segmentation processing on the RGB image using an environment assessment agent to generate a pixel-level semantic label map, wherein the semantic label map is labeled with at least passable areas, obstacle categories, and dynamic target categories; perform inverse projection transformation processing on the depth image using the environment assessment agent to generate three-dimensional point cloud data spatially aligned with the semantic label map; extract a subset of dynamic target point clouds from the three-dimensional point cloud data based on the dynamic target categories labeled in the semantic label map, and perform inter-frame difference calculation on the point cloud subsets of multiple consecutive frames to generate motion vector information for each dynamic target; and generate a static obstacle distribution map based on the obstacle categories in the semantic label map and the three-dimensional point cloud data, and combine the motion vector information to generate obstacle information containing the static obstacle positions and dynamic obstacle movement trends.

[0070] In an optional implementation, the analysis module is further specifically used to: perform instruction type matching processing on the task instructions through the task parsing agent, match the task instructions with a preset instruction template library, and determine the task type corresponding to the task instructions; query the priority level associated with the task type from a preset priority mapping table, and generate the corresponding priority quantification coefficient according to the priority level; associate and bind the task type with the priority quantification coefficient to generate task association information containing type identifier and priority coefficient.

[0071] In one optional implementation, the generation module is specifically used to: generate RGB and depth images of the sample environment through an environment simulator, and generate a first set of candidate target points based on the RGB and depth images; randomly select a target point from the first set of candidate target points, and control the robot to move according to the randomly selected target point; acquire map data of the explored and unexplored areas after the robot moves, record the distance and time the robot travels to the target point, and acquire the area of ​​the newly added unexplored area after the move; update the local map based on the map data of the explored and unexplored areas, and perform a weighted evaluation on each candidate target point in the first set of candidate target points based on the distance traveled, the time traveled, and the area of ​​the newly added unexplored area to generate an evaluation weight for each candidate target point; and sort the candidate target points according to their evaluation weights to generate the first set of candidate target points.

[0072] In an optional implementation, the decision-making module further includes: a generation unit, configured to calculate the task correlation degree of each candidate target point in the first candidate target point set based on the priority quantification coefficient in the task correlation information, and generate a target correlation degree coefficient between each candidate target point and the task target area; an evaluation unit, configured to evaluate the path safety of each candidate target point based on the dynamic obstacle movement trend information in the obstacle information, and generate a path safety coefficient corresponding to each candidate target point; an analysis unit, configured to perform path resource consumption analysis on each candidate target point based on the device power consumption, and generate a resource consumption rate coefficient corresponding to each candidate target point; and a determination unit, configured to determine the comprehensive weight coefficient of each candidate target point based on the target correlation coefficient, the path safety coefficient, and the resource consumption rate coefficient, and to rank each candidate target point according to the comprehensive weight coefficient, and determine the candidate target point with the highest comprehensive weight coefficient as the optimal target point for robot movement.

[0073] In one optional implementation, the navigation decision-making device based on deep learning and intelligent agents further includes an update module. The update module is configured to: receive global hotspot area information from an edge-side server via an environment assessment agent; perform weight enhancement processing on candidate target points located within the global hotspot area in the first candidate target point set based on the global hotspot area information to generate a second candidate target point set; receive predicted movement ranges of dynamic obstacles uploaded by an edge-side server via an environment assessment agent; update the weights of candidate target points falling within the predicted movement range in the second candidate target point set based on the predicted movement range to generate a third candidate target point set; and update the first candidate target point set based on the third candidate target point set before determining the next optimal target point.

[0074] According to the navigation decision-making device based on deep learning and intelligent agents provided in this application, RGB and depth images of the current environment are collected, along with the robot's battery level and user-inputted task commands. An environmental assessment agent analyzes the RGB and depth images to determine obstacle information in the current environment, and a task parsing agent parses the task commands to obtain task-related information. A first set of candidate target points is generated based on the RGB and depth images. The obstacle information, battery level, task-related information, and the first set of candidate target points are input into a preset reinforcement learning model for comprehensive evaluation to determine the optimal target point for the robot to move from the first set of candidate target points. By implementing this application, the environmental assessment agent processes RGB and depth images to obtain obstacle information, the task parsing agent parses task commands to obtain task-related information, and the battery level and candidate target points are combined and input into a reinforcement learning model for comprehensive evaluation. This achieves multi-source heterogeneous information fusion decision-making, effectively avoiding decision oscillations caused by single-sensor failure or task changes, thereby improving the robustness of the navigation system in dynamic environments.

[0075] According to the scheme provided in this application Figure 4 An electronic device is provided as an embodiment of this application. This electronic device can be used to implement the navigation decision-making method based on deep learning and intelligent agents in the foregoing embodiments, and mainly includes: The system includes a memory 401, a processor 402, and a computer program 403 stored on the memory 401 and executable on the processor 402. The memory 401 and the processor 402 are communicatively connected. When the processor 402 executes the computer program 403, it implements the navigation decision-making method based on deep learning and intelligent agents described in the foregoing embodiments. The number of processors can be one or more.

[0076] The memory 401 can be a high-speed random access memory (RAM) or a non-volatile memory, such as a disk storage device. The memory 401 is used to store executable program code, and the processor 402 is coupled to the memory 401.

[0077] Furthermore, embodiments of this application also provide a computer-readable storage medium, which may be disposed in the electronic device described in the above embodiments, and the computer-readable storage medium may be as described above. Figure 4 The memory in the illustrated embodiment.

[0078] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the navigation decision-making method based on deep learning and intelligent agents described in the foregoing embodiments. Furthermore, the computer-readable storage medium can also be a USB flash drive, external hard drive, read-only memory (ROM), RAM, magnetic disk, or optical disk, or any other medium capable of storing program code.

[0079] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A navigation decision-making method based on deep learning and intelligent agents, characterized in that, include: Acquire RGB and depth images of the current environment, and obtain the robot's battery level and user-inputted task commands; The environmental assessment agent analyzes the RGB image and the depth image to determine the obstacle information in the current environment, and the task analysis agent analyzes the task instructions to obtain task association information. A first set of candidate target points is generated based on the RGB image and the depth image; The obstacle information, the device battery level, the task association information, and the first candidate target point set are input into a preset reinforcement learning model for comprehensive evaluation to determine the optimal target point for the robot to move from the first candidate target point set.

2. The navigation decision-making method based on deep learning and intelligent agents according to claim 1, characterized in that, The step of analyzing the RGB image and the depth image using an environmental assessment agent to determine obstacle information in the current environment includes: The RGB image is semantically segmented by an environmental assessment agent to generate a pixel-level semantic label map, which is labeled with at least passable areas, obstacle categories and dynamic target categories. The environmental assessment agent performs inverse projection transformation on the depth image to generate 3D point cloud data aligned with the semantic label graph space. Based on the dynamic target categories marked in the semantic tag map, a subset of the dynamic target's point cloud is extracted from the 3D point cloud data, and inter-frame difference calculation is performed on the point cloud subsets of multiple consecutive frames to generate motion vector information of each dynamic target. Based on the obstacle categories in the semantic tag map and the 3D point cloud data, a static obstacle distribution map is generated, and combined with the motion vector information, obstacle information containing the static obstacle positions and dynamic obstacle movement trends is generated.

3. The navigation decision-making method based on deep learning and intelligent agents according to claim 1, characterized in that, The step of parsing the task instructions through the task parsing agent to obtain task association information includes: The task parsing agent performs instruction type matching processing on the task instruction, matching the task instruction with a preset instruction template library to determine the task type corresponding to the task instruction; The priority level associated with the task type is queried from the preset priority mapping table, and the corresponding priority quantization coefficient is generated according to the priority level. The task type is associated with the priority quantification coefficient to generate task association information containing type identifier and priority coefficient.

4. The navigation decision-making method based on deep learning and intelligent agents according to claim 1, characterized in that, The step of generating a first candidate target point set based on the RGB image and the depth image includes: An RGB image and a depth image of the sample environment are generated using an environment simulator, and a first set of candidate target points for the sample is generated based on the RGB image and the depth image. Randomly select a sample target point from the first set of candidate target points, and control the robot to move according to the randomly selected sample target point; Obtain map data of the explored and unexplored areas after the robot moves, record the distance and time the robot travels to the target point, and obtain the area of ​​the newly added unexplored area after the move; The local map is updated based on the map data of the explored and unexplored areas, and the candidate target points in the first candidate target point set are weighted and evaluated based on the travel distance, the travel time, and the area of ​​the newly added unexplored area to generate the evaluation weight of each candidate target point. The candidate target points are sorted according to their evaluation weights to generate the first candidate target point set.

5. The navigation decision-making method based on deep learning and intelligent agents according to claim 3, characterized in that, The step of inputting the obstacle information, the device battery level, the task association information, and the first candidate target point set into a preset reinforcement learning model for comprehensive evaluation, and determining the optimal target point for the robot to move from the first candidate target point set, includes: Based on the priority quantification coefficient in the task association information, the task association degree is calculated for each candidate target point in the first candidate target point set, and the target association degree coefficient between each candidate target point and the task target area is generated. Based on the dynamic obstacle movement trend information in the obstacle information, the path safety of each candidate target point is evaluated, and the path safety coefficient corresponding to each candidate target point is generated. Based on the device power consumption, path resource consumption analysis is performed on each candidate target point to generate resource consumption rate coefficients corresponding to each candidate target point. Based on the target correlation coefficient, the path safety coefficient, and the resource consumption rate coefficient, the comprehensive weight coefficient of each candidate target point is determined, and the candidate target points are sorted according to the comprehensive weight coefficient. The candidate target point with the highest comprehensive weight coefficient is determined as the optimal target point for robot movement.

6. The navigation decision-making method based on deep learning and intelligent agents according to claim 4, characterized in that, After the step of sorting the candidate target points according to their evaluation weights to generate a first set of candidate target points, the method further includes: The environment assessment agent receives global hotspot area information from the edge server and performs weight boosting on candidate target points located in the global hotspot area in the first candidate target point set based on the global hotspot area information to generate a second candidate target point set. The dynamic obstacle prediction range is uploaded by the receiving server of the environment assessment agent. The weights of the candidate target points in the second candidate target point set that fall within the predicted movement range are updated according to the predicted movement range to generate a third candidate target point set. Before the next optimal target point is determined, the first candidate target point set is updated based on the third candidate target point set.

7. A navigation decision-making device based on deep learning and intelligent agents, characterized in that, The navigation decision-making device based on deep learning and intelligent agents is used to implement the navigation decision-making method based on deep learning and intelligent agents as described in claim 1, wherein the navigation decision-making device based on deep learning and intelligent agents includes: The acquisition module is used to acquire RGB and depth images of the current environment, as well as obtain the robot's battery level and user-inputted task commands. The analysis module is used to analyze the RGB image and the depth image through the environment assessment agent to determine the obstacle information of the current environment, and to analyze the task instructions through the task analysis agent to obtain task association information; The generation module is used to generate a first set of candidate target points based on the RGB image and the depth image; The decision module is used to input the obstacle information, the device battery level, the task association information, and the first candidate target point set into a preset reinforcement learning model for comprehensive evaluation, and to determine the optimal target point for the robot to move from the first candidate target point set.

8. The navigation decision-making device based on deep learning and intelligent agents according to claim 7, characterized in that, The decision-making module includes: The generation unit is used to calculate the task association degree of each candidate target point in the first candidate target point set according to the priority quantization coefficient in the task association information, and generate the target association degree coefficient between each candidate target point and the task target area. The evaluation unit is used to evaluate the path safety of each candidate target point based on the dynamic obstacle movement trend information in the obstacle information, and generate the path safety coefficient corresponding to each candidate target point. The analysis unit is used to perform path resource consumption analysis on each candidate target point based on the power consumption of the device, and generate resource consumption rate coefficients corresponding to each candidate target point. The determining unit is used to determine the comprehensive weight coefficient of each candidate target point based on the target correlation coefficient, the path safety coefficient, and the resource consumption rate coefficient, and to sort the candidate target points according to the comprehensive weight coefficient, and to determine the candidate target point with the highest comprehensive weight coefficient as the optimal target point for robot movement.

9. An electronic device, characterized in that, Includes memory and processor, of which: The processor is used to execute computer programs stored in the memory; When the processor executes the computer program, it implements the steps in the navigation decision-making method based on deep learning and intelligent agents as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the navigation decision-making method based on deep learning and intelligent agents as described in any one of claims 1 to 6.

Images