Decision method and device of robot

Abstract

This application relates to the field of target search and tracking technology, and particularly to a decision-making method and apparatus for a robot. The method includes: generating search uncertainty and tracking uncertainty of the working environment based on graph structure information; simulating the movement process of the search target and the tracked target using a pre-built one-way entropy diffusion model to obtain simulation data, and introducing global abstract nodes to construct a two-level graph structure of the search target and the tracked target; generating a tunable reward function to construct a robot decision model; and using the decision model to output collaborative search and tracking decision actions. This solves the problems caused by the fact that related technologies typically rely on fixed search strategies and fixed perception models, which easily lead to insufficient consideration of uncertainties and dynamic changes in complex environments, resulting in insufficient or inaccurate acquisition of target position information during the search process, thereby reducing target search efficiency and increasing the risk of target loss.

Description

Technical Field

[0001] This application relates to the field of target tracking technology, and in particular to a decision-making method and apparatus for a robot. Background Technology

[0002] In related technologies, target search and tracking technologies typically rely on vision, infrared, lidar, acoustic or vital signs sensors mounted on robots to perceive environmental information and complete target area coverage based on set search strategies or path planning algorithms; when a potential target is detected, the target is continuously located through target recognition and tracking algorithms.

[0003] However, in related technologies, search strategies mostly adopt rule-based or static model-based planning methods, which have limited adaptability to environmental uncertainties and target motion characteristics. When there are multiple targets or targets are moving, robots have difficulty adjusting search and tracking strategies in a timely manner, which can easily lead to problems such as low search efficiency, target loss, or repeated searches. At the same time, the target search and tracking processes are usually independent of each other, lacking collaborative modeling of the search and tracking phases, making it difficult to fully utilize historical observation information to optimize subsequent search behavior, which urgently needs to be addressed. Summary of the Invention

[0004] This application provides a decision-making method and apparatus for robots to solve the problem that in related technologies, target search and tracking technologies usually rely on fixed search strategies and fixed perception models, which easily leads to insufficient consideration of uncertainties and dynamic changes in the complex environment, resulting in insufficient or inaccurate acquisition of target position information during the search process, thereby reducing target search efficiency and increasing the risk of target loss.

[0005] The first aspect of this application provides a decision-making method for a robot, comprising the following steps: generating graph structure information based on the robot's operating environment, and generating search uncertainty and tracking uncertainty of the operating environment based on the graph structure information; simulating the movement process of at least one search target and at least one tracking target using a pre-built unidirectional entropy diffusion model to obtain simulation data, and introducing global abstract nodes to construct a two-level graph structure of the at least one search target and the at least one tracking target; generating a tunable reward function based on the search uncertainty, the tracking uncertainty, the simulation data, and the two-level graph structure to construct a decision model for the robot, and using the decision model to output decision actions for collaborative search and tracking.

[0006] Through the above technical means, the embodiments of this application can generate graph structure information according to the robot's working environment, and combine it with a pre-built one-way entropy diffusion model and a two-level graph structure to construct the robot's decision model, so as to output decision actions for collaborative search and tracking. It can uniformly model the spatial connectivity, target state information and uncertainty information in the environment, and continuously update the graph structure information during task execution, thereby achieving collaborative search and tracking even when the number of targets and the initial position of the targets are unknown. At the same time, by introducing a propagation and update mechanism for uncertainty information into the graph structure, the robot can adaptively adjust its search and tracking strategy according to the dynamic changes of the target and the environment, thereby improving the stability, efficiency and robustness of the robot in performing collaborative search and tracking tasks in complex working environments.

[0007] Optionally, in one embodiment of this application, after constructing the two-level graph structure of the at least one search target and the at least one tracking target, the method further includes: aggregating the first edge between the spatial nodes of the two-level graph structure using a preset graph convolutional network operator to obtain the local topological information of the first edge; and extracting the second edge between the spatial node and the global abstract node using a preset graph attention network operator to obtain global information related to the current decision.

[0008] Through the above technical means, the embodiments of this application can use graph convolution network operators and graph attention network operators to extract local information and global information respectively, introduce a heterogeneous message passing mechanism, realize collaborative interaction between different types of nodes and different levels of information, thereby achieving effective fusion of local information and global information, forming a unified representation of the environmental state, and providing more comprehensive and accurate input for collaborative search and tracking decisions.

[0009] Optionally, in one embodiment of this application, the search uncertainty and tracking uncertainty of the operating environment are expressed as follows: , in, This indicates uncertainty in the working environment. This indicates the uncertainty of the search. This indicates the tracking uncertainty.

[0010] Through the above technical means, the embodiments of this application can determine the uncertainty of the working environment based on the search uncertainty and the tracking uncertainty. Taking into account factors such as target motion characteristics, sensor observation errors and dynamic changes in the environment, the uncertainty of the environmental state is quantified and updated, thereby providing a basis for subsequent graph structure modeling, uncertainty propagation and collaborative search and tracking decisions.

[0011] Optionally, in one embodiment of this application, the expression of the pre-built unidirectional entropy diffusion model is: , in, It is the Laplace matrix of the graph. It is the diffusion coefficient related to the target estimation velocity. , They represent Nodes in the time-space graph ,node The search uncertainty entropy value.

[0012] Through the above technical means, the embodiments of this application can use the one-way entropy diffusion model to propagate and update uncertain information in the working environment, and spread the information of high uncertainty areas in the environment to adjacent nodes in the graph structure along a predetermined direction, so as to depict the evolution process of uncertainty with spatial and temporal changes. It can avoid the ineffective backflow of uncertain information while ensuring the orderly propagation of information, thereby realizing effective modeling of search uncertainty and tracking uncertainty, and providing a stable and interpretable environmental uncertainty representation for the decision model of collaborative search and tracking.

[0013] Optionally, in one embodiment of this application, the expression for the tunable reward function is: , in, This represents a tunable reward function. , These represent the weights for search and tracking, respectively. Indicates search uncertainty. This indicates tracking uncertainty.

[0014] Through the above technical means, the embodiments of this application can dynamically adjust the contribution ratio of different task objectives in the reward function by adjusting the search weight and tracking weight, thereby guiding the robot to make adaptive trade-offs between search behavior and tracking behavior at different task stages, enabling the decision model to better adapt to environmental changes and task requirements, and improving the flexibility and overall execution effect of the collaborative search and tracking process.

[0015] A second aspect of this application provides a decision-making device for a robot, comprising: a generation module, configured to generate graph structure information based on the robot's operating environment, to generate search uncertainty and tracking uncertainty of the operating environment based on the graph structure information; a construction module, configured to simulate the movement process of at least one search target and at least one tracking target using a pre-built unidirectional entropy diffusion model to obtain simulation data, and to introduce global abstract nodes to construct a two-level graph structure of the at least one search target and the at least one tracking target; and a decision module, configured to generate a tunable reward function based on the search uncertainty, the tracking uncertainty, the simulation data, and the two-level graph structure to construct a decision model for the robot, and to output a collaborative search and tracking decision action using the decision model.

[0016] Optionally, in one embodiment of this application, it further includes: an aggregation module, used to aggregate the first edge between the spatial nodes of the two-level graph structure using a preset graph convolutional network operator to obtain local topological information of the first edge; and an extraction module, used to extract the second edge between the spatial node and the global abstract node using a preset graph attention network operator to obtain global information related to the current decision.

[0017] Optionally, in one embodiment of this application, the search uncertainty and tracking uncertainty of the operating environment are expressed as follows: , in, This indicates uncertainty in the working environment. This indicates the uncertainty of the search. This indicates the tracking uncertainty.

[0018] Optionally, in one embodiment of this application, the expression of the pre-built unidirectional entropy diffusion model is: , in, It is the Laplace matrix of the graph. It is the diffusion coefficient related to the target estimation velocity. , They represent Nodes in the time-space graph ,node The search uncertainty entropy value.

[0019] Optionally, in one embodiment of this application, the expression for the tunable reward function is: , in, This represents a tunable reward function. , These represent the weights for search and tracking, respectively. Indicates search uncertainty. This indicates tracking uncertainty.

[0020] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the robot decision-making method as described in the above embodiments.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the robot decision-making method described above.

[0022] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the robot decision-making method described above.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a robot decision-making method according to an embodiment of this application; Figure 2 This is a schematic diagram of the graphical structure information of the operating environment according to one embodiment of this application; Figure 3 This is a schematic diagram of a two-level diagram structure according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating the principle of a GNN (Graph Neural Network) model according to an embodiment of this application. Figure 5 A flowchart illustrating a robot decision-making method according to an embodiment of this application; Figure 6 This is a block diagram of a decision-making device for a robot according to an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0025] Figure label: 10 - Robot decision-making device; 100 - Generation module, 200 - Construction module, 300 - Decision module; 701 - Memory, 702 - Processor, 703 - Communication interface. Detailed Implementation

[0026] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0027] The following description, with reference to the accompanying drawings, illustrates a robot decision-making method and apparatus according to embodiments of this application. Addressing the technical problem mentioned in the background art, where target search and tracking technologies typically rely on fixed search strategies and fixed perception models, making it difficult to fully consider uncertainties and dynamic changes in complex environments, resulting in insufficient or inaccurate acquisition of target location information during the search process, thereby reducing target search efficiency and increasing the risk of target loss, this application provides a robot decision-making method. In this method, graph structure information is generated based on the robot's operating environment, and a decision model for the robot is constructed by combining a pre-built unidirectional entropy diffusion model and a two-level graph structure to output collaborative search and tracking decision actions. This method allows the task to be initiated even when the number of targets and their initial positions are completely unknown. Furthermore, the environmental information is continuously modeled and updated through the graph structure and entropy diffusion mechanism, effectively completing the search and tracking task. Simultaneously, by explicitly modeling the impact of target movement on the search process, the robot can intelligently revisit relevant areas based on dynamic changes in the target, significantly improving search efficiency and task success rate in the operating environment. Moreover, the decision-making strategy can be dynamically adjusted according to environmental changes, thereby improving the stability, efficiency, and robustness of the robot in performing search and tracking tasks in complex operating environments. This solves the problem that target search and tracking technologies typically rely on fixed search strategies and fixed perception models, which can easily lead to insufficient consideration of uncertainties and dynamic changes in complex environments, resulting in insufficient or inaccurate acquisition of target location information during the search process, thereby reducing target search efficiency and increasing the risk of target loss.

[0028] Specifically, Figure 1 This is a flowchart illustrating a robot decision-making method provided in an embodiment of this application.

[0029] like Figure 1 As shown, the robot's decision-making method includes the following steps: In step S101, graph structure information is generated based on the robot's operating environment, so as to generate search uncertainty and tracking uncertainty of the operating environment based on the graph structure information.

[0030] Among them, search uncertainty can refer to the fact that in the working environment, the existence of the target, the spatial distribution of the target, and the state of the environment cannot be accurately known in advance, thus resulting in uncertainty in the search results; tracking uncertainty can refer to the fact that in the working environment, due to changes in the target's movement behavior and discontinuous observation information, the estimation of the target's state is biased and unstable.

[0031] As a concrete example, such as Figure 2 As shown in the embodiments of this application, the two-dimensional environment of robot operation can be abstracted into a graph structure. vertex set An edge set represents a set of reachable waypoints in the environment. This represents the spatial connectivity between waypoints. The graph structure can include, but is not limited to, robots, targets, obstacles, and observed environmental information, thus characterizing the spatial relationships between the robot and environmental elements.

[0032] Based on this, the embodiments of this application can provide a unified description of the dual uncertainties of the operating environment: the embodiments of this application can describe each graph node Associated with a Bernoulli random variable Its entropy This can characterize the probability uncertainty of the existence of undiscovered targets at this node, i.e., search uncertainty, and its updates typically follow the following: , in: express The dynamic changes.

[0033] Meanwhile, embodiments of this application can be applied to each discovered target. Maintain a belief state that is discretely distributed on the graph. Its entropy It can characterize the tracking uncertainty of target position estimation.

[0034] Optionally, in one embodiment of this application, the search uncertainty and tracking uncertainty of the operating environment can be expressed as: , in, This indicates uncertainty in the working environment. Indicates search uncertainty. Indicates tracking uncertainty. .

[0035] The embodiments of this application can integrate robot state, target information and environmental constraint information under the same structure based on a unified graph representation, thereby providing basic support for subsequent collaborative decision-making and path planning.

[0036] In step S102, the movement process of at least one search target and at least one tracking target is simulated using a pre-built unidirectional entropy diffusion model to obtain simulation data, and a global abstract node is introduced to construct a two-level graph structure of at least one search target and at least one tracking target.

[0037] It can be noted that the pre-constructed one-way entropy diffusion model can be a probabilistic entropy diffusion model based on a graph structure or a learning entropy diffusion model based on a neural network. It can propagate and update information entropy along the edge direction of the graph structure, thereby simulating the spatial diffusion process of uncertain information in the environment. It can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0038] Global abstract nodes can be used to characterize the distribution of uncertain information in the environment at a global scale, supporting the global propagation and updating of a pre-built unidirectional entropy diffusion model. A two-level graph structure can be a graph structure that models environmental and task information at the global and local levels respectively, and achieves information interaction through cross-layer connections.

[0039] Optionally, in one embodiment of this application, the expression of the pre-constructed one-way entropy diffusion model can be: , in, It is the Laplace matrix of the graph. It is the diffusion coefficient related to the target estimation velocity. , They represent Nodes in the time-space graph ,node The search uncertainty entropy value.

[0040] It is understood that the core of the pre-built unidirectional entropy diffusion model in this application embodiment lies in its unidirectionality. It can ensure that uncertainty (entropy) can only "diffuse" from high-value nodes to adjacent low-value nodes, thereby simulating the process of an undiscovered target moving from an unexplored area to an explored area. This allows the robot to not only explore unknown areas, but also to intelligently revisit explored areas to discover targets that may have moved there, thereby overcoming the limitations of traditional static search models.

[0041] Optionally, in one embodiment of this application, after constructing a two-level graph structure of at least one search target and at least one tracking target, the method further includes: aggregating the first edge between the spatial nodes of the two-level graph structure using a preset graph convolutional network operator to obtain the local topological information of the first edge; and extracting the second edge between the spatial node and the global abstract node using a preset graph attention network operator to obtain global information related to the current decision.

[0042] Specifically, such as Figure 3 As shown, this embodiment first introduces a global abstract node on top of the basic environment graph to construct a two-layer hierarchical graph structure, thereby achieving cross-regional global information aggregation. More importantly, this embodiment can adopt a heterogeneous message passing architecture: for edges between spatial nodes, the GCN (Graph Convolutional Network) operator is used to effectively aggregate local topological information; for edges between spatial nodes and the global abstract node, the GATv2 (Graph Attention Network v2) operator can be used. This operator uses the robot's local neighborhood information as a "query" to calculate dynamic attention weights on all spatial nodes, thereby extracting the global information most relevant to the current decision.

[0043] In step S103, a tunable reward function is generated based on search uncertainty, tracking uncertainty, simulation data, and a two-level graph structure to construct the robot's decision model. The decision model is then used to output collaborative search and tracking decision actions.

[0044] In the embodiments of this application, the tunable reward function can be its constituent items, weights, or calculation methods, and is a reward function that is dynamically adjusted according to the task stage, environmental state, or level of uncertainty. The robot's decision-making model can be a decision-making model based on analytical modeling or a data-driven decision-making model, which can generate decision results based on environmental state information, thereby improving the flexibility and adaptability of robot decision-making.

[0045] Optionally, in one embodiment of this application, the expression for the tunable reward function can be: , in, This represents a tunable reward function. , These represent the weights for search and tracking, respectively. Indicates search uncertainty. This indicates tracking uncertainty.

[0046] As one possible way to achieve this, such as Figure 4As shown, where, The features of the neighboring nodes output by the first layer. The features of the neighboring nodes are the output of the second layer. This represents the location node features output from the third layer. This embodiment of the application can decouple the benefits of search (environmental entropy reduction) from tracking (target belief entropy reduction) and use weights... and This enables flexible task trade-offs. Furthermore, based on this reward function, embodiments of this application can use algorithms such as proximal policy optimization to train the GNN decision-maker end-to-end. After training, sensor data can be converted into graph observations in real time, and the GNN decision-maker can directly output the next action primitive, thereby realizing real-time decision-making in collaborative search and tracking tasks.

[0047] like Figure 5 As shown below, a specific example will be used to further illustrate the decision-making method of the robot in this application embodiment.

[0048] In the embodiments of this application, at time A robot can obtain an action primitive from a decision-making strategy. After performing this action, the states of the robot and the target evolve into... For new observations The observed values ​​can be updated to The observed graph is first processed by the GNN model to extract learned features from the robot's neighboring nodes (up to 4 neighboring nodes in the case of a square lattice) and then sent to the FC (Fully Connected) layer to obtain the logits of different actions. , , , Then, using the softmax function, the distribution of the discrete action space is derived, so that the corresponding decision action can be selected based on the probability distribution.

[0049] The robot decision-making method proposed in this application generates graph structure information based on the robot's working environment and constructs a robot decision model by combining a pre-built unidirectional entropy diffusion model and a two-level graph structure. This model outputs collaborative search and tracking decision actions, enabling the robot to initiate tasks even when the number and initial position of targets are completely unknown. Furthermore, the robot continuously models and updates environmental information through graph structure and entropy diffusion mechanisms, effectively completing search and tracking tasks. Simultaneously, by explicitly modeling the impact of target movement on the search process, the robot can intelligently revisit relevant areas based on dynamic changes in targets, significantly improving search efficiency and task success rate in the working environment. Moreover, the decision-making strategy can be dynamically adjusted according to environmental changes, thereby improving the stability, efficiency, and robustness of the robot in performing search and tracking tasks in complex working environments.

[0050] Next, the decision-making device for a robot according to an embodiment of this application is described with reference to the accompanying drawings.

[0051] Figure 6 This is a block diagram of the decision-making device of a robot according to an embodiment of this application.

[0052] like Figure 6 As shown, the robot's decision-making device 10 includes: a generation module 100, a construction module 200, and a decision-making module 300.

[0053] The generation module 100 is used to generate graph structure information based on the robot's operating environment, so as to generate search uncertainty and tracking uncertainty of the operating environment based on the graph structure information.

[0054] Module 200 is used to simulate the movement process of at least one search target and at least one tracking target using a pre-built unidirectional entropy diffusion model to obtain simulation data, and to introduce global abstract nodes to construct a two-level graph structure of at least one search target and at least one tracking target.

[0055] The decision module 300 is used to generate a tunable reward function based on search uncertainty, tracking uncertainty, simulation data and a two-level graph structure to build a decision model for the robot, and to output collaborative search and tracking decision actions using the decision model.

[0056] Optionally, in one embodiment of this application, it further includes an aggregation module and an extraction module.

[0057] The aggregation module is used to aggregate the first edge between spatial nodes of a two-level graph structure using a preset graph convolutional network operator to obtain the local topology information of the first edge.

[0058] The extraction module is used to extract the second edge between spatial nodes and global abstract nodes using a preset graph attention network operator to obtain global information related to the current decision.

[0059] Optionally, in one embodiment of this application, the expressions for the search uncertainty and tracking uncertainty of the operating environment are: , in, This indicates uncertainty in the working environment. Indicates search uncertainty. This indicates tracking uncertainty.

[0060] Optionally, in one embodiment of this application, the expression of the pre-constructed one-way entropy diffusion model is: , in, It is the Laplace matrix of the graph. It is the diffusion coefficient related to the target estimation velocity. , They represent Nodes in the time-space graph ,node The search uncertainty entropy value.

[0061] Optionally, in one embodiment of this application, the expression for the tunable reward function is: , in, This represents a tunable reward function. , These represent the weights for search and tracking, respectively. Indicates search uncertainty. This indicates tracking uncertainty.

[0062] It should be noted that the foregoing explanation of the robot decision-making method embodiment also applies to the robot decision-making device of this embodiment, and will not be repeated here.

[0063] The robot decision-making device proposed in this application generates graph structure information based on the robot's working environment and constructs a robot decision model by combining a pre-built unidirectional entropy diffusion model and a two-level graph structure. This model outputs collaborative search and tracking decision actions, enabling the robot to initiate tasks even when the number of targets and their initial positions are completely unknown. Furthermore, it continuously models and updates environmental information through graph structure and entropy diffusion mechanisms, effectively completing search and tracking tasks. Simultaneously, by explicitly modeling the impact of target movement on the search process, the robot can intelligently revisit relevant areas based on dynamic changes in the targets, significantly improving search efficiency and task success rate in the working environment. Moreover, it can dynamically adjust decision strategies according to environmental changes, thereby improving the stability, efficiency, and robustness of the robot in performing search and tracking tasks in complex working environments.

[0064] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 701, the processor 702, and the computer program stored on the memory 701 and executable on the processor 702.

[0065] When the processor 702 executes the program, it implements the robot decision-making method provided in the above embodiments.

[0066] Furthermore, electronic devices also include: Communication interface 703 is used for communication between memory 701 and processor 702.

[0067] The memory 701 is used to store computer programs that can run on the processor 702.

[0068] The memory 701 may include high-speed RAM memory, and may also include NVM (Non-Volatile Memory), such as at least one disk storage device.

[0069] If the memory 701, processor 702, and communication interface 703 are implemented independently, then the communication interface 703, memory 701, and processor 702 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0070] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0071] The processor 702 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0072] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the robot's decision-making method as described above.

[0073] This application also provides a computer program product, including a computer program that can run computer instructions. When the computer instructions are executed by a processor, they implement the robot decision-making method provided in this application.

[0074] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0075] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0076] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0077] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0078] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0079] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0080] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0081] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A decision-making method for a robot, characterized in that, Includes the following steps: Generate graph structure information based on the robot's operating environment, and generate search uncertainty and tracking uncertainty of the operating environment based on the graph structure information; The movement process of at least one search target and at least one tracking target is simulated using a pre-built unidirectional entropy diffusion model to obtain simulation data, and a global abstract node is introduced to construct a two-level graph structure of the at least one search target and the at least one tracking target. A tunable reward function is generated based on the search uncertainty, the tracking uncertainty, the simulation data, and the two-level graph structure to construct the robot's decision model. The decision model is then used to output collaborative search and tracking decision actions.

2. The method according to claim 1, characterized in that, After constructing the two-level graph structure of the at least one search target and the at least one tracking target, the method further includes: A preset graph convolutional network operator is used to aggregate the first edge between the spatial nodes of the two-level graph structure to obtain the local topological information of the first edge; A preset graph attention network operator is used to extract the second edge between the spatial node and the global abstract node to obtain global information related to the current decision.

3. The method according to claim 1, characterized in that, The expressions for the search uncertainty and tracking uncertainty of the operating environment are: ， in, This indicates uncertainty in the working environment. This indicates the uncertainty of the search. This indicates the tracking uncertainty.

4. The method according to claim 1, characterized in that, The expression for the pre-constructed one-way entropy diffusion model is: ， in, It is the Laplace matrix of the graph. It is the diffusion coefficient related to the target estimation velocity. , They represent Nodes in the time-space graph ,node The search uncertainty entropy value.

5. The method according to claim 1, characterized in that, The expression for the tunable reward function is: ， in, This represents a tunable reward function. , These represent the weights for search and tracking, respectively. Indicates search uncertainty. This indicates tracking uncertainty.

6. A decision-making device for a robot, characterized in that, include: The generation module is used to generate graph structure information based on the robot's operating environment, so as to generate the search uncertainty and tracking uncertainty of the operating environment based on the graph structure information; A construction module is used to simulate the movement process of at least one search target and at least one tracking target using a pre-built unidirectional entropy diffusion model to obtain simulation data, and to introduce global abstract nodes to construct a two-level graph structure of the at least one search target and the at least one tracking target. The decision module is used to generate a tunable reward function based on the search uncertainty, the tracking uncertainty, the simulation data, and the two-level graph structure to construct the robot's decision model, and to output collaborative search and tracking decision actions using the decision model.

7. The apparatus according to claim 6, characterized in that, Also includes: The aggregation module is used to aggregate the first edge between the spatial nodes of the two-level graph structure using a preset graph convolutional network operator to obtain the local topology information of the first edge. The extraction module is used to extract the second edge between the spatial node and the global abstract node using a preset graph attention network operator, so as to obtain global information related to the current decision.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor executing the program to implement the decision-making method of the robot as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the decision-making method of the robot as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the decision-making method of the robot as described in any one of claims 1-5.

Images