Intelligent agent behavior planning method, electronic device, medium and program product

Abstract

The present invention provides an intelligent agent behavior planning method, electronic device, medium and program product. The intelligent agent behavior planning method includes: constructing an environment map based on acquired environment data; using information entropy as a measurement method for uncertainty, and quantifying uncertainty by analyzing the information density and known information of each area in the environment map; and the intelligent agent plans behavior based on information entropy. By dynamically constructing an environment map and selecting behavior driven by information entropy, real-time perception of a complex environment and behavior optimization are achieved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to an intelligent body behavior planning method, electronic equipment, medium and program product. Background Art

[0002] Agent behavior planning is a complex technology involving artificial intelligence, machine learning, decision theory and control theory, which aims to enable agents to make reasonable decisions and behaviors in dynamic and uncertain environments. In dynamic environments, existing agent behavior planning methods are usually unable to effectively handle the complexity and uncertainty in the environment, making it difficult for agents to achieve optimal behavior during exploration and decision-making. Summary of the invention

[0003] In response to the problems existing in the prior art, the present invention provides an intelligent agent behavior planning method, electronic device, medium and program product, which at least partially solve the problem that the prior art cannot effectively handle the complexity and uncertainty in the environment.

[0004] In a first aspect, an embodiment of the present disclosure provides an agent behavior planning method, comprising:

[0005] Building an environmental map based on the acquired environmental data;

[0006] Using information entropy as a measure of uncertainty, the uncertainty is quantified by analyzing the information density and known information of each area in the environmental map;

[0007] The agent plans its behavior based on information entropy.

[0008] Optionally, the acquired environmental data is represented in a PG format, where the PG format includes geometric information and multivariate attributes of the object, where the multivariate attributes are attributes of the object generated by reasoning and parsing a computer vision model on an RGB image;

[0009] The multivariate attributes include:

[0010] common_base:rdf_type: used to represent the category information of an object;

[0011] common_base:CVReliability: used to indicate the confidence of object attribute recognition;

[0012] visible: used to indicate the visibility state of an object.

[0013] Optionally, constructing an environmental map based on the acquired environmental data includes:

[0014] The acquired environmental data is represented according to a set format to obtain an environmental map;

[0015] Boundary detection is performed based on the acquired environment data, thereby detecting boundaries of the environment map and boundaries of objects in the environment map.

[0016] Optionally, performing boundary detection based on the acquired environmental data, thereby detecting the boundary of the environmental map and the boundary of the object in the environmental map, includes:

[0017] Generate point cloud data from RGB-D images in environmental data;

[0018] The point cloud data is parsed to obtain the three-dimensional position information of each point;

[0019] Boundary detection is performed based on the 3D position information of each point.

[0020] Optionally, the step of constructing an environmental map based on the acquired environmental data further includes:

[0021] The environmental map is constructed in the form of a two-dimensional table, where each cell contains multiple attributes. The cells are dynamically updated according to the new environmental data, so that the environmental map can be dynamically updated in real time.

[0022] Optionally, the real-time dynamic updating of the environment map includes:

[0023] Determine whether the map boundaries need to be extended based on the coordinates in the parsed setting format;

[0024] If expansion is needed, continue to determine the new boundary range with coordinates and expand the environmental map to the new boundary;

[0025] When expanding the map, check the objects in the current environment map. If the object exists in the old map, clear the corresponding old object information, and then put all the updated object information back to the expanded environment map.

[0026] Optionally, when expanding the map, split the environment map into multiple small blocks and use multithreading to process each sub-block in parallel.

[0027] Optionally, the method of using information entropy as a measure of uncertainty includes:

[0028] When constructing an environmental map in the form of a two-dimensional table, the information entropy is set to a set entropy value when the cell is initially constructed, and the set entropy value is used to represent the undetermined state of the cell;

[0029] Calculate the information entropy of each object in the environment map based on the obtained environment data;

[0030] The calculated information entropy of each object is updated to the corresponding cell, allowing each cell to contain information of multiple objects at the same time. The state of the cell is reflected by the sum of the information entropy of all objects in the cell.

[0031] Optionally, the method of using information entropy as a measure of uncertainty includes:

[0032] When the high entropy area is explored, the cell data within the exploration perspective is updated;

[0033] And the information entropy in the corresponding cell data of the explored area is entropy reduced.

[0034] Optionally, the agent plans behavior based on information entropy, including:

[0035] Divide the current environment map into blocks by region, and calculate the total information entropy of each region one by one;

[0036] Sort the calculated regional entropy values ​​from large to small to generate an entropy value sequence;

[0037] Return the coordinates of the area with the highest entropy value as the target of the agent's action.

[0038] Optionally, the step of returning the coordinates of the region with the highest entropy value as the target of the agent's action also includes:

[0039] Based on the regional coordinates, the agent adjusts its perspective to focus on the marked high entropy area for exploration;

[0040] Within the agent's field of view, all objects in the observed cells will be marked as explored, and the information entropy in the explored cells will be reduced.

[0041] In a second aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0042] at least one processor; and,

[0043] a memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the intelligent agent behavior planning method described in any one of the first aspects.

[0045] In a third aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the intelligent agent behavior planning methods described in the first aspect.

[0046] In a fourth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements any of the intelligent agent behavior planning methods described in the first aspect.

[0047] The present invention provides an intelligent agent behavior planning method, electronic device, medium and program product, wherein the intelligent agent behavior planning method realizes real-time perception and behavior optimization of complex environments by dynamically constructing environmental maps and information entropy-driven behavior selection, thereby achieving the purpose of improving the decision-making efficiency of the intelligent agent and enhancing the intelligent agent's ability to cope with complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0049] Figure 1 A flowchart of an intelligent agent behavior planning method provided in an embodiment of the present disclosure;

[0050] Figure 2 A principle block diagram of an intelligent agent behavior planning device provided in an embodiment of the present disclosure;

[0051] Figure 3 A functional block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0052] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0053] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0054] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0055] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0056] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0057] The application of information entropy in agent behavior planning is mainly reflected in the following aspects:

[0058] Quantification of uncertainty in decision-making: Information entropy is used to quantify the uncertainty faced by agents in the decision-making process. In multi-agent situational reasoning, information entropy is used to measure the uncertainty in the state transition process of each agent, and thus serves as a parameter of the reward and punishment function. In this way, agents can adjust their behavior according to the value of information entropy to reduce uncertainty and optimize decision-making.

[0059] Balance between exploration and exploitation: In reinforcement learning, information entropy is used as a regular term added to the reward function to encourage the agent to explore states or behaviors with high uncertainty. By increasing the entropy of the strategy, the agent is encouraged to try different behaviors, thereby increasing its understanding of the environment, which helps the agent find better strategies in complex environments.

[0060] Situation assessment and reasoning: In multi-agent systems, information entropy is used for situation assessment, which is the core content of high-level information fusion. Situation reasoning models and algorithms based on information entropy can help agents understand and predict the behavior of other agents, thereby making better collaborative or confrontational decisions.

[0061] Metrics of group intelligence: Information entropy is also used to measure the degree of group intelligence, especially in multi-agent reinforcement learning. By calculating group entropy, we can analyze the information interaction between agents and the degree of convergence of group intelligence. When group entropy decreases, it indicates that the uncertainty of group strategy decreases, the effectiveness of interaction between agents decreases, and the group as a whole evolves towards an ordered structure.

[0062] Optimizing model performance: Information entropy can be used to evaluate and optimize the performance of AI models. During model training, information entropy can help identify where the model has uncertainty, thereby guiding the optimization of model parameters and improving the accuracy and stability of the model.

[0063] Feature selection and data classification: In machine learning, information entropy is used to measure the purity of a data set, that is, the degree of uncertainty of each category of data in the data set. By calculating information entropy, the optimal partitioning attribute can be selected to maximize the purity of the segmented data set, thereby improving the accuracy of the classification algorithm.

[0064] Image processing and computer vision: Information entropy is used to evaluate the complexity or texture features of an image. In image processing, the texture complexity of an image can be evaluated by calculating the information entropy of the image, thereby designing better image processing algorithms, such as image compression, image segmentation, etc.

[0065] For ease of understanding, Figure 1 As shown, this embodiment discloses an agent behavior planning method, including:

[0066] Step S101: constructing an environment map based on the acquired environment data;

[0067] Build an environmental map based on the acquired environmental data, including:

[0068] The acquired environmental data is represented according to a set format to obtain an environmental map;

[0069] Represent various types of information in the environment on the map so that the intelligent agent can perceive the location, characteristics, status, etc. in the environment.

[0070] This representation method improves the agent's ability to understand the environment and helps it make reasonable decisions in complex situations.

[0071] Boundary detection is performed based on the acquired environment data, thereby detecting boundaries of the environment map and boundaries of objects in the environment map.

[0072] Optionally, performing boundary detection based on the acquired environmental data, thereby detecting the boundary of the environmental map and the boundary of the object in the environmental map, includes:

[0073] Generate point cloud data from RGB-D images in environmental data;

[0074] The point cloud data is parsed to obtain the three-dimensional position information of each point;

[0075] Boundary detection is performed based on the 3D position information of each point.

[0076] As the agent acts, it collects environmental data in real time through sensors or other input devices.

[0077] Use this data to build a map of the environment, representing various features and obstacles in the environment.

[0078] Maps dynamically update to reflect real-time environmental changes, ensuring they are always up to date.

[0079] Optionally, the step of constructing an environmental map based on the acquired environmental data further includes:

[0080] The environmental map is constructed in the form of a two-dimensional table, where each cell contains multiple attributes. The cells are dynamically updated according to the new environmental data, so that the environmental map can be dynamically updated in real time.

[0081] Optionally, the real-time dynamic updating of the environment map includes:

[0082] Determine whether the map boundaries need to be extended based on the coordinates in the parsed setting format;

[0083] If expansion is needed, continue to determine the new boundary range with coordinates and expand the environmental map to the new boundary;

[0084] When expanding the map, check the objects in the current environment map. If the object exists in the old map, clear the corresponding old object information, and then put all the updated object information back to the expanded environment map.

[0085] Optionally, when expanding the map, split the environment map into multiple small blocks and use multithreading to process each sub-block in parallel.

[0086] Step S102: using information entropy as a measure of uncertainty, quantifying uncertainty by analyzing the information density and known information of each area in the environment map;

[0087] Using information entropy as a measure of uncertainty, uncertainty is quantified by analyzing the information density and known information of each area in the map.

[0088] Each time new environmental data is received, the system dynamically adjusts the distribution of information entropy based on the new information, ensuring that the calculation of information entropy always reflects the uncertainty level of the current environment.

[0089] Information entropy is calculated by region so that the agent can identify areas of high uncertainty and prioritize its exploration and behavior.

[0090] Optionally, the method of using information entropy as a measure of uncertainty includes:

[0091] When constructing an environmental map in the form of a two-dimensional table, the information entropy is set to a set entropy value when the cell is initially constructed, and the set entropy value is used to represent the undetermined state of the cell;

[0092] Calculate the information entropy of each object in the environment map based on the obtained environment data;

[0093] The calculated information entropy of each object is updated to the corresponding cell, allowing each cell to contain information of multiple objects at the same time. The state of the cell is reflected by the sum of the information entropy of all objects in the cell.

[0094] Optionally, the method of using information entropy as a measure of uncertainty includes:

[0095] When the high entropy area is explored, the cell data within the exploration perspective is updated;

[0096] And the information entropy in the corresponding cell data of the explored area is entropy reduced.

[0097] Step S103: The agent plans behavior based on information entropy.

[0098] By comparing the information entropy of different areas, the next action strategy of the intelligent agent is determined.

[0099] The agent chooses high information entropy regions to explore in order to reduce uncertainty in the environment and thus optimize its behavior.

[0100] This strategy enables the intelligent agent to make efficient decisions and explore in a constantly changing environment, gradually reduce the information entropy in the environment, and achieve comprehensive control over the environment.

[0101] Optionally, the agent plans behavior based on information entropy, including:

[0102] Divide the current environment map into blocks by region, and calculate the total information entropy of each region one by one;

[0103] Sort the calculated regional entropy values ​​from large to small to generate an entropy value sequence;

[0104] Return the coordinates of the area with the highest entropy value as the target of the agent's action.

[0105] Optionally, the step of returning the coordinates of the region with the highest entropy value as the target of the agent's action also includes:

[0106] Based on the regional coordinates, the agent adjusts its perspective to focus on the marked high entropy area for exploration;

[0107] Within the agent's field of view, all objects in the observed cells will be marked as explored, and the information entropy in the explored cells will be reduced.

[0108] Agent behavior planning is a complex process involving multiple links such as environmental perception, decision making, and behavior execution. The following is a detailed description of the agent behavior planning method, including building an environment map based on the acquired environmental data, using information entropy as a measure of uncertainty, and the process of the agent planning behavior based on information entropy.

[0109] Build an environmental map based on the acquired environmental data: The first step in the behavior planning of the intelligent agent is to build an environmental map. This step involves SLAM (Simultaneous Localization and Mapping) technology, which allows the intelligent agent to navigate in an unknown environment while building a map of the environment. SLAM technology includes two key parts: localization and mapping. Localization refers to determining the position and posture of the intelligent agent in the environment, while mapping uses perception information to build a map of the environment. By perceiving the surrounding environment through the onboard sensors (such as radars and cameras) and extracting information such as feature points and edges, the intelligent agent can build a feature map or grid map of the environment.

[0110] Using information entropy as a measure of uncertainty: Information entropy is a concept in information theory used to measure the amount of information, which can be used to quantify uncertainty. In agent behavior planning, information entropy is used to measure the information density and known information of each area in the environment map, thereby quantifying uncertainty.

[0111] Agents plan behaviors based on information entropy: When planning behaviors, agents consider the information entropy in the environment map to determine their action strategies. Agents can use information entropy to identify areas of the environment with the most uncertain information and prioritize exploring these areas to reduce uncertainty. For example, in multi-agent situational reasoning, algorithms based on information entropy are used to evaluate the state transitions of agents and serve as a parameter in the reward and punishment function. In this way, agents can explore the environment more effectively and optimize their behavior planning.

[0112] like Figure 2 As shown, this embodiment also discloses an intelligent agent behavior planning system, including: an environment information representation module, a dynamic environment map construction module, an uncertainty quantification module using information entropy, and an information entropy driven behavior selection module;

[0113] Environmental information representation module:

[0114] Environmental data collection and representation in Parsing Graph format. The intelligent agent collects environmental data in real time through sensors (such as RGB-D cameras). After being received, these data are preprocessed and represented in PG (Parsing Graph) format. PG is an abstract representation of objects, which contains rich attribute information and is used to describe the detailed characteristics of objects in the scene. The PG format includes not only the geometric boundaries of objects (such as vertex coordinates), but also attributes such as common_base:rdf_type for object category, common_base:CVReliability for confidence, and visible for visibility. Through PG, geometric boundaries and multiple types of semantic attributes can be integrated in a structured form to obtain a more comprehensive and intuitive description of objects in the scene.

[0115] This representation method provides detailed data support for the construction of environmental maps. For example, the following is an example of PG format:

[0116] {

[0117] "desk": {

[0118] "common_base:maxVertexX": [-15.616144554505766],

[0119] "common_base:maxVertexY": [-271.0740216920877],

[0120] "common_base:minVertexX": [-116.16347275666874],

[0121] "common_base:minVertexY": [-289.5429113616396],

[0122] "rdf:type": ["common_base:desk"],

[0123] "visible": [true],

[0124] "common_base:isVirtual": [false],

[0125] "common_base:CVReliability": [1]

[0126] },

[0127] "table": {

[0128] "common_base:maxVertexX": [-15.616144554505766],

[0129] "common_base:maxVertexY": [-271.0740216920877],

[0130] "common_base:minVertexX": [-116.16347275666874],

[0131] "common_base:minVertexY": [-289.5429113616396],

[0132] "rdf:type": ["table"],

[0133] "visible": [true],

[0134] "common_base:isVirtual": [false],

[0135] "common_base:CVReliability": [1]

[0136] },

[0137] "Robotbaby": {

[0138] "common_base:maxVertexX": [-15.616144554505766],

[0139] "common_base:maxVertexY": [-271.0740216920877],

[0140] "common_base:minVertexX": [-116.16347275666874],

[0141] "common_base:minVertexY": [-289.5429113616396],

[0142] "rdf:type": ["reptilian_acoustic"],

[0143] "visible": [true],

[0144] "common_base:isVirtual": [false],

[0145] "common_base:CVReliability": [1],

[0146] "humanPose": {

[0147] "eye_jiont_l": [5.3426432609558105, -3.945061683654785,93.61125183105469],

[0148] "eye_jiont_r": [5.342660427093506, 3.939558506011963,93.6116943359375]

[0149] }

[0150] },

[0151] "NPC": {

[0152] "common_base:maxVertexX": [-15.616144554505766],

[0153] "common_base:maxVertexY": [-271.0740216920877],

[0154] "common_base:minVertexX": [-116.16347275666874],

[0155] "common_base:minVertexY": [-289.5429113616396],

[0156] "rdf:type": ["NPC"],

[0157] "visible": [true],

[0158] "common_base:isVirtual": [false],

[0159] "common_base:CVReliability": [1]

[0160] }

[0161] }

[0162] In this PG format, the bounding box of each object is represented by common_base:maxVertexX, common_base:minVertexX, common_base:maxVertexY and common_base:

[0163] minVertexY and other fields are defined to ensure that the position and size of objects in the map are accurately represented. This structure provides basic data for the system to build an environmental map, so that the map can reflect the layout and characteristics of objects in the scene and support the intelligent agent to perceive the environment and make behavioral decisions.

[0164] Boundary detection based on point cloud data: In the process of processing environmental data, the RGB-D images collected by the intelligent agent through sensors are used to generate point cloud data. These point cloud data are parsed in the system to provide the three-dimensional position information of each point in the scene. The system uses these point cloud data and applies the Hough Transform (HoughLinesP) algorithm for boundary detection. The Hough Transform can efficiently identify the straight lines in the point cloud data, which are marked as boundary information on the environmental map. The marked boundaries help the intelligent agent identify and avoid obstacles, ensuring the safety and effectiveness of path planning and navigation.

[0165] Point cloud data acquisition: The agent uses RGB-D sensors (such as Kinect or LIDAR) to collect 3D data of the environment and generate point clouds. These point cloud data contain the X, Y, and Z coordinates of each point in the scene, providing basic data for subsequent boundary detection.

[0166] Point cloud data preprocessing: Before boundary detection, point cloud data usually needs to be preprocessed, including filtering and denoising, downsampling to reduce the amount of calculation, and possible registration operations to ensure the accuracy and usability of the point cloud data.

[0167] Hough Transform (HoughLinesP) algorithm application: Hough Transform is a feature extraction technique that was originally used to detect straight lines in images. In point cloud data processing, Hough Transform is used to identify straight lines in point clouds, which usually represent boundaries or structural lines in the environment. Hough Transform identifies straight lines by mapping point cloud data into parameter space and finding local maxima in this space.

[0168] Boundary detection: Using the Hough transform algorithm, the agent can efficiently identify straight lines from point cloud data. These straight lines are marked as boundary information on the environment map. The results of boundary detection help the agent understand the structure of the environment and identify the location of obstacles.

[0169] Path planning and navigation The boundary information of the markers is crucial for the path planning of the agent. The agent can plan a safe path based on this boundary information, avoid obstacles, and ensure the safety and effectiveness of navigation. In path planning, boundary information is used to define free space and obstacle space, and the agent searches for the optimal path from the starting point to the end point in the free space.

[0170] Environment Map Marking: The detected boundary information is added to the environment map to form a detailed description of the environment. This map not only contains the location of obstacles, but may also contain other important environmental features such as stairs, doors, etc. This information is very useful for the agent's high-level navigation and task planning.

[0171] Real-time update and dynamic planning: In a dynamic environment, the agent needs to update its environment map in real time and dynamically adjust the path planning. The continuous collection of point cloud data and real-time processing of boundary detection enable the agent to adapt to environmental changes, such as moving obstacles or new paths.

[0172] Hough Transform (HoughLinesP) is an algorithm for line detection and an implementation of Hough Transform. The basic idea of ​​Hough Transform is to map points in image space to curves in parameter space and identify lines by detecting the intersection of these curves.

[0173] The basic idea of ​​the Hough transform is to map points in the image space to curves in the parameter space. For line detection, the line can be represented in the image space in the form of y=mx+b, but when dealing with vertical lines, the slope m tends to infinity. Therefore, the Hough transform uses r=xcosθ+ysinθ in the polar coordinate system to represent the line, where r is the distance from the line to the origin and θ is the direction angle of the line. Each point in the image is represented as a sine curve in the parameter space, and the intersection of all these curves represents the straight line passing through these points.

[0174] HoughLinesP is an implementation of the cumulative probabilistic Hough transform (PPHT), which returns the endpoints of the detected lines, that is, each line is represented by its two endpoints (x1, y1), (x2, y2). Compared to HoughLines, HoughLinesP does not detect complete straight lines in the entire image, but finds straight line segments. This method is more effective for detecting short line segments because it reduces the amount of calculation and improves efficiency. HoughLinesP allows additional parameters to be specified to control the minimum length and maximum break of the detected line segments, which makes it more flexible and efficient when processing actual images.

[0175] In addition, boundary detection is not only used to mark obstacles, but also helps identify the layout of rooms and areas. By performing topological analysis on the detected lines, the system can infer environmental structures such as walls and door frames. This boundary information is stored in the MapCell data structure of the map, where each MapCell may store object information including boundary attributes to ensure that the map has sufficient details in both objects and structures. This detailed boundary detection process enhances the agent's environmental perception ability in complex environments.

[0176] MapCell is an object used to represent a map cell, commonly used in geographic information systems (GIS) and game development. The definition and purpose of MapCell vary in different contexts.

[0177] In Geographic Information Systems (GIS), MapCell is used to visualize geographic data. Users can add a Map cell to a project, select a data frame containing geographic data, and select a layer type (such as points, lines, polygons, etc.). Map cells can visualize up to 100,000 rows of data, exceeding this limit will cause performance issues.

[0178] In 2D game development, MapCell is usually used to represent a single cell on the map. Each MapCell contains information such as floor type, obstacles, items, etc. For example, in a tile-based map implementation, each MapCell can contain information such as floor type, obstacles, and whether it is possible to move to the cell.

[0179] In the robot navigation system, the MapCell class is used to determine the distance between the map cell and the target point and the global path, helping the local planner to generate the path. The MapCell class contains the coordinate information of the cell, which is used to calculate the distance and score.

[0180] Dynamically build environment map module:

[0181] The map is in a two-dimensional table (std::vector <std::vector <mapcell>>), each cell (MapCell) represents a small grid in the map, and its definition is as follows:

[0182] struct MapCell {

[0183] std::unordered_map <std::string, std::unordered_map<std::string,std::string> > items;

[0184] double entropy;

[0185] bool is_revealed;

[0186] };

[0187] MapCell contains multiple attributes, such as object information, information entropy value, whether it has been explored, etc. The agent dynamically updates these cells according to new sensor data during operation, thereby achieving real-time response to environmental changes.

[0188] Receiving data and adjusting boundaries

[0189] The system first receives environmental data from sensors or other input devices and determines whether the map boundary needs to be expanded by parsing the coordinates in the PG format (such as maxVertexX, minVertexX, maxVertexY, minVertexY). This step ensures that the map can cover the new environmental area and prepare for subsequent map updates.

[0190] Boundary expansion and initialization: Based on the newly received data, the system calculates the new boundary range of the map and expands the map to accommodate these new boundaries. At this time, the expansion of the boundary needs to be accurately managed to ensure that the new area is seamlessly connected to the existing map. The newly added rows and columns need to be initialized correctly to maintain consistency in subsequent processing.

[0191] Replication and maintenance of old data: During the expansion process, existing map data must be copied to the expanded map structure to maintain historical information. This step requires efficient data migration and replication, which is especially critical when dealing with large-scale maps. Ensuring that data is not lost or erroneous during replication is an important technical difficulty in this stage. In order to improve the efficiency of map expansion, the entire map is split into multiple small blocks, and each sub-block is processed in parallel using multiple threads. This method significantly improves the data processing speed, but it is necessary to solve the thread synchronization problem in the design to ensure data consistency and integrity during concurrent access.

[0192] Identification and update of object information: When expanding the map, the system will check the objects in the current composition. If the object already exists in the old map, the corresponding old object information must be cleared first to prevent redundancy or conflict. Then, all the object information received this time is re-placed on the expanded map to ensure that the current map accurately reflects the state of the environment.

[0193] Multithreaded data synchronization and security: Since multiple threads are executed in parallel, access to shared data structures must be safe. To this end, appropriate synchronization mechanisms, such as mutexes or read-write locks, are needed to prevent data competition and inconsistency issues, thereby ensuring the stability of map expansion.

[0194] Real-time guarantee: Dynamic environments require that map expansion and updates can be completed within a limited time to maintain the system's ability to respond quickly to environmental changes. This places high demands on algorithm optimization and thread scheduling, and requires reasonable allocation of computing resources to ensure real-time response of the system.

[0195] Through the above steps and technical implementations, not only the dynamic expansion of the map is achieved, but also the data integrity and real-time updating of environmental information are ensured, so that the intelligent agent can quickly perceive and optimize its behavior in a complex dynamic environment.

[0196] Module to quantify uncertainty using information entropy:

[0197] The system uses information entropy to quantify and represent the uncertainty in the environment. The calculation of information entropy is closely related to the confidence of each object. When receiving environmental data (PG format), the system calculates the initial information entropy of each object based on the object's confidence field common_base:CVReliability and saves it to the map structure. The calculation formula is as follows:

[0198]

[0199] Among them, p represents the confidence of the object (the value range is between 0 and 1), and the system calculates the corresponding entropy value to represent the degree of uncertainty of the object on the map.

[0200] Initial information entropy when building a map: During the initial construction of the map, in order to effectively quantify the uncertainty in the environment, each MapCell needs to set an initial information entropy value. This initial information entropy setting can help the agent identify unexplored areas in the initial stage and promote exploration and perception.

[0201] Specifically, when MapCell is initially constructed, the information entropy is set to 10, indicating the undetermined state of the current area. This value represents the maximum uncertainty assumption for an unknown area without further data support. As sensor data is input and the map is dynamically updated, the information entropy is adjusted based on the detection and confidence of objects in the environment. When new data is received and parsed, the information entropy is recalculated and updated based on the common_base:CVReliability of the object to reflect the actual uncertainty of the current area.

[0202] This initial information entropy setting provides a benchmark for the agent's exploration in the environment, allowing it to prioritize high information entropy areas and achieve effective exploration and behavior optimization.

[0203] Save information entropy to MapCell: Each MapCell represents a basic unit of the map, and its items attribute is used to store all object information belonging to the unit. When an object's bounding box spans multiple MapCells, the object's information entropy will be added to these MapCells. Specifically:

[0204] The items in each MapCell will contain the identifier of the object and its properties (including information entropy).

[0205] MapCell.entropy represents the sum of the information entropy of all objects in the cell, reflecting the overall uncertainty of the area.

[0206] This design allows a MapCell to contain information about multiple objects at the same time, ensuring that the superposition effect of information entropy is correctly represented in complex environments.

[0207] Entropy increase and entropy reduction process: When adding objects to the map, the calculation and accumulation of information entropy is an entropy increase process. Whenever a new object is introduced into the map, its corresponding entropy will be added to the MapCell within the coverage area, increasing the overall information entropy of these cells.

[0208] On the other hand, when the agent performs tasks and explores high entropy areas, the MapCell within the field of view will be updated. These areas are considered to have been explored, and their information entropy will be reduced, indicating that the uncertainty has been partially eliminated. The specific operations include:

[0209] The agent marks the explored MapCells within the viewing range.

[0210] Reduce the entropy of the corresponding objects in these MapCells and update MapCell.entropy to reflect the reduced entropy state after exploration.

[0211] This mechanism ensures that the intelligent agent can gradually reduce the information entropy on the map and, through information-driven exploration, gradually control the uncertainty of the environment and optimize behavioral choices.

[0212] Information entropy driven behavior selection module:

[0213] Based on the level of information entropy, the agent is guided to explore and make decisions. In this scheme, information entropy is not only used to quantify the uncertainty in the environment, but also to drive the agent's behavior selection. The system calculates the information entropy on the map in real time to determine the priority areas for the agent to explore, so that the agent can actively reduce the uncertainty in the environment and optimize its behavior decision.

[0214] Real-time calculation of information entropy: The system has a dedicated thread responsible for dividing the map into blocks and calculating the information entropy of each area in real time. This process is implemented by the MapQuery::GetHighEntropyRegions function. The function is performed through the following steps:

[0215] Regional entropy calculation: Divide the current map into blocks by region, and calculate the total information entropy of each region one by one. The calculation of information entropy is based on the accumulation of MapCell.entropy, which represents the sum of the information entropy of objects in all MapCells in the region.

[0216] Sorting and selection: Sort the calculated regional entropy values ​​from large to small to generate an entropy value sequence.

[0217] Locate high entropy areas: Return the coordinates of the area with the highest entropy value as the target for the next action of the agent.

[0218] In this way, the system can continuously monitor and update the information entropy distribution on the map and always keep the identification of high entropy areas to ensure that the agent focuses on the parts of the environment with the highest information density and the greatest uncertainty.

[0219] Agent behavior triggering and exploration: When the system determines the coordinate with the largest entropy in the current area, the agent will actively trigger the behavior task, such as issuing a look-at task, and perform the following steps:

[0220] The agent observes the high entropy area: The agent adjusts its perspective and focuses on the marked high entropy area for exploration.

[0221] Reduction of information entropy: Within the scope of the agent's view, all objects in the observed MapCell will be marked as explored. At this time, the system reduces the entropy value in these MapCells and updates the overall information entropy to reflect the state that the uncertainty in the environment has been partially eliminated.

[0222] Exploration feedback: The agent’s exploration behavior makes the map information more accurate and richer, gradually reducing the uncertainty of the overall environment.

[0223] This information entropy-driven behavior selection enables the agent to efficiently select exploration paths and prioritize areas with the greatest uncertainty, thereby achieving continuous optimization of the dynamic environment. This mechanism not only improves the agent's decision-making efficiency, but also enhances its ability to cope with complex environments.

[0224] The information entropy of all areas within the agent's field of view has become 0. This means that the agent has completed exploration within the field of view, which is a process of entropy elimination, meaning that these areas have been explored and marked as known. Objects such as doors or plants can be detected through boundary detection. The area with the highest information entropy in the current scene attracts the agent's attention, which means that the area has not been fully explored. The agent may choose to move or adjust the view in the future to explore this area and further eliminate information entropy.

[0225] When the agent detects a high entropy area, it triggers the agent to turn, thereby exploring the unexplored high information entropy area. The direction of the agent's view changes with the turning action. Within the agent's new view range, the information entropy has been eliminated, and the information entropy of all cells within the view becomes 0, which means that the agent has explored these areas and obtained relevant environmental information. When the agent explores a room with a closed door, because the house is closed, the agent cannot see the inside through the boundary of the house. Therefore, the information entropy of the area inside the house is not cleared and remains at a high value. Although the agent has adjusted its perspective and explored some areas within the field of view, the information entropy inside the house has not changed due to the obstruction of the physical boundary. This means that the agent needs other actions or strategies to further explore these blocked areas, such as trying to open the door or observing from other perspectives.

[0226] The electronic device disclosed in this embodiment includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0227] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the electronic device performs all or part of the steps of the agent behavior planning method of each embodiment of the present disclosure.

[0228] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0229] like Figure 3 The present invention provides a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, which is suitable for implementing the electronic device in the embodiment of the present invention. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0230] like Figure 3 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0231] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device allows the electronic device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 3 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0232] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, all or part of the steps of the agent behavior planning method of the embodiment of the present disclosure are executed.

[0233] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0234] The computer-readable storage medium disclosed in this embodiment stores non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the agent behavior planning method of each embodiment of the present disclosure are executed.

[0235] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0236] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0237] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0238] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be recognized by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, meaning "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0239] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a disjunctive list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0240] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0241] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0242] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0243] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.< / mapcell>

Claims

1. A method for intelligent agent behavior planning, characterized in that: include: Building an environmental map based on the acquired environmental data; Using information entropy as a measure of uncertainty, the uncertainty is quantified by analyzing the information density and known information of each area in the environmental map; The agent plans its behavior based on information entropy; The acquired environmental data is represented in PG format, which includes geometric information and multivariate attributes of objects; The multivariate attributes include: common_base:rdf_type: used to represent the category information of the object; common_base:CVReliability: used to represent the confidence of the object attribute recognition; the environment map is constructed in the form of a two-dimensional table, each cell represents a small grid in the map, the cell contains multiple attributes, and each cell contains multiple object information at the same time. The state of the cell is reflected by the sum of the information entropy of all objects in the cell; When the cell is initially constructed, the information entropy is set to the set entropy value. The set entropy value is used to represent the uncertain state of the cell. When new data is received and parsed, the information entropy will be recalculated and updated according to the common_base:CVReliability of the object to reflect the actual uncertainty of the current area. The agent plans behavior based on information entropy, including: The current environment map is divided into blocks by region, and the total information entropy of each region is calculated one by one. The total information entropy of each region represents the sum of the information entropy of objects in the cells in the region; the calculated regional entropy values ​​are sorted from large to small to generate an entropy value sequence; the coordinates of the region with the highest entropy value are returned as the target of the next action of the intelligent agent; when the high entropy area is explored, the cell data within the exploration perspective is updated, and the information entropy in the corresponding cell data of the explored area is entropy reduced.

2. The agent behavior planning method according to claim 1, characterized in that: The multivariate attributes are attributes of the object generated by reasoning and parsing the RGB image using a computer vision model; The multivariate attributes also include: visible: used to indicate the visibility state of an object.

3. The agent behavior planning method according to claim 1, characterized in that: The step of constructing an environmental map based on the acquired environmental data includes: The acquired environmental data is represented according to a set format to obtain an environmental map; Boundary detection is performed based on the acquired environment data, thereby detecting boundaries of the environment map and boundaries of objects in the environment map.

4. The agent behavior planning method according to claim 3, characterized in that: The performing boundary detection based on the acquired environment data, thereby detecting the boundary of the environment map and the boundary of the object in the environment map, includes: Generate point cloud data from RGB-D images in environmental data; The point cloud data is parsed to obtain the three-dimensional position information of each point; Boundary detection is performed based on the 3D position information of each point.

5. The agent behavior planning method according to claim 3, characterized in that: The step of constructing an environmental map based on the acquired environmental data further includes: Cells are dynamically updated based on new environmental data, allowing the environmental map to be dynamically updated in real time.

6. The agent behavior planning method according to claim 5, characterized in that: The real-time dynamic updating of the environment map includes: Determine whether the map boundaries need to be extended based on the coordinates in the parsed setting format; If expansion is needed, continue to determine the new boundary range with coordinates and expand the environmental map to the new boundary; When expanding the map, check the objects in the current environment map. If the object exists in the old map, clear the corresponding old object information, and then put all the updated object information back to the expanded environment map.

7. The agent behavior planning method according to claim 6, characterized in that: When expanding the map, the environment map is split into multiple small blocks, and each sub-block is processed in parallel using multi-threading.

8. The agent behavior planning method according to claim 1, characterized in that: The method of using information entropy as a measure of uncertainty includes: When constructing an environmental map in the form of a two-dimensional table, the information entropy is set to a set entropy value when the cell is initially constructed, and the set entropy value is used to represent the undetermined state of the cell; Calculate the information entropy of each object in the environment map based on the obtained environment data; The calculated information entropy of each object is updated to the corresponding cell, allowing each cell to contain information of multiple objects at the same time. The state of the cell is reflected by the sum of the information entropy of all objects in the cell.

9. The agent behavior planning method according to claim 1, characterized in that: The step of returning the coordinates of the area with the highest entropy value as the target of the next action of the agent also includes: Based on the regional coordinates, the agent adjusts its perspective to focus on the marked high entropy area for exploration; Within the agent's field of view, all objects in the observed cells will be marked as explored, and the information entropy in the explored cells will be reduced.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the intelligent agent behavior planning method described in any one of claims 1-9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the intelligent agent behavior planning method described in any one of claims 1-9.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the intelligent agent behavior planning method described in any one of claims 1-9 is implemented.

Images