Intelligent agent behavior modeling method, system, medium, and computer device

Abstract

The application provides an agent behavior modeling method, system, medium and computer device, and the method comprises the following steps: acquiring scene image data; generating a scene knowledge base according to the scene image data; generating a behavior knowledge base according to the scene knowledge base; and determining the maneuvering behavior of the agent according to the scene knowledge base, the behavior knowledge base and a historical behavior base. The method has the self-evolution ability of the agent behavior modeling method of the embodiment of the application to adapt to unknown scenes by acquiring the scene image data, generating the scene knowledge base and the behavior knowledge base. Based on the reinforcement learning strategy, the behavior generation model based on the normalized radial basis function is used to realize the data fusion of the behavior knowledge base and the historical behavior base as the model training sample. The reward function is optimized by combining the deep learning with the reverse reinforcement learning and utilizing the complex correlation of the environment and the state characteristics.

Description

Technical Field

[0001] This invention relates to the field of simulation modeling, and more specifically to methods, systems, media, and computer devices for modeling the behavior of intelligent agents. Background Technology

[0002] Traditional simulation modeling methods approach behavior decision-making from an event-driven perspective, constructing models for intelligent agents. This includes building state machines and writing script rules. These methods require specialized expertise and are very labor-intensive, while still struggling to guarantee the robustness and accuracy of the final model. Current simulation modeling techniques, whether for single-agent or multi-agent behavior decision-making, fall short of reality and fail to meet the needs of practical development. In particular, no effective modeling framework or theory has yet been proposed for the behavior decision-making problem in single-agent simulation modeling. Existing single-agent modeling methods mainly fall into three categories: rule-based modeling emphasizing the simulation of human actions, cognitive modeling emphasizing the simulation of human thinking, and approximate modeling based on artificial intelligence. The first approach, in particular, is a major research focus. This method emphasizes building the modeling framework from a behavioral dimension, often requiring significant human involvement related to the task. For example, building state machines and writing script rules require substantial external knowledge, emphasizing the simulation of human actions rather than cognitive processes. Intelligent modeling, on the other hand, emphasizes using machines to replace humans in making behavioral decisions in a simulation environment and enables the model to self-evolve to enhance its robustness. Most existing methods for modeling agent behavior rely on rule-based state machines to control the agent. This approach is not only labor-intensive but also has limited robustness, requiring repeated modeling when the scenario changes. Therefore, an effective simulation modeling behavior generation method should not only save manpower but also possess self-evolution capabilities to adapt to unknown scenarios. This invention focuses on modeling the maneuvering behavior of agents in simulation platforms. By designing a new modeling framework and introducing intelligent algorithms, it addresses the problems of insufficient knowledge, large workload, and tedious work in traditional modeling methods for maneuvering behavior modeling. Summary of the Invention

[0003] To address the aforementioned problems, one objective of this invention is to provide a method, system, readable storage medium, and computer device for modeling intelligent agent behavior, enabling intelligent modeling of agent behavior and allowing the model to self-evolve to enhance its robustness.

[0004] To achieve the above objectives, in a first aspect, embodiments of the present invention provide a method for modeling the behavior of an intelligent agent, the method comprising:

[0005] Acquire scene image data;

[0006] Generate a scene knowledge base based on scene image data;

[0007] Generate a behavior knowledge base based on the scenario knowledge base;

[0008] The agent's maneuvering behavior is determined based on the behavioral knowledge base and the historical behavior base.

[0009] Furthermore, the scene image data includes images to be identified corresponding to different scenes; the scene knowledge base includes scene state vectors corresponding to multiple different scenes; generating the scene knowledge base based on the scene image data specifically includes:

[0010] Feature extraction is performed on the image to be recognized to obtain a feature map;

[0011] Determine the fused feature map based on the feature map;

[0012] The scene state vector is determined based on the fused feature map.

[0013] Furthermore, the feature map includes shallow feature maps and deep feature maps; the feature map includes c channels, where c is greater than or equal to 1 and is an integer; the fused feature map is determined based on the feature map, specifically including:

[0014] Determine the weight coefficients for each channel of the feature map;

[0015] The shallow fused feature map is obtained by multiplying the features of each channel of the shallow feature map with the corresponding weight coefficient and then performing an addition operation.

[0016] The deep fusion feature map is obtained by multiplying the features of each channel of the deep feature map with the corresponding weight coefficient and then performing an addition operation.

[0017] Upsample the deep fused feature map to obtain a cropped deep feature map of the same size as the shallow feature map;

[0018] The deep feature map and the shallow fused feature map are added together to obtain the fused feature map.

[0019] Further, the weight coefficients corresponding to each channel of the feature map are determined, specifically including:

[0020] The features corresponding to each channel are normalized and input into the fully connected layer to obtain the weight vector corresponding to that channel.

[0021] The weight vector is mapped to a range of 0 to 1 using the sigmoid function to obtain the weight coefficients.

[0022] Furthermore, a behavioral knowledge base is generated based on the scenario knowledge base, specifically including:

[0023] Multiple evolved individuals are randomly generated as the initial population, and each evolved individual includes an action vector describing the agent's maneuvering behavior.

[0024] The initial population is crossovered and mutated using a genetic algorithm to generate multiple mutated individuals.

[0025] The evaluation function is determined based on the scene state vector in the scene knowledge base;

[0026] The mutated individuals are evaluated based on the evaluation function, and those with evaluation values ​​higher than the preset value are selected as the optimal individuals and added to the behavioral knowledge base.

[0027] Furthermore, the historical behavior database includes historical action vectors describing the agent's historical maneuvers; the agent's maneuvers are determined based on the behavior knowledge base and the historical behavior database, specifically including:

[0028] Select the historical action vector that best matches the current scene from the historical behavior library;

[0029] The selected historical action vectors are fused with data from the behavior knowledge base and used as training samples for the behavior generation model.

[0030] The agent's maneuvering behavior is determined based on the behavioral knowledge base and the behavioral generation model.

[0031] Furthermore, the historical action vector that best matches the current scene is selected from the historical behavior database, specifically including:

[0032] Calculate the correlation between the historical action vectors in the historical behavior database and the data in the behavior knowledge base corresponding to the current scene;

[0033] Historical action vectors with a relevance higher than a preset relevance threshold are selected as the historical action vectors that best match the current scene.

[0034] Secondly, embodiments of the present invention provide an intelligent agent behavior modeling system, comprising:

[0035] The data acquisition module is used to acquire scene image data;

[0036] The scene knowledge base generation module is used to generate a scene knowledge base based on scene image data;

[0037] The behavior knowledge base generation module is used to generate a behavior knowledge base based on the scenario knowledge base;

[0038] The maneuver behavior generation module is used to determine the maneuver behavior of the intelligent agent based on the behavior knowledge base and the historical behavior base.

[0039] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described agent behavior modeling methods.

[0040] Fourthly, embodiments of the present invention provide a computer device comprising:

[0041] One or more processors;

[0042] Storage device for storing one or more programs;

[0043] When one or more programs are executed by one or more processors, the one or more processors implement any of the agent behavior modeling methods described above.

[0044] The intelligent agent behavior modeling method, system, readable storage medium, and computer device provided in the embodiments of the present invention have the following beneficial effects:

[0045] According to the intelligent agent behavior modeling method and system of the present invention, by generating a scene knowledge base from acquired scene image data and evolving it into a behavior knowledge base, the intelligent agent behavior modeling method of the present invention possesses the ability to self-evolve, thereby achieving adaptation to unknown scenes. Through inverse reinforcement neural networks and a behavior generation model based on normalized radial basis functions, it is possible to fuse maneuvering behaviors in the current perceived environment with historical behaviors cached in the historical behavior database to generate the final maneuvering behavior of the intelligent agent; the historical behavior database is used to store maneuvering behaviors over a past period of time to assist in the generation of maneuvering behaviors in the current scene.

[0046] The agent behavior modeling method of this invention employs a reinforcement learning strategy to construct a behavior generation model based on a normalized radial basis function, thereby achieving data fusion between a behavior knowledge base and a historical behavior database, which serves as training samples for the model. By combining deep learning with inverse reinforcement learning, the complex correlation between environmental and state features is utilized to optimize the reward function. This combination of deep learning and inverse reinforcement learning enables the learning of the reward function from historical behavioral experience, refines the structure of the reward function from domain knowledge, and overcomes the lack of expert examples for scene situations. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1This is a schematic flowchart illustrating an agent behavior modeling method according to an embodiment of the present invention;

[0049] Figure 2 yes Figure 1 A schematic flowchart of step S102;

[0050] Figure 3 yes Figure 2 A schematic flowchart of step S1022;

[0051] Figure 4 yes Figure 1 A schematic flowchart of step S103;

[0052] Figure 5 yes Figure 1 A schematic flowchart of step S104;

[0053] Figure 6 This is a schematic diagram of the structure of an intelligent agent behavior modeling system according to an embodiment of the present invention;

[0054] Figure 7 This is a schematic block diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] To better understand this invention, the following is combined with... Figures 1 to 7 The present invention provides a detailed description of the intelligent agent behavior modeling method, system, readable storage medium, and computer device. It should be noted that these embodiments are not intended to limit the scope of the invention.

[0057] Figure 1 This is a schematic flowchart illustrating an agent behavior modeling method according to an embodiment of the present invention. Figure 1 As shown, the method may specifically include the following steps:

[0058] S101: Acquire scene image data.

[0059] Specifically, scene image data can be acquired through image sensors. This scene image data includes different images to be recognized for different scenes. A scene can correspond to one image to be recognized, or it can correspond to multiple images to be recognized.

[0060] S102: Generate a scene knowledge base based on scene image data.

[0061] Specifically, scene image data of different scenarios acquired by image sensors are used to calculate and generate scene state vectors corresponding to different scenarios, and the obtained scene state vectors are added to the scene knowledge base.

[0062] In one embodiment, such as Figure 2 As shown, step S102 may specifically include:

[0063] S1021: Extract features from the image to be recognized to obtain a feature map.

[0064] Specifically, the feature maps can include shallow feature maps and deep feature maps, which can be used to extract features from the image to be recognized using a deep convolutional neural network. The deep convolutional neural network includes n convolutional layers. The shallow feature map is determined by the first m convolutional layers, and the deep feature map is determined by the last nm convolutional layers, where m is greater than or equal to 1 and is an integer, and n is greater than m and is an integer. Optionally, the deep convolutional neural network includes 4 convolutional layers, where the shallow feature map is determined by the first two convolutional layers, and the deep feature map is determined by the last two convolutional layers. Furthermore, the feature maps can include c channels, meaning that both the shallow and deep feature maps can each include c channels, where c is greater than or equal to 1 and is an integer.

[0065] S1022: Determine the fused feature map based on the feature map.

[0066] In one embodiment, such as Figure 3 As shown, step S1022 may specifically include:

[0067] S10221: Determine the weight coefficients corresponding to each channel of the feature map.

[0068] Specifically, the features corresponding to each channel are normalized and input into the fully connected layer to obtain the weight vector corresponding to that channel; the weight vector is then mapped to a range of 0 to 1 using the sigmoid function to obtain the weight coefficients.

[0069] The feature map obtained after feature extraction can be represented as a three-dimensional matrix c×h×w, where c represents the number of channels, and h and w represent the height and width of the feature map, respectively. First, the features corresponding to each channel of the feature map are normalized and then input into a fully connected layer to convert them into a c×1×1 weight vector. This vector is then mapped to a weight coefficient value e between 0 and 1 using the sigmoid function. This allows us to determine the weight of each channel in both the shallow and deep feature maps.

[0070] S10222: Multiply the features corresponding to each channel of the shallow feature map with the corresponding weight coefficients and then perform an addition operation to obtain the shallow fused feature map.

[0071] The specific calculation formula is as follows:

[0072]

[0073] in, F represents the shallow fusion feature map. lc This represents the feature corresponding to the c-th channel in the shallow feature map, e lc This represents the weight coefficient corresponding to the c-th channel in the shallow feature map.

[0074] S10223: Multiply the features corresponding to each channel of the deep feature map with the corresponding weight coefficients and then perform an addition operation to obtain the deep fused feature map.

[0075] The specific calculation formula is as follows:

[0076]

[0077] in, F represents the deep fusion feature map. dc This represents the feature corresponding to the c-th channel in the deep feature map, e dc This represents the weight coefficient corresponding to the c-th channel in the deep feature map.

[0078] S10224: Upsample the deep fusion feature map to obtain a cropped deep feature map of the same size as the shallow feature map.

[0079] Specifically, since shallow feature maps and deep feature maps have different feature dimensions, they need to be adjusted to the same scale before fusing multiple sets of feature maps. Deep feature maps, which contain deep feature information, lose a lot of spatial structure information. Therefore, this embodiment of the invention selects shallow feature maps, which contain shallow feature information, as the size standard to upsample the deep feature maps. In this way, the fused feature map carries not only the spatial structure information of the shallow feature maps but also the semantic information of the deep feature maps.

[0080] S10225: Perform an addition operation between the clipped deep feature map and the shallow fused feature map to obtain the fused feature map.

[0081] Specifically, the cropped deep feature map obtained after upsampling is added to the shallow fusion feature map to obtain the fusion feature map. Different weights can be set according to the influence of the cropped deep feature map and the shallow fusion feature map on the final fusion feature map before the addition operation is performed.

[0082] S1023: Determine the scene state vector based on the fused feature map.

[0083] Specifically, after semantic segmentation of the obtained fused feature map, it is then processed through a convolutional neural network to obtain a scene state vector. Each scene corresponds to one or more scene state vectors, which can include information such as the number, location, distance, and direction of equipment (e.g., vehicles, drones) in the scene. The scene state vectors are stored in a scene knowledge base.

[0084] S103: Generate a behavior knowledge base based on the scenario knowledge base.

[0085] In one embodiment, such as Figure 4 As shown, step S103 may specifically include:

[0086] S1031: Randomly generate multiple evolved individuals as the initial population. Each evolved individual includes an action vector describing the agent's maneuvering behavior.

[0087] Specifically, multiple evolved individuals are randomly generated as the initial population for the genetic algorithm. Each evolved individual includes an action vector describing the agent's maneuvering behavior, and the action vector may include information such as position, direction, and distance.

[0088] S1032: Genetic algorithm is used to perform crossover mutation on the initial population to generate multiple mutant individuals.

[0089] Specifically, the initial population is cross-mutated using a genetic algorithm to generate multiple mutated individuals, thereby increasing the population size.

[0090] S1033: Determine the evaluation function based on the scene state vector in the scene knowledge base.

[0091] Specifically, by using the information from the scene state vector, and based on task objectives such as minimizing time, minimizing the probability of being intercepted, and minimizing the probability of being discovered, evaluation indicators can be designed and evaluation functions can be generated. The evaluation function can output evaluation values ​​based on the behavior of the current evolved individual to evaluate the quality of the current individual's behavior.

[0092] S1034: Evaluate the mutated individuals according to the evaluation function, and select the mutated individuals with evaluation values ​​higher than the preset values ​​as the optimal individuals and add them to the behavioral knowledge base.

[0093] Specifically, based on the designed evaluation function and genetic algorithm, the population is continuously cross-mutated and selected to simulate the genetic evolution process in nature, and finally the optimal individual produced in the process is obtained and added to the behavioral knowledge base.

[0094] By designing action vectors of maneuvering behaviors as evolutionary individuals, and using tasks such as minimizing time, minimizing the probability of being intercepted, and minimizing the probability of being detected, the optimal maneuvering behavior is solved using genetic algorithms and evaluation functions, thus forming a behavioral knowledge base.

[0095] The behavior knowledge base is a set of optimal maneuver behaviors that match the scene state vectors in the scene knowledge base, obtained using a genetic algorithm and an evaluation function. First, a population of maneuver behaviors is generated. Then, an evaluation function is designed based on the task objective, and the population is continuously cross-mutated and selected to obtain the optimal individual or population generated in the process, which is then added to the knowledge base. The algorithm's input is the scene state vector corresponding to the current scene, and the output is maneuver behavior data, such as action vectors describing the agent's maneuver behavior.

[0096] S104: Determine the agent's maneuvering behavior based on the behavior knowledge base and the historical behavior base.

[0097] Specifically, the historical behavior database can include historical action vectors that describe the agent's historical maneuvers.

[0098] In one embodiment, such as Figure 5 As shown, step S104 may specifically include:

[0099] S1041: Select the historical action vector that best matches the current scene from the historical behavior library.

[0100] In one embodiment, step S1041 may specifically include:

[0101] Calculate the correlation between the historical action vectors in the historical behavior database and the data in the behavior knowledge base corresponding to the current scene;

[0102] Historical action vectors with a relevance higher than a preset relevance threshold are selected as the historical action vectors that best match the current scene.

[0103] In embodiments of the present invention, when selecting historical action vectors from the historical behavior database, in order to adapt to the current scenario, a correlation calculation is first performed. The historical action vectors are used as samples to perform correlation analysis with the action vectors in the behavior knowledge base generated and evolved in the current scenario. The historical action vectors with a correlation higher than a preset correlation threshold are selected as the historical action vectors that best match the current scenario, and the model training is carried out with experience guidance.

[0104] S1042: The selected historical action vectors are fused with the data from the behavior knowledge base and used as training samples for the behavior generation model.

[0105] Specifically, the selected historical action vectors that best match the current scene are fused with action vectors from the behavior knowledge base and used as training samples for the behavior generation model. The model parameters of the behavior generation model are then optimized using a deep inverse reinforcement learning neural network as the reward function. In one embodiment of the invention, a neural network is used to approximate the data in inverse reinforcement learning, and the neural network is set up before the traditional inverse reinforcement learning model.

[0106] Suppose the reward function has the following formula:

[0107] R(s)=θ T ·f(s)

[0108] Where R(s) is the reward value of the reward function, f(s) is the basis of the reward function, and θ T The coefficients of the reward function are artificially set, where f(s) can be approximated by a neural network, eliminating the need for table lookup.

[0109] Inverse reinforcement learning can learn reward functions from historical behavioral experience without relying on experts to set them. It discovers reward function structures from domain knowledge. By combining deep learning with inverse reinforcement learning, it utilizes the complex correlation between environmental and state features to optimize reward functions. This effectively avoids the problem of insufficient expert examples. Therefore, this embodiment of the invention uses a deep inverse reinforcement neural network to optimize the reward function reward value.

[0110] S1043: Determine the agent's maneuvering behavior based on the behavior knowledge base and the behavior generation model.

[0111] Specifically, the behavior generation model can be a behavior generation model based on normalized radial basis functions. Its input is data from the behavior knowledge base, and its output is the generated maneuver behavior of the agent. The model uses the fusion data from the behavior knowledge base and the historical behavior database as training samples.

[0112] The above describes the agent behavior modeling method according to embodiments of the present invention. The following sections will combine... Figures 6 to 7 A smart agent behavior modeling system and computer device according to embodiments of the present invention are described.

[0113] Figure 6 This is a schematic diagram of the structure of an intelligent agent behavior modeling system 200 according to an embodiment of the present invention, as shown below. Figure 6 As shown, the intelligent agent behavior modeling system 200 includes:

[0114] Data acquisition module 201 is used to acquire scene image data;

[0115] Scene knowledge base generation module 202 is used to generate a scene knowledge base based on scene image data;

[0116] The behavior knowledge base generation module 203 is used to generate a behavior knowledge base based on the scenario knowledge base.

[0117] The maneuver behavior generation module 204 is used to determine the maneuver behavior of the intelligent agent based on the behavior knowledge base and the historical behavior base.

[0118] According to embodiments of the present invention, the agent behavior modeling method and system generate a scene knowledge base using scene image data and evolve it to generate a behavior knowledge base, enabling the agent behavior modeling method of the present invention to have self-evolution capabilities to adapt to unknown scenes. Through inverse reinforcement neural networks and a behavior generation model based on normalized radial basis functions, it is possible to fuse maneuvering behaviors in the current perceived environment with historical behaviors cached in the historical behavior database to generate the agent's final maneuvering behavior. By combining deep learning with inverse reinforcement learning, the reward function is optimized by utilizing the complex correlation between environmental and state features.

[0119] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described intelligent agent behavior modeling method.

[0120] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. Of course, there are other types of readable storage media, such as quantum memories, graphene memories, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0121] Figure 7 This is a structural diagram illustrating an exemplary hardware architecture of a computer device capable of implementing the agent behavior modeling method according to embodiments of the present invention. Figure 7 As shown, the computer device 1000 includes an input device 1001, an input interface 1002, a central processing unit 1003, a memory 1004, an output interface 1005, and an output device 1006. The input interface 1002, the central processing unit 1003, the memory 1004, and the output interface 1005 are interconnected via a bus 1007. The input device 1001 and the output device 1006 are connected to the bus 1007 via the input interface 1002 and the output interface 1005, respectively, and are thus connected to other components of the computer device 1000.

[0122] Specifically, input device 1001 receives input information from the outside and transmits the input information to central processing unit 1003 through input interface 1002; central processing unit 1003 processes the input information based on computer-executable instructions stored in memory 1004 to generate output information, temporarily or permanently stores the output information in memory 1004, and then transmits the output information to output device 1006 through output interface 1005; output device 1006 outputs the output information to the outside of computer device 1000 for user use.

[0123] In one embodiment, Figure 7 The computer device 1000 shown can be implemented as an intelligent agent behavior modeling system, which may include: a memory configured to store programs; and a processor configured to run the programs stored in the memory to execute the intelligent agent behavior modeling method described in the above embodiments.

[0124] According to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network, and / or installed from a removable storage medium.

[0125] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions that, when run on a computer, cause the computer to perform the methods described in the various embodiments above. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0126] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for modeling the behavior of an intelligent agent, characterized in that, include: Acquire scene image data; The scene image data includes images to be identified corresponding to different scenes; A scene knowledge base is generated based on the scene image data; The scenario knowledge base includes scenario state vectors corresponding to multiple different scenarios; The method of generating a scene knowledge base based on the scene image data specifically includes: extracting features from the image to be identified to obtain a feature map; determining a fused feature map based on the feature map; determining a scene state vector based on the fused feature map; and generating a behavior knowledge base based on the scene knowledge base. The method of generating a behavior knowledge base based on the scene knowledge base specifically includes: randomly generating multiple evolved individuals as an initial population, each evolved individual including an action vector describing the agent's maneuvering behavior; performing crossover mutation on the initial population using a genetic algorithm to generate multiple mutated individuals; determining an evaluation function based on the scene state vector in the scene knowledge base; evaluating the mutated individuals according to the evaluation function, and selecting mutated individuals with evaluation values ​​higher than a preset value as the optimal individuals, adding them to the behavior knowledge base; and generating a behavior knowledge base based on the behavior knowledge base. The system uses a knowledge base and a historical behavior database to determine the agent's maneuvering behavior. The historical behavior database includes historical action vectors describing the agent's past maneuvering behaviors. Specifically, determining the agent's maneuvering behavior based on the behavior knowledge base and the historical behavior database includes: selecting the historical action vector from the historical behavior database that best matches the current scene; calculating the correlation between the historical action vector in the historical behavior database and the data in the behavior knowledge base corresponding to the current scene; selecting historical action vectors with a correlation higher than a preset correlation threshold as the historical action vectors best matching the current scene; fusing the selected historical action vectors with the data in the behavior knowledge base and using them as training samples for the behavior generation model; and determining the agent's maneuvering behavior based on the behavior knowledge base and the behavior generation model.

2. The agent behavior modeling method according to claim 1, characterized in that, The feature map includes a shallow feature map and a deep feature map; the feature map includes c channels, where c is greater than or equal to 1 and is an integer; the determination of the fused feature map based on the feature map specifically includes: determining the weight coefficient corresponding to each channel of the feature map; multiplying the feature of each channel of the shallow feature map with the corresponding weight coefficient and then performing an addition operation to obtain a shallow fused feature map; multiplying the feature of each channel of the deep feature map with the corresponding weight coefficient and then performing an addition operation to obtain a deep fused feature map; upsampling the deep fused feature map to obtain a clipped deep feature map of the same size as the shallow feature map; and performing an addition operation between the clipped deep feature map and the shallow fused feature map to obtain the fused feature map.

3. The agent behavior modeling method according to claim 2, characterized in that, The determination of the weight coefficient corresponding to each channel of the feature map specifically includes: normalizing the features corresponding to each channel and inputting them into a fully connected layer to obtain the weight vector corresponding to that channel; and mapping the weight vector to a range of 0 to 1 using the sigmoid function to obtain the weight coefficient.

4. The agent behavior modeling method according to claim 1, characterized in that, The behavior generation model is a behavior generation model based on normalized radial basis functions. Its input is the data of the behavior knowledge base, and its output is the generated maneuver behavior of the intelligent agent.

5. The agent behavior modeling method according to claim 1, characterized in that, A deep inverse reinforcement neural network is used as a reward function to optimize the model parameters of the behavior generation model.

6. A system for modeling the behavior of an intelligent agent, characterized in that, include: The data acquisition module is used to acquire scene image data; The scene image data includes images to be identified corresponding to different scenes; A scene knowledge base generation module is used to generate a scene knowledge base based on the scene image data; the scene knowledge base includes scene state vectors corresponding to multiple different scenes; the scene knowledge base generation module is specifically used to: extract features from the image to be identified to obtain a feature map; determine a fused feature map based on the feature map; and determine scene state vectors based on the fused feature map. A behavior knowledge base generation module is used to generate a behavior knowledge base based on the scene knowledge base; the behavior knowledge base generation module is specifically used to: randomly generate multiple evolved individuals as an initial population, each evolved individual including an action vector describing the agent's maneuvering behavior; perform crossover mutation on the initial population using a genetic algorithm to generate multiple mutated individuals; determine an evaluation function based on the scene state vectors in the scene knowledge base; and evaluate the mutated individuals based on the evaluation function. The system selects the mutant individuals with evaluation values ​​higher than a preset value as the optimal individuals and adds them to the behavior knowledge base. A maneuver behavior generation module is used to determine the maneuver behavior of the agent based on the behavior knowledge base and a historical behavior database. The historical behavior database includes historical action vectors describing the agent's historical maneuver behavior. Specifically, the maneuver behavior generation module is used to: select the historical action vector in the historical behavior database that best matches the current scene; calculate the correlation between the historical action vector in the historical behavior database and the data in the behavior knowledge base corresponding to the current scene; select the historical action vector with a correlation higher than a preset correlation threshold as the historical action vector that best matches the current scene; and fuse the selected historical action vector with the data in the behavior knowledge base as training samples for the behavior generation model. The agent's maneuvering behavior is determined based on the behavioral knowledge base and the behavioral generation model.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the agent behavior modeling method as described in any one of claims 1-5.

8. A computer device, characterized in that, It includes: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the agent behavior modeling method as described in any one of claims 1-5.

Images