Agent-based behavior simulation method and device, electronic equipment and storage medium

Abstract

The application provides an agent-based behavior simulation method and device, electronic equipment and storage medium, and relates to the technical field of artificial intelligence. Three-dimensional perception data of the environment around the agent and agent state data are obtained. Environment state features are extracted from the three-dimensional perception data, and agent state features are extracted from the agent state data. Target state features are obtained based on the environment state features and the agent state features, and agent simulation behavior features are obtained based on the target state features. The target state features include state features of multiple channels and state features of multiple spaces, thereby solving the technical problem that the simulation effect of micro population behavior cannot be effectively improved in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a behavior simulation method and device based on an agent, an electronic device and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, reinforcement learning, as one of the branches of artificial intelligence technology, has received more and more attention and has been applied in a wider range. With the further application and promotion of reinforcement learning, applying reinforcement learning models to simulate micro population behavior has gradually become a research hotspot in the related technical field. Vision is the most important way for humans to perceive the external environment. However, due to the constraints of computing power and computing resources, there are still many difficulties in realizing large-scale micro population behavior simulation by relying only on visual perception.

[0003] In the prior art, a ray perception method is used to replace the visual perception method to simulate micro population behavior, so as to solve the problem that the visual perception method is constrained by computing power and computing resources. However, since there is still a big gap between the ray perception method and the visual perception method in terms of the perception range, etc., there is a big deviation between the object types and the range perceived by the ray perception method and the actual situation, and the problem of loss of environmental space information is easily caused, thereby causing the simulation effect of micro population behavior to not meet the actual application requirements.

[0004] Therefore, how to effectively improve the simulation effect of micro population behavior in the prior art is a technical problem to be solved by the related field technical personnel. SUMMARY

[0005] The present application provides a behavior simulation method and device based on an agent, an electronic device and a storage medium, to solve the technical problem that the simulation effect of micro population behavior cannot be effectively improved in the prior art.

[0006] The present application provides a behavior simulation method based on an agent, comprising: acquiring three-dimensional perception data of the environment around the agent and agent state data; extracting environment state features from the three-dimensional perception data and agent state features from the agent state data; acquiring target state features based on the environment state features and the agent state features, and acquiring simulation behavior features of the agent based on the target state features, wherein the target state features include state features of multiple channels and state features of multiple spaces.

[0007] The application provides an agent-based behavior simulation method, which comprises the following steps: obtaining a target state feature based on an environment state feature and an agent state feature; and obtaining an agent simulation behavior feature based on the target state feature, wherein the environment state feature comprises a plurality of channel state features and a plurality of space state features; the target state feature is obtained based on the agent state feature and the environment state feature; and the simulation behavior feature comprises a rotation behavior feature and a movement behavior feature.

[0008] The application provides an agent-based behavior simulation method, wherein the step of adjusting the channel weight of the environment state feature and adjusting the space weight of the environment state feature to obtain the environment state feature after weight adjustment comprises the following steps: adjusting the channel weight of the environment state feature, and obtaining a first weight feature corresponding to the state feature of different channels in the environment state feature after channel weight adjustment; obtaining a first state feature after channel weight adjustment based on the environment state feature and the first weight feature; adjusting the space weight of the first state feature, and obtaining a second weight feature corresponding to the state feature of different spaces in the first state feature after space weight adjustment; obtaining a second state feature after space weight adjustment based on the first state feature and the second weight feature, and taking the second state feature as the environment state feature after weight adjustment.

[0009] The application provides an agent-based behavior simulation method, wherein the step of adjusting the channel weight of the environment state feature, and obtaining a first weight feature corresponding to the state feature of different channels in the environment state feature after channel weight adjustment comprises the following steps: aggregating the space state feature on each channel in the environment state feature based on an average pooling layer to obtain a first average pooling feature; aggregating the space state feature on each channel in the environment state feature based on a maximum pooling layer to obtain a first maximum pooling feature; adjusting the channel weight of the first average pooling feature and the first maximum pooling feature; adding the first average pooling feature and the first maximum pooling feature after channel weight adjustment to obtain a first weight adjustment feature; and normalizing the first weight adjustment feature to obtain the first weight feature corresponding to the state feature of different channels in the environment state feature after channel weight adjustment.

[0010] According to the behavior simulation method based on the agent provided by the application, the first state feature is subjected to spatial weight adjustment, and a second weight feature corresponding to the state features of different spaces in the first state feature after the spatial weight adjustment is obtained, which comprises: based on an average pooling layer, channel state features on each space in the first state feature are aggregated to obtain a second average pooling feature; based on a maximum pooling layer, channel state features on each space in the first state feature are aggregated to obtain a second maximum pooling feature; based on a channel direction, the second average pooling feature and the second maximum pooling feature are subjected to feature splicing and convolution processing to obtain a second weight adjustment feature; the second weight adjustment feature is subjected to normalization processing to obtain the second weight feature corresponding to the state features of different spaces in the first state feature.

[0011] According to the behavior simulation method based on the agent provided by the application, the three-dimensional perception data of the environment around the agent is obtained, which comprises: based on a pre-constructed visual perception-like module, environment perception data of the environment around the agent is obtained, the visual perception-like module comprises horizontal perception rays arranged in the horizontal line-of-sight direction of the agent and vertical perception rays arranged in the vertical line-of-sight direction of the agent; based on the environment perception data, the three-dimensional perception data is obtained; the three-dimensional perception data comprises horizontal dimension data, vertical dimension data and type dimension data, the horizontal dimension information represents the position of the perception ray in the horizontal direction, the vertical dimension data represents the position of the perception ray in the vertical direction, and the type dimension data represents the type of the environment perception data obtained by the perception ray.

[0012] The application further provides a behavior simulation device based on an agent, comprising: a data acquisition module, configured to acquire three-dimensional perception data of an environment around an agent and agent state data; a feature extraction module, configured to extract environment state features from the three-dimensional perception data and agent state features from the agent state data; a behavior simulation module, configured to acquire target state features based on the environment state features and the agent state features, and acquire simulation behavior features of the agent based on the target state features, wherein the target state features comprise state features of multiple channels and state features of multiple spaces.

[0013] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the behavior simulation method based on the agent according to any one of the above when executing the program.

[0014] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the behavior simulation method based on the agent according to any one of the above.

[0015] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the agent-based behavior simulation method according to any one of the above.

[0016] The agent-based behavior simulation method, device, electronic equipment and storage medium provided by the application can obtain target state features with a relatively large amount of information by combining three-dimensional perception data of the environment around the agent and state data of the agent, so that the simulation behavior features of the agent can be accurately simulated based on the target state features, thereby eliminating the defects of the prior art, such as the error in the perception result and the loss of environmental space information caused by replacing the visual perception method with the ray perception method, effectively improving the simulation effect of micro population behavior, and solving the technical problem that the simulation effect of micro population behavior cannot be effectively improved in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0018] Figure 1 is one of the flowcharts of the agent-based behavior simulation method provided by the application;

[0019] Figure 2 is another flowchart of the agent-based behavior simulation method provided by the application;

[0020] Figure 3 is a third flowchart of the agent-based behavior simulation method provided by the application;

[0021] Figure 4 is a fourth flowchart of the agent-based behavior simulation method provided by the application;

[0022] Figure 5 is a fifth flowchart of the agent-based behavior simulation method provided by the application;

[0023] Figure 6a is a sixth flowchart of the agent-based behavior simulation method provided by the application;

[0024] Figure 6b is a schematic diagram of the horizontal perception ray in the visual perception module in the embodiment of the application;

[0025] Figure 6cis a schematic diagram of a vertical perception ray in a visual perception module in an embodiment of the present application;

[0026] Figures 7a to 7d is a schematic diagram of a crowd simulation result based on a three-dimensional crowded environment in a specific embodiment two of the present application;

[0027] Figures 8a to 8h is a schematic diagram of a crowd simulation result based on a three-dimensional unknown environment in a specific embodiment two of the present application;

[0028] Figure 9 is a structural schematic diagram of the behavior simulation device based on an agent provided by the present application;

[0029] Figure 10 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0031] The following will be described in combination with Figure 1 Figure 6 describes the behavior simulation method based on an agent of the present application. As shown in the figure, the present application provides a behavior simulation method based on an agent, which comprises: Figure 1

[0032] Step S1, obtaining three-dimensional perception data of the environment around the agent and agent state data.

[0033] Among them, the three-dimensional perception data includes the perception data of the environment around the agent and the perception data between the agent and the surrounding environment. The agent state data represents the state data of the agent itself.

[0034] Step S2, extracting environment state features from the three-dimensional perception data, and extracting agent state features from the agent state data.

[0035] A deep learning-based feature extraction network can be used to extract environment state features from three-dimensional perception data and agent state features from agent state data. Other feature extraction networks can also be used to realize the feature extraction function, and the present application does not make specific limitations.

[0036] ​In step S3, a target state feature is obtained based on the environment state feature and the agent state feature, and a simulation behavior feature of the agent is obtained based on the target state feature. The target state feature includes state features of multiple channels and state features of multiple spaces.

[0037] Further, the environment state feature and the agent state feature are subjected to feature splicing processing, and a target state feature is obtained based on the state feature after feature splicing. The target state feature contains all key state features in the three-dimensional perception data and the agent state data.

[0038] The target state feature can be converted into the simulation behavior feature of the agent by using a feature conversion network based on deep learning, or other feature conversion methods can be used to realize the feature conversion function, and the present application does not make specific limitations. The steps S1 to S3 can obtain a target state feature with a relatively large amount of information by combining the three-dimensional perception data of the environment around the agent and the agent state data, that is, the target state feature contains channel state features and space state features with a relatively large amount of information. Therefore, the simulation behavior feature of the agent can be accurately simulated based on the target state feature, so as to eliminate the defects of the existing technology that the perception result has errors and the environmental space information is lost when the ray perception method is used instead of the visual perception method, effectively improve the simulation effect of the micro crowd behavior, and solve the technical problem that the simulation effect of the micro crowd behavior cannot be effectively improved in the prior art.

[0040] In one embodiment, the step S1 includes obtaining auxiliary perception data based on the agent, encoding the auxiliary perception data into a one-dimensional vector, and normalizing the one-dimensional vector to obtain agent state data. The agent state data includes the current position of the agent, the current walking direction (i.e., the current orientation of the agent), and the current walking speed. Further, the numerical value of the normalized agent state data is kept within the numerical range of [-1, 1].

[0041] In one embodiment, the step S1 further includes normalizing the environment perception data and encoding the normalized environment perception data into three-dimensional perception data. The three-dimensional perception data is a three-dimensional matrix. The three-dimensional perception data includes the types of objects around the agent, the distances between the agent and the surrounding objects, and the height differences between the agent and the surrounding objects. Further, the numerical value of the normalized environment perception data is kept within the numerical range of [0, 1].

[0042] In one embodiment, as shown in FIG. 1, Figure 2 The step S3 includes steps S31 to S33, wherein:

[0043] Step S31, adjusting the channel weight and the spatial weight of the environment state feature to obtain the environment state feature after weight adjustment; the environment state feature includes state features of multiple channels and state features of multiple spaces.

[0044] The "channel" represents a data type of the three-dimensional perception data of the environment around the agent. The "space" represents two spatial dimensions corresponding to the three-dimensional perception data of the environment around the agent, i.e., a horizontal spatial dimension and a vertical spatial dimension. The "three-dimensional" represents the three dimensions of the horizontal spatial dimension, the vertical spatial dimension, and the channel dimension, wherein the channel dimension is the data type dimension.

[0045] Step S32, obtaining the target state feature based on the agent state feature and the environment state feature after weight adjustment.

[0046] Step S33, obtaining the simulation behavior feature of the agent based on the target state feature, wherein the simulation behavior feature includes a rotation behavior feature and a movement behavior feature.

[0047] In one embodiment, the above steps S31 and S32 specifically include the following steps:

[0048] (1) Constructing a feature extraction network based on deep learning. The environment state feature is input into a convolution layer (Conv), an attention mechanism layer (AM), and a fully connected layer (FC) in the feature extraction network to extract a first key state feature with more abundant information from the environment state feature. The attention mechanism layer (AM) is used to adjust the channel weight and the spatial weight of the environment state feature to enhance important channel or spatial state features and suppress unimportant channel or spatial state features.

[0049] (2) Inputting the agent state feature into two fully connected layers (FC) in the feature extraction network to extract a second key state feature with more abundant information from the agent state feature. The first key state feature and the second key state feature are spliced into a target state feature containing all key state features in the three-dimensional perception data and the agent state data, so that the target state feature contains channel state features and spatial state features with abundant information and high importance, thereby eliminating the defect of the loss of environmental spatial information caused by replacing the visual perception method with the ray perception method in the prior art, and further improving the simulation effect of the simulation behavior feature of the agent.

[0050] The above feature extraction process uses a nonlinear activation function to introduce a nonlinear factor in the feature extraction network, so that the feature extraction network approximates the state feature function of the environment around the agent and the state feature function of the agent.

[0051] In one embodiment, the step S33 comprises: constructing a behavior simulation network based on deep learning, inputting the target state feature with adjusted weights into one fully connected layer (FC) in the behavior simulation network to obtain a rotation behavior feature, and inputting the target state feature with adjusted weights into another fully connected layer (FC) in the behavior simulation network to obtain a moving behavior feature.

[0052] In one embodiment, Figure 3 For the specific steps of adjusting the channel weight and the spatial weight of the environment state feature based on the attention mechanism layer in the present application, as shown in Figure 3 The step S32 comprises steps S321 to S324, wherein:

[0053] The step S321 adjusts the channel weight of the environment state feature, and obtains first weight features corresponding to state features of different channels in the environment state feature after channel weight adjustment.

[0054] The step S322 obtains first state features after channel weight adjustment based on the environment state feature and the first weight features.

[0055] Further, the first state features after channel weight adjustment are obtained by element multiplication of the environment state feature and the first weight features.

[0056] The step S323 adjusts the spatial weight of the first state feature, and obtains second weight features corresponding to state features of different spaces in the first state feature after spatial weight adjustment.

[0057] The step S324 obtains second state features after spatial weight adjustment based on the first state feature and the second weight features, and takes the second state features as the environment state feature with adjusted weights.

[0058] Further, the second state features after spatial weight adjustment are obtained by element multiplication of the first state feature and the second weight features.

[0059] In one embodiment, as shown in Figure 4 The step S321 comprises steps S41 to S45, wherein:

[0060] The step S41 aggregates spatial state features on each channel of the environment state feature based on an average pooling layer to obtain first average pooling features.

[0061] The step S42 aggregates spatial state features on each channel of the environment state feature based on a maximum pooling layer to obtain first maximum pooling features.

[0062] Step S43: Adjust the channel weights of the first average pooling feature and the first max pooling feature.

[0063] Step S44: Add the first average pooling feature and the first max pooling feature after adjusting the channel weights element by element to obtain the first weight-adjusted feature.

[0064] Step S45: Normalize the first weight adjustment feature to obtain the first weight feature corresponding to the state features of different channels in the environmental state features after channel weight adjustment.

[0065] In one embodiment, steps S41 to S43 include: constructing a weight adjustment network based on an attention mechanism; inputting environmental state features into an average pooling layer in the weight adjustment network to obtain a first average pooling feature; inputting the environmental state features into a max pooling layer in the weight adjustment network to obtain a first max pooling feature; and inputting the first average pooling feature and the first max pooling feature into two fully connected layers (FC) in the weight adjustment network, respectively, so that the fully connected layers adjust the channel weights of the first average pooling feature and the first max pooling feature through weight sharing between fully connected layers.

[0066] Furthermore, the aforementioned environmental state features, first average pooling features, first max pooling features, and first weight adjustment features are feature vectors or feature maps. The first weight feature is a weight vector.

[0067] In one embodiment, such as Figure 5 As shown, step S323 above includes steps S51 to S54, wherein:

[0068] Step S51: Aggregate the channel state features in each space of the first state features based on the average pooling layer to obtain the second average pooling feature.

[0069] Step S52: Aggregate the channel state features in each space of the first state features based on the max pooling layer to obtain the second max pooling feature.

[0070] Step S53: Based on the channel direction, the second average pooling feature and the second max pooling feature are concatenated and convolved to obtain the second weighted adjustment feature.

[0071] Step S54: Normalize the second weight adjustment feature to obtain the second weight feature corresponding to the state features of different spaces in the first state feature.

[0072] In one embodiment, steps S51 to S53 include: constructing a weight adjustment network based on an attention mechanism; inputting the first state feature into the average pooling layer of the weight adjustment network to obtain a second average pooling feature; inputting the first state feature into the max pooling layer of the weight adjustment network to obtain a second max pooling feature; concatenating the second average pooling feature and the second max pooling feature along the channel direction to obtain a concatenated pooling feature; and performing convolution processing on the concatenated pooling feature to achieve feature fusion and weight adjustment of the second average pooling feature and the second max pooling feature in the concatenated pooling feature to obtain a second weight-adjusted feature.

[0073] Furthermore, the aforementioned first state feature, second average pooling feature, second max pooling feature, concatenated pooling feature, and second weight adjustment feature are feature vectors or feature maps. The second weight feature is a weight vector.

[0074] In one embodiment, such as Figure 6a As shown, step S1 above includes steps S11 to S12, wherein:

[0075] Step S11: Obtain environmental perception data of the surrounding environment of the agent based on a pre-built vision-like perception module. The vision-like perception module includes a horizontal perception ray set in the horizontal line of sight of the agent and a vertical perception ray set in the vertical line of sight of the agent.

[0076] The horizontal and vertical sensing rays are used to actively acquire environmental perception data such as the types of objects around the agent and the distances between the agent and these objects. The height difference between the agent and the surrounding objects is determined by the position of the sensing rays. The agent's horizontal line of sight represents the direction of sight parallel to the horizontal plane. The agent's vertical line of sight represents the direction of sight perpendicular to the horizontal plane.

[0077] Step S12: Obtain three-dimensional perception data based on environmental perception data; the three-dimensional perception data includes horizontal dimension data, vertical dimension data and type dimension data. The horizontal dimension information represents the position of the perception ray in the horizontal direction, the vertical dimension data represents the position of the perception ray in the vertical direction, and the type dimension data represents the type of environmental perception data obtained by the perception ray.

[0078] It should be noted that when the environmental perception data includes three types of perception data, the mixed state feature in step S31 includes state features of three channels, that is, the number of channels of the mixed state feature is the number of data types of the environmental perception data.

[0079] In one embodiment, the environment perception data is encoded as three-dimensional perception data, i.e., a three-dimensional matrix, and the three-dimensional perception data is normalized so that all the environment perception data obtained by the perception ray is kept in the numerical range of [0, 1] to obtain normalized three-dimensional perception data, wherein the normalization method is shown in the following formula (1) and formula (2):

[0080] C i,j =c i,j / c (1)

[0081] D i,j =d i,j / d (2)

[0082] wherein c represents the total number of types of objects around the agent detected by the perception ray, d represents the maximum distance between the agent and its surrounding objects detected by the perception ray, c i,j represents the types of objects around the agent detected by the perception ray r i,j , and d i,j represents the distance between the agent and its surrounding objects detected by the perception ray r i,j ; C i,j represents the normalized result of the types of objects around the agent, and C i,j ∈[0, 1]. D i,j represents the normalized result of the distance between the agent and its surrounding objects, and D i,j ∈[0, 1].

[0083] In one embodiment, the horizontal perception ray includes a first horizontal perception ray set based on a first horizontal field of view range and a first horizontal distribution density, and a second horizontal perception ray set based on a second horizontal field of view range and a second horizontal distribution density; the second horizontal field of view range is set within the first horizontal field of view range, and the second horizontal distribution density is greater than the first horizontal distribution density.

[0084] Preferably, as shown in Figure 6b , the horizontal line of sight center direction (i.e., the 90-degree direction in Figure 6b ) is taken as the reference, the 60-degree range above and below it is set as the first horizontal field of view range (i.e., the 30-degree to 150-degree field of view range in Figure 6b ), and the 30-degree range above and below it is set as the second horizontal field of view range (i.e., the 60-degree to 120-degree field of view range in Figure 6b ). The first horizontal distribution density is to set one first horizontal perception ray every 15 degrees. The second horizontal distribution density is to set one second horizontal perception ray every 10 degrees.

[0085] In one embodiment, the vertical perception rays include first vertical perception rays set based on a first vertical field of view range and a first vertical distribution density, and second vertical perception rays set based on a second vertical field of view range and a second vertical distribution density; the second vertical field of view range is set within the first vertical field of view range, and the second vertical distribution density is greater than the first vertical distribution density.

[0086] Preferably, as shown in FIG. 1, the 60-degree range above and below the vertical line of sight center direction of the agent (i.e., the 90-degree direction in FIG. 1) is set as the first vertical field of view range (i.e., the 30-degree to 150-degree field of view range in FIG. 1), and the 30-degree range above and below the vertical line of sight center direction of the agent is set as the second vertical field of view range (i.e., the 60-degree to 120-degree field of view range in FIG. 1). The first vertical distribution density is one first vertical perception ray set every 15 degrees. The second vertical distribution density is one second vertical perception ray set every 10 degrees. Figure 6c Figure 6c Figure 6c Figure 6c

[0087] Two specific embodiments are provided below to further illustrate the agent-based behavior simulation method provided by the present application.

[0088] In a specific embodiment one, the agent-based behavior simulation method provided by the present application includes the following steps:

[0089] Step 1: Obtain three-dimensional perception data of the environment around the agent and agent state data; extract environment state features from the three-dimensional perception data, and extract agent state features from the agent state data; the environment state features include state features of multiple channels and state features of multiple spaces.

[0090] Step 2: Adjust the channel weights of the environment state features, and obtain the first weight features corresponding to the state features of different channels in the environment state features after the channel weight adjustment; based on the environment state features and the first weight features, obtain the first state features after the channel weight adjustment; adjust the space weights of the first state features, and obtain the second weight features corresponding to the state features of different spaces in the first state features after the space weight adjustment; based on the first state features and the second weight features, obtain the second state features after the space weight adjustment, and take the second state features as the environment state features after the weight adjustment.

[0091] Step 3: Based on the agent state features and the environment state features after the weight adjustment, obtain target state features; based on the target state features, obtain the simulation behavior features of the agent, including rotation behavior features and movement behavior features. In a specific embodiment two, the agent-based behavior simulation method provided by the present application is implemented by using a reinforcement learning algorithm, specifically including the following steps: ​​​​

[0092] Step (1): Acquire the 3D perception data of the environment surrounding the agent and the agent's state data at the current moment. The 3D perception data includes the types of objects around the agent, the distance between the agent and the surrounding objects, and the height difference between the agent and the surrounding objects. The agent's state data includes the agent's current position, current walking direction, and current walking speed.

[0093] Step (2): Input the 3D perception data and agent state data into the pre-constructed behavior decision network to obtain the simulated behavior features of the agent at the current moment. The simulated behavior features include rotation behavior features and movement behavior features, where the rotation behavior feature is the rotation angle of the agent, and the movement behavior feature is the walking speed of the agent. The behavior decision network includes a feature extraction network, a feature transformation network, and a behavior simulation network built based on deep learning, as well as a weight adjustment network built based on an attention mechanism. The feature extraction network uses a non-linear activation function, such as the ELU activation function.

[0094] Step (3): Control the agent to execute the action corresponding to the simulated behavioral characteristics, so that the agent's state data and the agent's surrounding environment change accordingly in the next moment. Obtain the current reward value at the current moment, and update the network parameters of the behavior decision network based on the current reward value; repeat the above steps until a preset number of executions is reached or a preset learning objective is achieved, wherein the preset learning objective is that the agent's cumulative reward value reaches the maximum value, or the agent's cumulative reward value tends to a stable reward value.

[0095] Furthermore, the state of the agent's surrounding environment at the current moment is s. t After the agent executes the action corresponding to the simulated behavioral feature, the agent's state data at the next time t+1 includes the agent's next orientation θ. t+1 and the next position p t+1 This causes the state of the agent's surrounding environment to change accordingly (i.e., s t →s t+1 ), where the agent's next orientation θ t+1 and the next position p t+1 As shown in formulas (3) and (4) respectively:

[0096] θ t+1 =θ t +Δ t ×ω t (3)

[0097] p t+1 =p t +Δt×v t (4)

[0098] wherein, θ t represents the current orientation of the agent at the current moment, θ t+1 represents the next orientation of the agent at the next moment, Δt represents the time interval between the current moment and the next moment, ω t represents the current rotation angle of the agent at the current moment, p t represents the current position of the agent at the current moment, p t+1 represents the next position of the agent at the next moment, v t represents the current walking speed of the agent at the current moment.

[0099] In summary, the agent-based behavior simulation method provided by the present application can place three-dimensional perception data in a three-dimensional environment for behavior simulation and data calculation by constructing a behavior decision network suitable for three-dimensional perception data, thereby avoiding the defects of the prior art, such as the loss of part of the environmental space information caused by using a two-dimensional environment for behavior simulation and data calculation, obtaining more accurate behavior simulation results, and further improving the simulation effect of micro crowd behavior.

[0100] The "bottleneck effect" is a typical self-organizing feature of a crowded crowd, which is specifically manifested as an arch-shaped phenomenon formed by the crowd at a narrow passage, and it occurs widely in public places such as schools, shopping malls, and subways. However, due to the highly crowded environment and the extremely narrow passage, the existing classical micro crowd simulation method will cause the agent to be unbalanced in force or the behavior to be chaotic when simulating the "bottleneck effect", which is specifically manifested as the arch being flat.

[0101] Figures 7a to 7d is a schematic diagram of the crowd simulation result based on the three-dimensional crowded environment in the second embodiment of the present application, as shown in Figures 7a to 7d The agent state data further includes the destination of the agent. In the initial stage, all agents move towards the passage (as shown in Figure 7a When some agents reach the passage, the agents cannot all pass through the passage at the same time due to the extremely narrow passage, which causes some agents to be blocked at the passage (as shown in Figure 7b The blocked agents always gather at the passage, that is, the whole shows the "arch" feature, until all agents pass through the passage (as shown in Figures 7c to 7d The simulation result conforms to the basic characteristics of the "bottleneck effect" of the crowded crowd, and all agents do not show obvious abnormal behavior in the whole simulation process.

[0102] Navigation and obstacle avoidance are behaviors or skills that an intelligent agent must have, that is, the intelligent agent does not collide with obstacles in the process of reaching the target, which often occurs in scenes with obstacles, and such scenes also exist widely in public places such as shopping malls and schools. Since the current microscopic crowd evacuation method generally depends on a two-dimensional environment (without considering the height information of the environment), it is difficult to realize the navigation and obstacle avoidance of intelligent agents in a three-dimensional unknown environment.

[0103] Figures 8a to 8h is a schematic diagram of the crowd simulation result based on a three-dimensional unknown environment in embodiment two of the present application. When the intelligent agent-based behavior simulation method provided in the above embodiment two is applied to the crowd simulation process in a three-dimensional crowded environment, the intelligent agent state data does not include the destination of the intelligent agent. In the initial stage, all intelligent agents move towards the destination (as shown in Figure 8a , the destination is the target in Figure 8a ), when encountering obstacles of different heights, the intelligent agents can actively avoid the obstacles and move between the obstacles (as shown in Figures 8b to 8c ), until finally reaching the destination (as shown in Figure 8d ). In addition, the motion trajectories of the intelligent agents are relatively random and dispersed, and relatively far from the walls on both sides, which is also more in line with the motion characteristics of real people.

[0104] In order to further test the ability of the intelligent agent-based behavior simulation method provided in the present application to navigate and avoid obstacles of intelligent agents in a three-dimensional unknown environment, the positions and heights of the obstacles in the three-dimensional unknown environment are randomly changed (as shown in Figures 8e to 8h ), and the intelligent agent-based behavior simulation method is used to simulate the navigation and obstacle avoidance of intelligent agents. As shown in Figures 8e to 8h , the intelligent agents can reach the destination while avoiding collisions, and their trajectories are relatively random and dispersed, and relatively far from the walls on both sides. It can be seen that the simulation result obtained by changing the positions and heights of the obstacles is very similar to the simulation result shown in Figures 8a to 8d , which indicates that the intelligent agents can better handle the navigation and obstacle avoidance tasks in a three-dimensional unknown environment, and also indicates that the intelligent agent-based behavior simulation method provided in the present application has strong robustness.

[0105] Figures 8e to 8h is a schematic diagram of the crowd simulation result based on a three-dimensional unknown environment in embodiment two of the present application, as shown in Figures 8e to 8h .

[0106] The intelligent agent-based behavior simulation device provided in the present application is described below. The intelligent agent-based behavior simulation device described below can be mutually corresponding to the intelligent agent-based behavior simulation method described above.

[0107] As shown in Figure 9As shown, the present application provides an agent-based behavior simulation device, the agent-based behavior simulation device 100 comprises a data acquisition module 10, a feature extraction module 20 and a behavior simulation module 30, wherein:

[0108] The data acquisition module 10 is configured to acquire three-dimensional perception data of the environment around the agent and agent state data.

[0109] The feature extraction module 20 is configured to extract environment state features from the three-dimensional perception data and agent state features from the agent state data.

[0110] The behavior simulation module 30 is configured to acquire target state features based on the environment state features and the agent state features, and to acquire simulation behavior features of the agent based on the target state features, the target state features comprising state features of multiple channels and state features of multiple spaces.

[0111] In one embodiment, the behavior simulation module 30 comprises a weight adjustment unit, a feature fusion unit and a behavior simulation unit, wherein:

[0112] The weight adjustment unit is configured to perform channel weight adjustment and spatial weight adjustment on the environment state features to obtain weight-adjusted environment state features; the environment state features comprise state features of multiple channels and state features of multiple spaces.

[0113] The feature fusion unit is configured to acquire target state features based on the agent state features and the weight-adjusted environment state features.

[0114] The behavior simulation unit is configured to acquire simulation behavior features of the agent based on the target state features, the simulation behavior features comprising rotation behavior features and movement behavior features.

[0115] In one embodiment, the weight adjustment unit comprises a first weight adjustment subunit, a first feature acquisition subunit, a second weight adjustment subunit and a second feature acquisition subunit, wherein:

[0116] The first weight adjustment subunit is configured to perform channel weight adjustment on the environment state features and to acquire first weight features corresponding to state features of different channels in the channel weight-adjusted environment state features.

[0117] The first feature acquisition subunit is configured to obtain first state features after channel weight adjustment based on the environment state features and the first weight features.

[0118] The second weight adjustment subunit is configured to perform spatial weight adjustment on the first state features and to acquire second weight features corresponding to state features of different spaces in the spatial weight-adjusted first state features.

[0119] The second feature acquisition subunit is configured to obtain a second state feature after spatial weight adjustment based on the first state feature and the second weight feature, and take the second state feature as the environment state feature after weight adjustment.

[0120] In one embodiment, the first weight adjustment subunit is further configured to aggregate the spatial state features on each channel in the environment state feature based on an average pooling layer to obtain a first average pooling feature; aggregate the spatial state features on each channel in the environment state feature based on a maximum pooling layer to obtain a first maximum pooling feature; perform channel weight adjustment on the first average pooling feature and the first maximum pooling feature; perform element addition on the first average pooling feature and the first maximum pooling feature after channel weight adjustment to obtain a first weight adjustment feature; and perform normalization processing on the first weight adjustment feature to obtain the first weight feature corresponding to the state features of different channels in the environment state feature after channel weight adjustment.

[0121] In one embodiment, the second weight adjustment subunit is further configured to aggregate the channel state features on each space in the first state feature based on an average pooling layer to obtain a second average pooling feature; aggregate the channel state features on each space in the first state feature based on a maximum pooling layer to obtain a second maximum pooling feature; perform feature splicing and convolution processing on the second average pooling feature and the second maximum pooling feature based on a channel direction to obtain a second weight adjustment feature; and perform normalization processing on the second weight adjustment feature to obtain the second weight feature corresponding to the state features of different spaces in the first state feature.

[0122] In one embodiment, the data acquisition module 10 comprises a perception data acquisition unit and a perception data encoding unit, wherein:

[0123] The perception data acquisition unit is configured to acquire environment perception data of an environment surrounding an agent based on a pre-constructed visual perception-like module, the visual perception-like module comprising horizontal perception rays arranged in a horizontal line-of-sight direction of the agent and vertical perception rays arranged in a vertical line-of-sight direction of the agent.

[0124] The perception data encoding unit is configured to acquire three-dimensional perception data based on the environment perception data; the three-dimensional perception data comprising horizontal dimension data, vertical dimension data, and type dimension data, the horizontal dimension data representing a position of the perception rays in a horizontal direction, the vertical dimension data representing a position of the perception rays in a vertical direction, and the type dimension data representing a type of the environment perception data acquired by the perception rays.

[0125] Figure 10 An example of a schematic diagram of the physical structure of an electronic device is shown in FIG. 1. Figure 10As shown, the electronic device can include a processor 1010, a communications interface 1020, a memory 1030, and a communications bus 1040, wherein the processor 1010, the communications interface 1020, and the memory 1030 complete mutual communication through the communications bus 1040. The processor 1010 can invoke a logic instruction in the memory 1030 to execute an agent-based behavior simulation method, which includes: obtaining three-dimensional perception data of an environment around an agent and agent state data; extracting environment state features from the three-dimensional perception data and agent state features from the agent state data; obtaining target state features based on the environment state features and the agent state features, and obtaining simulation behavior features of the agent based on the target state features, the target state features including state features of multiple channels and state features of multiple spaces.

[0126] In addition, the logic instruction in the memory 1030 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application or parts of the present application that essentially contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0127] On the other hand, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, so that the computer can execute the agent-based behavior simulation method provided by the above-mentioned method, which includes: obtaining three-dimensional perception data of an environment around an agent and agent state data; extracting environment state features from the three-dimensional perception data and agent state features from the agent state data; obtaining target state features based on the environment state features and the agent state features, and obtaining simulation behavior features of the agent based on the target state features, the target state features including state features of multiple channels and state features of multiple spaces.

[0128] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the agent-based behavior simulation method provided by any of the above methods, the method comprising obtaining three-dimensional perception data of an environment surrounding an agent and agent state data; extracting environment state features from the three-dimensional perception data and agent state features from the agent state data; obtaining target state features based on the environment state features and the agent state features, and obtaining simulation behavior features of the agent based on the target state features, the target state features comprising state features of multiple channels and state features of multiple spaces.

[0129] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0130] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0131] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An agent-based behavioral simulation method, characterized by, The method comprises: acquiring three-dimensional perception data of an environment around an agent and agent state data; extracting environment state features from the three-dimensional perception data and agent state features from the agent state data; acquiring target state features based on the environment state features and the agent state features, and acquiring simulated behavior features of the agent based on the target state features, the target state features comprising state features of multiple channels and state features of multiple spaces; the acquiring of the target state features based on the environment state features and the agent state features, and the acquiring of the simulated behavior features of the agent based on the target state features, comprises: performing channel weight adjustment and space weight adjustment on the environment state features to obtain weight-adjusted environment state features, the environment state features comprising state features of multiple channels and state features of multiple spaces; acquiring target state features based on the agent state features and the weight-adjusted environment state features; acquiring simulated behavior features of the agent based on the target state features, the simulated behavior features comprising rotation behavior features and movement behavior features; the performing of the channel weight adjustment and the space weight adjustment on the environment state features to obtain the weight-adjusted environment state features, comprises: performing channel weight adjustment on the environment state features, and acquiring first weight features corresponding to state features of different channels in the environment state features after the channel weight adjustment; acquiring first state features after the channel weight adjustment based on the environment state features and the first weight features; performing space weight adjustment on the first state features, and acquiring second weight features corresponding to state features of different spaces in the first state features after the space weight adjustment; acquiring second state features after the space weight adjustment based on the first state features and the second weight features, and taking the second state features as the weight-adjusted environment state features.

2. The agent-based behavior simulation method of claim 1, wherein, the performing of the channel weight adjustment on the environment state features, and the acquiring of the first weight features corresponding to state features of different channels in the environment state features after the channel weight adjustment, comprises: performing aggregation on spatial state features on each channel in the environment state features based on an average pooling layer to obtain first average pooling features; performing aggregation on spatial state features on each channel in the environment state features based on a maximum pooling layer to obtain first maximum pooling features; performing channel weight adjustment on the first average pooling features and the first maximum pooling features; performing element addition on the first average pooling features and the first maximum pooling features after the channel weight adjustment to obtain first weight adjustment features; performing normalization processing on the first weight adjustment features to obtain the first weight features corresponding to state features of different channels in the environment state features after the channel weight adjustment.

3. The agent-based behavior simulation method of claim 1, wherein, the performing of the space weight adjustment on the first state features, and the acquiring of the second weight features corresponding to state features of different spaces in the first state features after the space weight adjustment, comprises: aggregate, based on an average pooling layer, channel state features on each space in the first state feature to obtain a second average pooling feature; aggregate, based on a maximum pooling layer, channel state features on each space in the first state feature to obtain a second maximum pooling feature; perform feature splicing and convolution processing on the second average pooling feature and the second maximum pooling feature based on channel directions to obtain a second weight adjustment feature; perform normalization processing on the second weight adjustment feature to obtain a second weight feature corresponding to state features of different spaces in the first state feature.

4. The agent-based behavior simulation method according to any one of claims 1 to 3, characterized in that, The three-dimensional perception data of the environment around the agent includes: obtaining environmental perception data of the environment around the agent based on a pre-constructed visual perception module, the visual perception module including a horizontal perception ray arranged in a horizontal line-of-sight direction of the agent and a vertical perception ray arranged in a vertical line-of-sight direction of the agent; obtaining the three-dimensional perception data based on the environmental perception data; the three-dimensional perception data includes horizontal dimension data, vertical dimension data, and type dimension data, the horizontal dimension data representing a position of the perception ray in the horizontal direction, the vertical dimension data representing a position of the perception ray in the vertical direction, and the type dimension data representing a type of the environmental perception data obtained by the perception ray.

5. An agent-based behavior simulation apparatus, characterized by comprising: It includes: a data acquisition module configured to acquire three-dimensional perception data of an environment around an agent and agent state data; a feature extraction module configured to extract environmental state features from the three-dimensional perception data and agent state features from the agent state data; a behavior simulation module configured to acquire target state features based on the environmental state features and the agent state features, and to acquire simulated behavior features of the agent based on the target state features, the target state features including state features of multiple channels and state features of multiple spaces; the target state features based on the environmental state features and the agent state features, and the simulated behavior features of the agent based on the target state features, include: performing channel weight adjustment and spatial weight adjustment on the environmental state features to obtain weight-adjusted environmental state features; the environmental state features include state features of multiple channels and state features of multiple spaces; acquiring target state features based on the agent state features and the weight-adjusted environmental state features; acquiring simulated behavior features of the agent based on the target state features, the simulated behavior features including rotation behavior features and movement behavior features; the channel weight adjustment and spatial weight adjustment on the environmental state features to obtain the weight-adjusted environmental state features include: performing channel weight adjustment on the environmental state features and acquiring first weight features corresponding to state features of different channels in the environmental state features after channel weight adjustment; obtaining a first state feature after channel weight adjustment based on the environmental state features and the first weight features; The first state feature is subjected to spatial weight adjustment, and a second weight feature corresponding to the state feature of different spaces in the first state feature after spatial weight adjustment is obtained; Based on the first state feature and the second weight feature, a second state feature after spatial weight adjustment is obtained, and the second state feature is taken as the environment state feature after weight adjustment.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the agent-based behavior simulation method according to any one of claims 1 to 4 when executing the program.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the agent-based behavior simulation method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the agent-based behavior simulation method according to any one of claims 1 to 4.

Images