Multi-agent interactive simulation method based on stereoscopic vision information

Abstract

The invention provides a multi-agent interactive simulation method based on stereoscopic vision information, and the method comprises the steps: configuring a virtual binocular camera for each agent, dividing a visual field into a plurality of sectors, carrying out the nearest neighbor search, generating a focus list, and then calculating the physical speed of the agent; meanwhile, a reinforcement learning network is adopted to train an agent navigation strategy, and a navigation decision speed is generated in combination with a multi-level reward mechanism; carrying out adaptive weighted fusion on the navigation decision speed and the physical speed, dynamically adjusting a weight ratio according to environmental conditions, and generating a final speed instruction; and finally, executing the instruction in a simulation environment, driving the intelligent body to move and outputting a simulation result containing a track and a performance index. According to the method, the hypothesis that the intelligent agent has perfect perception ability in traditional crowd simulation is broken through, the reality sense of simulation, the behavior rationality and the robustness in complex scenes are remarkably improved, and the method can be widely applied to the fields of evacuation simulation, game development, virtual reality and the like.

Description

Technical Field

[0001] This invention relates to the field of machine genetic algorithm technology, and in particular to a multi-agent interaction simulation method based on stereo vision information. Background Technology

[0002] Crowd simulation has garnered significant attention in computer graphics due to its wide range of applications, including evacuation simulation, game development, and visual effects. Typically, individuals within a crowd are modeled as autonomous agents whose movements are governed by numerous, often unobservable, factors. Given the complexity and near-infinite variability of these factors, accurately simulating the decision-making processes of individual agents remains a major challenge.

[0003] Most existing crowd simulation models assume that agents can perfectly perceive the states of neighboring agents, including precise location and velocity, to guide their interactions. However, this assumption fundamentally contradicts human perception. Unlike robots, humans cannot perceive the precise spatial or physiological states of others, but rely heavily on visual input, which is inherently noisy, incomplete, and uncertain. Although vision-based crowd models have been explored, most methods contradictorily assume idealized perception mechanisms, which actually undermines their pursuit of realism.

[0004] Recent advances in reinforcement learning (RL) for crowd simulation have enabled agents to learn interaction strategies through trial and error in simulated environments. However, these methods suffer from key limitations: they either follow traditional interaction schemes (assuming accurate neighborhood states) or, while eliminating explicit interaction modeling, still rely on precise position and velocity data during training. These methods neglect a crucial aspect of human cognition—adaptive learning in the face of imperfect and incomplete perceptual feedback. Consequently, current RL-based crowd simulation models exhibit limited effectiveness under real-world perceptual constraints.

[0005] Therefore, a fundamental gap remains: previous research has failed to integrate the imprecise perception driven by vision into the reinforcement learning framework for crowd simulation. Bridging this gap requires reconciling two distinct paradigms: the subjectivity, noise, and local observability of human visual perception, and the objectivity and fully observable state transitions typically assumed in RL environments. Addressing this challenge will enable agents to learn collision avoidance and navigation strategies consistent with biological principles, thereby improving the realism and practicality of crowd simulation. Summary of the Invention

[0006] In view of the above, the main objective of this invention is to propose a multi-agent interaction simulation method based on stereoscopic vision information to solve the aforementioned technical problems.

[0007] This invention proposes a multi-agent interaction simulation method based on stereo vision information, the method comprising the following steps: Step 1: Equip each agent with a pair of virtual cameras, process the images from the left and right cameras using a semi-global block matching algorithm, and calculate the distance between the agents using the triangulation formula to obtain the agent's observation space position. Then, calculate the final observation velocity based on the agent's observation space position. Step 2: Divide the agent's field of view into multiple sectors and perform nearest neighbor queries to generate a focus list; input the agent's observed spatial position and final observed velocity in the focus list into the RVO algorithm to update the agent's physical velocity; Step 3: Adopt the Actor-Critic architecture and the near-end policy optimization algorithm, and incorporate a gating mechanism into the network architecture to train a reinforcement learning network; use the reinforcement learning network to train the agent to learn navigation and obstacle avoidance behaviors, and combine a multi-level reward mechanism to generate a navigation decision velocity vector.

[0008] Step 4: The navigation decision velocity vector is weighted and fused with the physical velocity of the agent, and the weight ratio is adjusted according to the environmental conditions to obtain the final velocity command.

[0009] Step 5: Drive the agent to move according to the final velocity command in the simulation environment, update the position state and detect collisions and target achievement to generate simulation results.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention better aligns with human perception characteristics, enhancing simulation realism. Existing technologies generally assume that intelligent agents can accurately acquire the positions, speeds, and other states of nearby individuals, which contradicts the reality that humans rely on vision and that perception is subject to noise and incompleteness. This invention simulates human stereoscopic perception through binocular vision, uses a semi-global block matching (SGBM) algorithm to estimate the positions of nearby intelligent agents, combines visual cues with physical information to infer speed, and introduces a confidence weight mechanism to handle scenarios with missing visual data. This makes the agent's perception more consistent with human cognitive patterns, thereby making crowd movement trajectories and obstacle avoidance behaviors closer to real pedestrian dynamics. 2. This invention balances navigation efficiency and safety by integrating reinforcement learning and physical models. Traditional rule-based models (such as RVO) rely on manual parameter tuning, making them difficult to handle complex scenarios. Some reinforcement learning-based models either assume precise perception or rely on accurate data during training. This invention designs an adaptive hybrid controller that dynamically integrates the output of reinforcement learning policies and the output of the improved RVO algorithm: reinforcement learning takes the lead in open areas to improve navigation efficiency, while physical models take the lead when obstacles are dense or the target is close to enhance obstacle avoidance accuracy and arrival accuracy, thus achieving a balance between efficiency and safety. 3. This invention shapes advanced human-like behaviors through a multi-level reward mechanism, enriching simulation details. Existing technologies mostly focus on basic navigation goals (such as obstacle avoidance and reaching the target), making it difficult to simulate the complex behavioral characteristics of human groups (such as group coordination and trajectory smoothness). This invention constructs a multi-level mechanism that includes basic navigation rewards (target approach and collision avoidance) and advanced behavioral rewards (biomechanical efficiency, social compliance, and path smoothness). Through course learning, advanced rewards are gradually activated, allowing the intelligent agent to exhibit human-like behaviors such as energy-saving movement and following similar individuals to form lanes. This enables the simulated human group to not only complete navigation tasks but also possess more delicate and heterogeneous realistic behavioral performance.

[0011] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the steps of a multi-agent interaction simulation method based on stereoscopic vision information proposed in this invention. Figure 2 This is a general framework diagram of a multi-agent interaction simulation method based on stereo vision information proposed in this invention. Figure 3 Schematic diagrams of four environments set up for this invention. Detailed Implementation

[0013] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0014] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0015] Please see Figure 1 This embodiment provides a multi-agent interaction simulation method based on stereo vision information, the method including the following steps: Step 1: Equip each agent with a pair of virtual cameras, process the images from the left and right cameras using a semi-global block matching algorithm, and calculate the distance between the agents using the triangulation formula to obtain the agent's observation spatial position. Then, calculate the final observation velocity based on the agent's observation spatial position.

[0016] Please see Figure 2 In step 1, each agent is equipped with a pair of virtual cameras. The images from the left and right cameras are processed using a semi-global block matching algorithm, and the distance between the agents is calculated using the triangulation formula to obtain the spatial position of the agents. Based on the spatial position of the agents, the final observation velocity is calculated. Specifically, this includes the following sub-steps: Each intelligent agent is equipped with two virtual cameras to simulate human stereoscopic vision and capture images from the left and right cameras; The parallax angle is obtained by processing the images from the left and right cameras using a semi-global block matching algorithm. The following relationship exists in the correspondence process: ; in, Indicates the parallax angle. This indicates taking the arctangent value. Indicates the focal length of the virtual camera. Indicates pixel parallax. This indicates the lens distortion correction factor, used to correct radial lens distortion, and ; The disparity angle is calculated using triangulation formulas to obtain the distance between the agent and its neighboring agents. The following relationship exists in the correspondence process: ; in, Represents intelligent agents With neighboring intelligent agents The distance between them Indicates the interpupillary distance. This indicates taking the tangent value. This represents the threshold value, used to avoid instability of the tangent function at small angles; Based on the distance between the agent and its neighboring agents, the spatial position of the neighboring agents relative to the agent is calculated. The following relationship exists in the correspondence process: ; in, Representing neighboring agents Compared to intelligent agents Horizontal angular offset, Representing neighboring agents Compared to intelligent agents Horizontal pixel coordinates, Indicates the horizontal width of the image. Indicates the field of view. Representing neighboring agents Compared to intelligent agents The spatial location of the observation Represents intelligent agents Location, This indicates taking the cosine value. Represents intelligent agents The unit vector directly in front of it. This indicates taking the sine value. Indicates and A vertical lateral unit vector; Instantaneous velocity is calculated based on the time difference between the observation spatial positions of neighboring agents and the agent itself, and the following relationship exists in the corresponding process: ; in, Representing neighboring agents In time instantaneous speed, Indicates time, Representing neighboring agents In time The observation location Indicates the simulation time step; Step frequency is calculated based on the personalized physical parameters configured during agent initialization. These personalized physical parameters include height, weight, and age, and the following relationship exists in the process: ; in, Indicates step frequency, All represent coefficients. Indicates height, represents weight, Indicates age; It should be noted that the coefficient All were determined through regression fitting of real-world walking data.

[0017] The step size is calculated based on the personalized physical parameters configured during agent initialization, and the following relationship exists in the corresponding process: ; in, Indicates the step size. Indicates the height weighting factor. Indicates the step frequency weighting factor. This represents the scaling factor that allows for online training. The basic speed of the agent is calculated based on step frequency and step length, and the following relationship exists in the process: ; in, This represents the basic speed of the intelligent agent; By fusing the instantaneous velocity and the agent's base velocity using weighted interpolation, the final observed velocity is obtained, and the following relationship exists in the corresponding process: ; in, Represents the confidence weights that change over time. This represents the maximum confidence level of a visual observation. Represents the natural constant. This represents the decay rate that controls the confidence level from decreasing over time. Indicates the timestamp of the last successful visual observation. This represents the cutoff threshold at which visual data is ignored. Indicates the final observed velocity. Representing neighboring agents The final observed direction of motion; Step 2: Divide the agent's field of view into multiple sectors and perform nearest neighbor queries to generate a focus list; input the agent's observed spatial position and final observed velocity in the focus list into the RVO algorithm to update the agent's physical velocity.

[0018] In step 2, the agent's field of view is divided into multiple sectors, and a nearest neighbor query is performed to generate a focus list. The observed spatial position and final observed velocity of the agent in the focus list are input into the RVO algorithm to update the agent's physical velocity. This includes the following sub-steps: Each agent is configured to look forward, and the agent's field of view coverage angle is... The field of view coverage angle is evenly divided into There are 1 sector, and each sector covers an angular width of 1. ; in, This indicates the field of view coverage angle of the intelligent agent. Indicates the number of sectors. Indicates the angular width covered by each sector; For each sector intelligent agent Identify visible neighbors located within the sector angle range; For intelligent agents The direction vector directly in front and the direction pointing to the neighboring agent normalized vector Calculate the angle between them to obtain the vector. with vector The included angle between them corresponds to the following relationship in the process: ; in, Representing vectors with vector The angle between them This indicates taking the arccosine value. Indicates pointing to neighboring agents The normalized vector, Represents intelligent agents The direction vector directly in front; Assume sectors are numbered starting from the leftmost boundary of the field of view and Then the neighboring intelligent agent Corresponding sector index The following relation exists: ; in, Indicates the index of the sector; After assigning neighboring agents to their respective regions, the distance is calculated. The minimized agent generates a focus list, and the following relationship exists in the corresponding process: ; in, Indicates that it is located in the sector Distance between the middle Minimal intelligent agent, This indicates that the parameter should be chosen to minimize the function. Indicates that it is located in the sector A collection of intelligent agents Indicates the focus list. Indicates the distance in the first sector Minimal intelligent agent, Indicates the first In each sector, make the distance Minimum intelligent agent; The observed spatial position and final observed velocity of the agents in the focus list are input into the RVO algorithm to update the physical velocity of the agents.

[0019] Step 3: Adopt the Actor-Critic architecture and the near-end policy optimization algorithm, and incorporate a gating mechanism into the network architecture to train a reinforcement learning network; use the reinforcement learning network to train the agent to learn navigation and obstacle avoidance behaviors, and combine a multi-level reward mechanism to generate a navigation decision velocity vector.

[0020] In step 3, the agent is trained using a reinforcement learning network to learn navigation and obstacle avoidance behaviors, and a navigation decision velocity vector is generated by combining a multi-level reward mechanism. This includes the following sub-steps: Reinforcement learning networks use gating mechanisms to process the spatial location of an agent's observations and time... The observed values ​​are defined as four-dimensional vectors, and the following relationship exists in the correspondence process: ; in, Indicates time The observed values, Represents the coordinates of the target point Axial components, Representing the agent's coordinates Axial components, Represents the coordinates of the target point Axial components, Representing the agent's coordinates Axial components; It should be noted that the coordinate system adopts a two-dimensional metric space representation without boundary constraints to adapt to the scale and layout of different scenarios; relative coordinates provide directional guidance for target-oriented behavior, while absolute coordinates support global situational awareness, which is especially suitable for dynamic or large-scale environments.

[0021] This simplified representation deliberately omits high-dimensional perceptual inputs (such as lidar, proximity sensors, or multi-agent state data) to reduce inference latency. Combined with the gating mechanism in the neural network architecture, this minimization of the observation space enables agents to dynamically emphasize task-related features (such as target direction), thereby achieving efficient and robust navigation in diverse scenarios such as open areas, narrow corridors, and congested bottlenecks.

[0022] The action space of an agent is defined as a continuous two-dimensional vector representing the velocity direction. This continuous two-dimensional vector representing the velocity direction includes a horizontal velocity component and a vertical velocity component. The following relationship exists in the corresponding process: ; in, Indicates time The actions taken by the intelligent agent Represents the horizontal velocity component, and ; Represents the vertical velocity component, and ; The horizontal and vertical velocity components are normalized and scaled to a reference velocity to obtain the actual velocity. The following relationship exists in this process: ; in, Indicates actual speed; The multi-tiered rewards include target rewards, progress rewards, time penalties, collision penalties, as well as biomechanical efficiency rewards, social compliance rewards, and path fluency rewards, and the corresponding relationships are as follows: ; in, Indicates the total reward. This represents the weight used to adjust the impact of basic navigation. Indicates the target reward. Indicates progress reward. Indicates a time penalty. Indicates a collision penalty. This represents the weight used to modulate the effect of biomechanical efficiency. This indicates a reward for biomechanical efficiency. This indicates the weight used to moderate the influence of social conformity. Indicates social conformity rewards, This represents the weights used to adjust the impact of path smoothness. Indicates a reward for smooth path movement; It should be noted that there is a time penalty. This is deducted from each time step to improve time efficiency.

[0023] The progress reward has the following relationship: ; in, This indicates the distance from the target in the previous step. Indicates the current distance; The collision penalty follows the following relationship: ; in, Indicates the minimum distance to any obstacle, wall, or other intelligent agent; The biomechanical efficiency reward has the following relationship: ; in, This indicates taking the minimum value. Indicates the agent's current speed. This represents the biomechanically optimal walking speed calculated based on the physiological characteristics of the agent. Among them, the social conformity reward has the following relationship: ; in, This indicates taking the maximum value. This indicates the alignment between the agent's velocity direction and the average heading of nearby agents with similar demographic characteristics; The path smoothness reward has the following relationship: ; in, Represents an exponential function. Indicates path curvature; It should be noted that biomechanical efficiency rewards incentivize agents to achieve energy-efficient movement by penalizing behaviors that deviate from the optimal walking speed; social conformity rewards promote real-world group behavior: agents tend to follow similar individuals, thus forming spontaneous lanes and flow patterns observed in real-world crowds; path fluency rewards encourage the generation of natural movement trajectories by favoring moderate path curvature (neither too straight nor too curved).

[0024] A near-end strategy optimization algorithm with integrated gating mechanism is used for training. Feature weights are assigned through element-wise multiplication, and the following relationship exists in the process: ; in, This indicates the features extracted after two layers of processing via a gating mechanism. Represents the feature layer. This represents the Sigmoid function. This indicates the characteristics after processing by the gating mechanism.

[0025] It should be noted that during the training phase, the agent acquires navigation and obstacle avoidance capabilities by interacting with a dynamically changing environment, which features constantly evolving obstacle layouts, path complexity, and agent density. An adaptive reward weighting mechanism shapes the agent's behavior to achieve scene-specific adaptation. To this end, we introduce a multi-level reward mechanism that combines low-level navigation objectives (such as target proximity and collision avoidance) with high-level behavioral objectives (such as trajectory smoothness and group coordination). At the same time, a course learning strategy is adopted to gradually increase the complexity of the environment, enabling the agent to seamlessly transition from simple to advanced scenarios.

[0026] Step 4: The navigation decision velocity vector is weighted and fused with the physical velocity of the agent, and the weight ratio is adjusted according to the environmental conditions to obtain the final velocity command.

[0027] In step 4, the navigation decision velocity vector is weighted and fused with the agent's physical velocity, and the weight ratio is adjusted according to environmental conditions to generate the final velocity command. This process includes the following sub-steps: The adaptive weights are dynamically determined using a preset adaptive function, and the following relationship exists in the process: ; in, Indicates adaptive weights, Represents an adaptive function. This represents the distance from the agent to the nearest obstacle. Represents the local agent density. This represents the distance from the agent to the navigation target. This represents unique characteristics of the intelligent agent (such as height, weight, and age). Indicates the benchmark weight. This represents the distance threshold from the agent to the target. The weights representing obstacle avoidance distance, Indicates the range of dynamic adjustment. Indicates the normalized obstacle distance. Represents the normalized neighbor density; Based on adaptive weights, the navigation decision velocity vector and the agent's physical velocity are weighted and fused to obtain the final velocity command. The following relationship exists in the corresponding process: ; in, Indicates the final speed command. This represents the velocity vector for navigation decisions.

[0028] It should be noted that adjusting the weight ratio according to environmental conditions specifically includes: when approaching the target, increasing the reliance on the agent's physical speed to improve arrival accuracy; in areas with dense obstacles, prioritizing the agent's physical speed to enhance obstacle avoidance capabilities; and in open or uncrowded areas, allowing the navigation decision speed vector to dominate navigation, thereby achieving efficient strategic movement.

[0029] Step 5: Drive the agent to move according to the final velocity command in the simulation environment, update the position state and detect collisions and target achievement to generate simulation results.

[0030] Please see Figure 3 To comprehensively evaluate the model's realism, generalization ability, and efficiency, we designed and implemented four different environments for simulation experiments, each with unique challenges: (1) Bottleneck scenario: Simulate an intelligent agent traversing a narrow channel, including two configurations: bidirectional and unidirectional; (2) Maze scenario: Set up staggered passages and walls, requiring the agent to plan paths and avoid obstacles in a narrow space; (3) Roundabout: The agents are arranged along the circular boundary and need to cross the central intersection to reach the endpoints at both ends of the diagonal; (4) L-shaped corridor: This scenario requires the agent to make a 90-degree turn, testing its ability to adapt to directions and handle corner congestion.

[0031] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0032] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0033] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0034] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A multi-agent interaction simulation method based on stereoscopic vision information, characterized in that, The method includes the following steps: Step 1: Equip each agent with a pair of virtual cameras, process the images from the left and right cameras using a semi-global block matching algorithm, and calculate the distance between the agents using the triangulation formula to obtain the agent's observation space position. Then, calculate the final observation velocity based on the agent's observation space position. Step 2: Divide the agent's field of view into multiple sectors and perform nearest neighbor queries to generate a focus list; input the agent's observed spatial position and final observed velocity in the focus list into the RVO algorithm to update the agent's physical velocity; Step 3: Adopt the Actor-Critic architecture and the near-end policy optimization algorithm, and incorporate a gating mechanism into the network architecture to train the reinforcement learning network; The agent is trained to learn navigation and obstacle avoidance behaviors using a reinforcement learning network, and a navigation decision velocity vector is generated by combining a multi-level reward mechanism. Step 4: The navigation decision velocity vector is weighted and fused with the physical velocity of the agent, and the weight ratio is adjusted according to the environmental conditions to obtain the final velocity command; Step 5: Drive the agent to move according to the final velocity command in the simulation environment, update the position state and detect collisions and target achievement to generate simulation results.

2. The multi-agent interaction simulation method based on stereoscopic vision information according to claim 1, characterized in that, In step 1, each agent is equipped with a pair of virtual cameras. The images from the left and right cameras are processed using a semi-global block matching algorithm, and the distance between the agents is calculated using triangulation formulas to obtain the spatial position of the agents. Based on the spatial position of the agents, the final observation velocity is calculated. Specifically, this includes the following sub-steps: Each intelligent agent is equipped with two virtual cameras to simulate human stereoscopic vision and capture images from the left and right cameras; The parallax angle is obtained by processing the images from the left and right cameras using a semi-global block matching algorithm. The parallax angle is calculated using triangulation formulas to obtain the distance between the agent and its neighboring agents; Based on the distance between the agent and its neighboring agents, the observation spatial position of the neighboring agents relative to the agent is calculated; Instantaneous velocity is calculated based on the time difference between the observation spatial position of neighboring agents and that of the agent. Step frequency is calculated based on the personalized physical parameters configured during agent initialization, including height, weight, and age. The step size is calculated based on the personalized physical parameters configured during agent initialization; The agent's base speed is calculated based on step frequency and step length. The final observed velocity is obtained by fusing the instantaneous velocity and the agent's base velocity using weighted interpolation.

3. The multi-agent interaction simulation method based on stereoscopic vision information according to claim 2, characterized in that, In the step of processing the images from the left and right cameras using a semi-global block matching algorithm to obtain the disparity angle, the following relationship exists: ； in, Indicates the parallax angle. This indicates taking the arctangent value. Indicates the focal length of the virtual camera. Indicates pixel parallax. Indicates the lens distortion correction factor; In the step of calculating the disparity angle using triangulation formulas to obtain the distance between the agent and its neighboring agents, the following relationship exists: ； in, Represents intelligent agents With neighboring intelligent agents The distance between them Indicates the interpupillary distance. This indicates taking the tangent value. Indicates the threshold; In the step of calculating the observed spatial position of a neighboring agent relative to the agent based on the distance between the agent and neighboring agents, the following relationship exists: ； in, Representing neighboring agents Compared to intelligent agents Horizontal angular offset, Representing neighboring agents Compared to intelligent agents Horizontal pixel coordinates, Indicates the horizontal width of the image. Indicates the field of view. Representing neighboring agents Compared to intelligent agents The spatial location of the observation Represents intelligent agents Location, This indicates taking the cosine value. Represents intelligent agents The unit vector directly in front of it. This indicates taking the sine value. Indicates and Vertical lateral unit vector.

4. The multi-agent interaction simulation method based on stereoscopic vision information according to claim 3, characterized in that, In the step of calculating instantaneous velocity based on the time difference between the observation spatial positions of neighboring agents and the agent, the following relationship exists: ； in, Representing neighboring agents In time instantaneous speed, Indicates time, Representing neighboring agents In time The observation location Indicates the simulation time step; In the step frequency calculation based on the personalized physical parameters configured during agent initialization, including height, weight, and age, the following relationship exists: ； in, Indicates step frequency, All represent coefficients. Indicates height, represents weight, Indicates age; In the step of calculating the step size based on the personalized physical parameters configured during agent initialization, the following relationship exists: ； in, Indicates the step size. Indicates the height weighting factor. Indicates the step frequency weighting factor. This represents the scaling factor that allows for online training. In the step of calculating the agent's base velocity based on step frequency and step length, the following relationship exists: ； in, This represents the basic speed of the intelligent agent; In the step of fusing instantaneous velocity and the agent's baseline velocity using weighted interpolation to obtain the final observed velocity, the following relationship exists: ； in, Represents the confidence weights that change over time. This represents the maximum confidence level of a visual observation. Represents the natural constant. This represents the decay rate that controls the confidence level from decreasing over time. Indicates the timestamp of the last successful visual observation. This represents the cutoff threshold at which visual data is ignored. Indicates the final observed velocity. Representing neighboring agents The final observed direction of motion.

5. The multi-agent interaction simulation method based on stereoscopic vision information according to claim 4, characterized in that, In step 2, the agent's field of view is divided into multiple sectors, and a nearest neighbor query is performed to generate a focus list. The observed spatial position and final observed velocity of the agent in the focus list are input into the RVO algorithm to update the agent's physical velocity. This specifically includes the following sub-steps: Each agent is configured to look forward, and the agent's field of view coverage angle is... The field of view coverage angle is evenly divided into There are 1 sector, and each sector covers an angular width of 1. ; in, This indicates the field of view coverage angle of the intelligent agent. Indicates the number of sectors. Indicates the angular width covered by each sector; For each sector intelligent agent Identify visible neighbors located within the sector angle range; in, Indicates the index of the sector; For intelligent agents The direction vector directly in front and the direction pointing to the neighboring agent normalized vector Calculate the angle between them to obtain the vector. with vector The included angle between them corresponds to the following relationship in the process: ； in, Indicates pointing to neighboring agents The normalized vector, Represents intelligent agents The direction vector directly in front, Representing vectors with vector The angle between them This indicates taking the inverse cosine value; Assume sectors are numbered starting from the leftmost boundary of the field of view and Then the neighboring intelligent agent Corresponding sector index The following relation exists: ； After assigning neighboring agents to their respective regions, the distance is calculated. The minimized agent generates a focus list, and the following relationship exists in the corresponding process: ； in, Indicates that it is located in the sector Distance between the middle Minimal intelligent agent, This indicates that the parameter should be chosen to minimize the function. Indicates that it is located in the sector A collection of intelligent agents Indicates the focus list. Indicates the distance in the first sector Minimal intelligent agent, Indicates the first In each sector, make the distance Minimum intelligent agent.

6. The multi-agent interaction simulation method based on stereoscopic vision information according to claim 5, characterized in that, In step 3, the agent is trained using a reinforcement learning network to learn navigation and obstacle avoidance behaviors, and a navigation decision velocity vector is generated by combining a multi-level reward mechanism. This specifically includes the following sub-steps: Reinforcement learning networks use gating mechanisms to process the spatial location of an agent's observations and time... The observed values ​​are defined as four-dimensional vectors, and the following relationship exists in the correspondence process: ； in, Indicates time The observed values, Represents the coordinates of the target point Axial components, Representing the agent's coordinates Axial components, Represents the coordinates of the target point Axial components, Representing the agent's coordinates Axial components; The action space of an agent is defined as a continuous two-dimensional vector representing the velocity direction. This continuous two-dimensional vector representing the velocity direction includes a horizontal velocity component and a vertical velocity component. The following relationship exists in the corresponding process: ； in, Indicates time The actions taken by the intelligent agent Represents the horizontal velocity component. Represents the vertical velocity component; The horizontal and vertical velocity components are normalized and scaled to a reference velocity to obtain the actual velocity. The following relationship exists in this process: ； in, Indicates actual speed; A near-end strategy optimization algorithm with integrated gating mechanism is used for training. Feature weights are assigned through element-wise multiplication, and the following relationship exists in the process: ； in, This indicates the features extracted after two layers of processing via a gating mechanism. Represents the feature layer. This represents the Sigmoid function. This indicates the characteristics after processing by the gating mechanism.

7. The multi-agent interaction simulation method based on stereoscopic vision information according to claim 6, characterized in that, The multi-tiered rewards include target rewards, progress rewards, time penalties, collision penalties, as well as biomechanical efficiency rewards, social compliance rewards, and path fluency rewards, and the corresponding relationships are as follows: ； in, Indicates the total reward. This represents the weight used to adjust the impact of basic navigation. Indicates the target reward. Indicates progress reward. Indicates a time penalty. Indicates a collision penalty. This represents the weight used to modulate the effect of biomechanical efficiency. This indicates a reward for biomechanical efficiency. This indicates the weight used to moderate the influence of social conformity. Indicates social conformity rewards, This represents the weights used to adjust the impact of path smoothness. Indicates a reward for smooth path movement; The progress reward has the following relationship: ； in, This indicates the distance from the target in the previous step. Indicates the current distance; The collision penalty follows the following relationship: ； in, Indicates the minimum distance to any obstacle, wall, or other intelligent agent; The biomechanical efficiency reward has the following relationship: ； in, This indicates taking the minimum value. Indicates the agent's current speed. This represents the biomechanically optimal walking speed calculated based on the physiological characteristics of the agent. Among them, the social conformity reward has the following relationship: ； in, This indicates taking the maximum value. This indicates the alignment between the agent's velocity direction and the average heading of nearby agents with similar demographic characteristics; The path smoothness reward has the following relationship: ； in, Represents an exponential function. Indicates path curvature.

8. The multi-agent interaction simulation method based on stereo vision information according to claim 7, characterized in that, In step 4, the navigation decision velocity vector is weighted and fused with the agent's physical velocity, and the weight ratio is adjusted according to environmental conditions to obtain the final velocity command. This process includes the following sub-steps: The adaptive weights are dynamically determined using a preset adaptive function. The navigation decision velocity vector and the agent's physical velocity are weighted and fused based on adaptive weights to obtain the final velocity command.

9. The multi-agent interaction simulation method based on stereoscopic vision information according to claim 8, characterized in that, In the step of dynamically determining adaptive weights through a preset adaptive function, the following relationship exists: ； in, Indicates adaptive weights, Represents an adaptive function. This represents the distance from the agent to the nearest obstacle. Represents the local agent density. This represents the distance from the agent to the navigation target. This represents the unique characteristics of an intelligent agent. Indicates the benchmark weight. This represents the distance threshold from the agent to the target. The weights representing obstacle avoidance distance, Indicates the range of dynamic adjustment. Indicates the normalized obstacle distance. Represents the normalized neighbor density; In the step of weighted fusion of the navigation decision velocity vector and the agent's physical velocity based on adaptive weights to obtain the final velocity command, the following relationship exists: ； in, Indicates the final speed command. This represents the velocity vector for navigation decisions.

Images