Vehicle state management method, device, equipment and product based on simulation environment

Abstract

The application discloses a vehicle state management method and device based on a simulation environment, equipment and products, and belongs to the technical field of simulation. The method comprises the following steps: acquiring a traffic simulation environment, wherein the traffic simulation environment comprises a target lane; a plurality of vehicle queues connected in front and back of the target lane are maintained, wherein the vehicles in each vehicle queue correspond to the same type of motion state, and the motion states corresponding to two adjacent vehicle queues in front and back are different, and the motion state comprises a forward state and a stop state; based on the passing state corresponding to the target lane, target vehicles in the plurality of vehicle queues that meet the updating condition are determined; and an updating operation is performed on the target vehicles, and the updating operation is used for updating at least one of the position, speed and motion state of the target vehicles, wherein the stationary vehicles in the stop queue except the passable vehicles are skipped, the stop queue refers to the vehicle queue in the stop state, invalid calculation can be eliminated, and the vehicle state updating efficiency of the simulation frame is improved.

Description

Technical Field

[0001] This application relates to the field of simulation technology, and in particular to a vehicle state management method, device, equipment and product based on a simulation environment. Background Technology

[0002] In the application of reinforcement learning in the field of intelligent control of urban traffic lights, it is necessary to rely on the traffic simulation environment to complete the training and strategy iteration of the agent. The core of the traffic simulation environment is to update the vehicle state frequently and efficiently to ensure the realism of the simulation and the efficiency of sample generation, so as to provide the reinforcement learning agent with accurate environmental observation information.

[0003] In related technologies, traffic simulation environments update the vehicle status of vehicles in the target lane by performing a full scan of each vehicle. In each simulation frame, the position, speed, and other status of all vehicles in the lane must be checked and updated one by one.

[0004] However, the aforementioned traversal calculation method leads to a significant increase in the computational cost of vehicle state updates as the number of simulated vehicles increases, resulting in a sharp decline in simulation efficiency and making it difficult to meet the requirements of reinforcement learning for generating high-frequency samples in traffic simulation environments. Summary of the Invention

[0005] This application provides a vehicle state management method, apparatus, device, and product based on a simulation environment. The technical solution includes the following aspects.

[0006] On the one hand, a vehicle state management method based on a simulation environment is provided, the method comprising: Acquire a traffic simulation environment, which includes the target lane; For the target lane maintenance, there are multiple vehicle queues connected before and after, wherein the vehicles in each vehicle queue correspond to the same type of movement state, and the movement states of two adjacent vehicle queues are different, including forward state and stopped state. Based on the traffic status corresponding to the target lane, the target vehicles that meet the update conditions in the multiple vehicle queues are determined; An update operation is performed on the target vehicle, the update operation being used to update at least one of the target vehicle's position, speed, and motion state, wherein stationary vehicles in the stop queue, excluding passable vehicles, are skipped, the stop queue referring to the queue of vehicles in the stopped state.

[0007] On the other hand, a vehicle state management device based on a simulation environment is provided, the device comprising: An acquisition module is used to acquire a traffic simulation environment, the traffic simulation environment including the target lane; The management module is used to maintain multiple vehicle queues connected to each other for the target lane. The vehicles in each vehicle queue correspond to the same type of movement state, and the movement states of two adjacent vehicle queues are different. The movement states include forward state and stopped state. The determination module is used to determine the target vehicle that meets the update conditions in the multiple vehicle queues based on the traffic status corresponding to the target lane. The management module is also used to perform an update operation on the target vehicle. The update operation is used to update at least one of the target vehicle's position, speed, and motion state, wherein stationary vehicles other than passable vehicles in the stop queue are skipped. The stop queue refers to the queue of vehicles in the stopped state.

[0008] In some embodiments, the acquisition module is further configured to acquire the phase type of the target traffic light corresponding to the target lane; The determining module is further configured to determine the traffic status corresponding to the target lane based on the phase type; The determining module is further configured to, based on the traffic status, and in conjunction with the order of the vehicle queue and the current movement status, filter the target vehicles that meet the update conditions.

[0009] In some embodiments, the acquisition module is further configured to acquire the movement state of the first queue in the target lane when the target lane is in a passable state, wherein the first queue is the first queue of vehicles arranged in the target lane according to the arrangement order. The determining module is further configured to determine the target vehicle that meets the update conditions based on the current motion state of the first queue.

[0010] In some embodiments, the determining module is further configured to: If the first queue belongs to the stopped queue, a moving queue is created as a new first queue. The moving queue refers to the queue of vehicles in the forward state. The new first queue includes the first vehicle in the original first queue. Vehicle queues other than the new first queue will be designated as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the new first queue and the subsequent target vehicles are identified as the target vehicles.

[0011] In some embodiments, the determining module is further configured to: If the first queue is a moving queue, then the vehicle queues other than the first queue are designated as subsequent queues, where the moving queue refers to the vehicle queues in the forward state. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first queue and the subsequent target vehicles are identified as the target vehicles.

[0012] In some embodiments, the acquisition module is further configured to acquire the current movement state of the first queue in the target lane when the target lane is impassable, wherein the first queue is the first queue of vehicles arranged in the target lane according to the arrangement order. The determining module is further configured to determine the target vehicle that meets the update conditions based on at least one of the current motion state of the first queue and the passable vehicle situation in the first queue.

[0013] In some embodiments, the determining module is further configured to: If the first queue belongs to the stop queue, skip the first queue; The remaining vehicle queues, starting from the second vehicle queue after the first queue, are designated as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

[0014] In some embodiments, the acquisition module is further configured to acquire the remaining distance between the head vehicle in the first queue and the stop line when the first queue is a moving queue; The determining module is also used to determine the passable vehicle situation in the first queue based on the remaining distance; The determining module is further configured to determine the target vehicle that meets the updated conditions based on the passable vehicle situation.

[0015] In some embodiments, the determining module is further configured to: If there are passable vehicles in the first queue, the remaining vehicle queue starting from the second vehicle queue after the first queue shall be used as the subsequent queue. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

[0016] In some embodiments, the determining module is further configured to: If there are no passable vehicles in the first queue, a stop queue is created as a new first queue, and the new first queue includes the first vehicle in the original first queue. The remaining vehicle queues, starting from the second vehicle queue after the new first queue, are used as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

[0017] In some embodiments, the determining module is further configured to: The stopping queue and the moving queue are determined from the subsequent queues based on the current motion state; Determine the passable vehicles in the stop queue of the subsequent queue; The passable vehicles in the stopped queue and all vehicles in the moving queue are identified as the subsequent target vehicles.

[0018] In some embodiments, the acquisition module is further configured to acquire the vehicle distance between the head vehicle in the stop queue and the tail vehicle in the previous vehicle queue. The determining module is further configured to, when the vehicle distance meets the passage conditions, determine the head vehicle as the passable vehicle, and move the head vehicle to the tail of the previous vehicle queue, until the vehicle distance corresponding to the head vehicle in the stopped queue no longer meets the passage conditions.

[0019] In some embodiments, the management module is further configured to divide each lane in the traffic simulation environment into multiple spatial grids, each spatial grid being a fixed area with a preset parking space capacity, each spatial grid corresponding to a vehicle information index, the vehicle information index being used to indicate the vehicles accommodated in the spatial grid; The management module is also used to update the vehicle information index corresponding to the spatial grid based on the update operation; The determining module is further configured to determine adjacent spatial grids that meet the adjacent detection conditions with respect to the first vehicle when there is a first vehicle in the traffic simulation environment that has a lane change requirement; The determining module is further configured to determine the reversible lane status of the first vehicle based on the vehicle information index corresponding to the adjacent spatial grid.

[0020] In some embodiments, the apparatus further includes a feedback module and a generation module; The feedback module is used to feed back the updated vehicle status information to the status perception unit of the traffic simulation environment based on the update operation. The generation module is used to generate observation information of the traffic simulation environment by the state perception unit. The observation information is used to provide perceptible traffic state data to the reinforcement learning agent, which is used to learn traffic control decisions based on the traffic simulation environment.

[0021] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the vehicle state management method based on simulation environment as described in any of the embodiments of this application above.

[0022] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored in the storage medium, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the vehicle state management method based on a simulation environment as described in any of the embodiments of this application above.

[0023] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the vehicle state management method based on a simulation environment as described in any of the above embodiments.

[0024] The beneficial effects of the technical solutions provided in this application include at least the following: By maintaining alternating vehicle queues for the target lane according to its motion state, determining the target updated vehicle based on the lane traffic status, and skipping non-passable stationary vehicles in the stop queue, invalid calculations in vehicle state updates are eliminated, significantly reducing the computational cost of vehicle state updates in traffic simulation, improving the efficiency of vehicle state updates in simulation frames, and adapting to the high-frequency vehicle state update requirements of traffic simulation. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of a computer system provided in an exemplary embodiment of this application; Figure 2 This is a schematic diagram of an environment simulator architecture provided in an exemplary embodiment of this application; Figure 3 This is a flowchart of a vehicle state management method based on a simulation environment provided in an exemplary embodiment of this application; Figure 4 This is a flowchart of a method for determining a target vehicle provided in an exemplary embodiment of this application; Figure 5 This is a flowchart of a stationary vehicle removal method provided in an exemplary embodiment of this application; Figure 6 This is a flowchart of a nearby vehicle detection method provided in an exemplary embodiment of this application; Figure 7 This is a schematic diagram of nearby vehicle detection provided in an exemplary embodiment of this application; Figure 8 This is a flowchart of a reinforcement learning method based on a simulation environment provided in an exemplary embodiment of this application; Figure 9 This is a schematic diagram illustrating a traffic simulation environment application provided in an exemplary embodiment of this application; Figure 10 This is a comparison diagram of the effects provided by an exemplary embodiment of this application; Figure 11 This is a structural block diagram of a vehicle state management device based on a simulation environment provided in an exemplary embodiment of this application; Figure 12 This is a structural block diagram of a vehicle state management device module based on a simulation environment provided in an exemplary embodiment of this application; Figure 13 This is a structural block diagram of a terminal provided in an exemplary embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0028] It should be understood that although the terms first, second, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, a first parameter may also be referred to as a second parameter without departing from the scope of this disclosure, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0029] First, the computer system described in this application will be introduced. Please refer to... Figure 1 The illustration shows a schematic diagram of a computer system provided in an exemplary embodiment of this application, the computer system including: computer device 10.

[0030] The computer device 10 is equipped with an environment simulator and a reinforcement learning agent. The environment simulator is used to provide a traffic simulation environment, which can be a simulation environment of the traffic environment in any area. The reinforcement learning agent is used to learn traffic management strategies based on the traffic simulation environment.

[0031] Based on the traffic simulation environment provided by the environment simulator, the vehicle state management method provided in the embodiments of this application can be executed to improve the efficiency of vehicle state management and adapt to the requirements of reinforcement learning.

[0032] Schematic, computer device 10 acquires a traffic simulation environment, which includes a target lane; maintains multiple vehicle queues connected to each other for the target lane, wherein vehicles in each queue correspond to the same type of motion state, and adjacent vehicle queues correspond to different motion states, including forward and stationary states; based on the traffic state corresponding to the target lane, it determines the target vehicle that meets the update conditions among the multiple vehicle queues; and performs an update operation on the target vehicle, which updates at least one of the target vehicle's position, speed, and motion state, wherein stationary vehicles other than those that can pass through the stop queue are skipped, and the stop queue refers to a queue of vehicles in a stationary state.

[0033] The traffic simulation environment is a virtual traffic scenario built using computer technology. This environment performs full-element digital modeling and high-frequency dynamic simulation of the road network topology, traffic facility layout, vehicle movement behavior, and traffic control rules of real urban traffic systems. It accurately reproduces the starting, stopping, driving, queuing, and evolution patterns of real traffic flow in virtual space. This environment can systematically measure, analyze, and quantitatively evaluate multi-dimensional traffic state data. It can measure individual driving data such as the location coordinates, real-time speed, starting and stopping status, and distance to the vehicle in front or the stop line of a single vehicle. It can also measure group operation indicators such as queue length, average speed, average waiting time, and traffic flow for a single lane or even the entire road network. Simultaneously, it can simulate complex conditions such as peak hours, rain and snow, traffic accidents, and road construction, testing the traffic efficiency, congestion mitigation effects, and robustness of different traffic control schemes, vehicle behavior strategies, and algorithm models under various scenarios. It also generates standardized observation data, providing stable and efficient experimental support for the decision-making training, strategy iteration, and performance evaluation of reinforcement learning agents.

[0034] The aforementioned computer equipment can be a terminal, a server, or any other device; this application does not limit this.

[0035] The aforementioned terminal is optional and can be a desktop computer, laptop computer, mobile phone, tablet computer, e-book reader, Moving Picture Experts Group Audio Layer III (MP3) player, Moving Picture Experts Group Audio Layer IV (MP4) player, smart TV, smart vehicle, and other types of terminal devices. This application embodiment does not limit the specific terminal device to these types.

[0036] It is worth noting that the aforementioned servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud security, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0037] Cloud technology refers to a managed technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computing, storage, processing, and sharing.

[0038] In some embodiments, the server described above can also be implemented as a node in a blockchain system.

[0039] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant regions. For example, the operational data and account information involved in this application were obtained with full authorization.

[0040] To further clarify, this application may display a prompt interface, pop-up window, or output voice prompts before and during the collection of user-related data (e.g., account information, historical operation data, and real-time operation data involved in this application). These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their relevant data is being collected. This ensures that the application only begins the steps for collecting user-related data after receiving confirmation from the user regarding the prompt interface or pop-up window; otherwise (i.e., without receiving confirmation from the user), the steps for collecting user-related data end, meaning no user-related data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of relevant user data must comply with the relevant laws, regulations, and standards of the relevant regions.

[0041] In related technologies, traffic simulation environments update the vehicle state within a target lane by performing a full-vehicle scan. Each simulation frame requires checking and updating the position, speed, and other states of every vehicle in the lane. However, this traversal computational method causes the computational cost of vehicle state updates to increase significantly with the number of simulated vehicles, leading to a sharp decline in simulation efficiency and making it difficult to meet the high-frequency sample generation requirements of reinforcement learning for traffic simulation environments.

[0042] Based on this, this application provides a vehicle state management method based on a simulation environment, which is executed by an environment simulator (also known as a traffic simulator). The environment simulator for online reinforcement learning is modularized, and the essential basic dynamic mechanisms are atomized. An environment that interacts with the intelligent agent is formed by combining basic dynamic mechanisms through flexibly adjustable strategies. The strategy combination is selected according to the algorithm requirements, and methods such as offline adjustment and online dynamic adjustment of the strategy combination of the basic dynamic mechanism of the environment are provided.

[0043] First, the environment simulator used in the embodiments of this application will be described. Please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of an environment simulator architecture provided in an exemplary embodiment of this application, such as... Figure 2 As shown, the environment simulator has a 4-layer architecture, from bottom to top: basic service layer 210, atomic mechanism layer 220, strategy composition layer 230, and interaction interface layer 240.

[0044] The basic service layer 210 is the infrastructure that supports the dynamic operation of the environment simulator, including communication units, data units, log units, etc.

[0045] 1. The communication unit supports communication between reinforcement learning environment instances and reinforcement learning agents, including functions such as current communication based on shared memory and cross-node communication based on the network.

[0046] 2. The data unit supports functions such as intermediate data caching and solidification for state space transitions of reinforcement learning environment instances, result data caching and solidification, and agent action trajectory caching and planning for visualization.

[0047] 3. The log unit supports the printing of various debugging information during the runtime of the environment instance.

[0048] The atomic mechanism layer 220 consists of various atomized mechanisms for the simulation environment. These mechanisms can be combined appropriately to realize different functions of the environment, the action space of the agent, and the behavior strategies of interactive objects.

[0049] 1. The vehicle movement unit models the mechanisms of vehicle speed, start-stop acceleration, and following distance under different road conditions.

[0050] 2. The movement unit realizes the topology function of the vehicle-driving area in the environment.

[0051] 3. The rigid body collision unit realizes the detection of variable conditions before the vehicle changes lanes.

[0052] 4. The observation mapping unit realizes the function of collecting and encoding environmental observation information visible at different intersections.

[0053] 5. The traffic light phase control unit at the intersection enables the setting of traffic light phase commands based on input, thereby controlling the feasible area.

[0054] 6. The road condition unit simulates factors affecting vehicle acceleration and speed, such as friction in the driving lane under different weather conditions.

[0055] 7. The vehicle generation unit implements the logic for generating new vehicles entering the simulation environment at the edge of the environment.

[0056] The strategy combination layer 230 combines atomic mechanisms on demand to achieve diversity and flexibility in environmental dynamics. All of these layers can be dynamically adjusted by parameterization as needed.

[0057] 1. Weather configuration strategy: Flexible combination of different weather conditions to form rich and variable weather factors, thereby affecting vehicle behavior in the environment and realizing the diversity of vehicle status at each intersection in the simulator.

[0058] 2. Traffic flow configuration strategy: Import road configuration information from the real environment and implement the combined configuration of feasible street areas in the simulator.

[0059] 3. Vehicle generation strategy: Set the vehicle generation pattern. This could be to generate new vehicles at a fixed period or according to a random distribution, as well as the birthplace and destination of each vehicle.

[0060] 4. Environmental observation mapping strategy settings: When making decisions, the traffic lights at each intersection can see the corresponding traffic information of which intersections, which can be either global or local observation.

[0061] 5. Traffic light configuration strategy: Set the available action space (optional phase) for the current traffic light.

[0062] The interaction interface layer 240 provides functions such as interaction protocol between environment and traffic light control strategy, visualization interface for environmental state changes, historical trajectory reproduction, environmental diversification strategy management and dynamic strategy update.

[0063] 1. A visual interactive interface displays in real time the interaction process between the strategy and the traffic simulator, as well as the changes in environmental conditions.

[0064] 2. The historical trajectory demonstration tool enables the reproduction of the environment and strategy interaction process based on the historical interaction action trajectory, and obtains the trajectory of environmental state changes.

[0065] 3. The interaction protocol between the strategy and the simulator enables functions such as assembling observation data after environment initialization, parsing and executing intelligent agent action protocols during environment dynamics runtime, and assembling observations after state space updates.

[0066] Once the intersection topology is determined, the changes in the environmental state (local or global) observed by the strategy in the simulator are determined by a combination of factors: weather factors, vehicle behavior, and traffic light phases. Changes in weather factors and traffic light phases are low-frequency behaviors; the former may occur only every few minutes or even hours, while the latter may occur every few seconds to tens of seconds. Therefore, their changes themselves do not significantly affect the simulator's performance. However, vehicle behavior and state are continuous, high-frequency changes, requiring updates in every simulation frame to accurately reflect the specific changes in the simulated environment. As mentioned earlier, if each vehicle is evaluated individually in each frame, the complexity of this state update increases quadratically with the number of vehicles.

[0067] The vehicle state management method based on simulation environment provided in this application includes two methods that can significantly reduce the complexity required for vehicle state updates: stationary vehicle elimination method and nearby vehicle detection method.

[0068] The stationary vehicle elimination method targets multiple vehicle queues connected before and after the target lane in a traffic simulation environment. Each vehicle queue contains vehicles of the same type of motion state, and the motion states of two adjacent vehicle queues are different. The motion states include forward and stationary states. Based on the traffic state corresponding to the target lane, target vehicles that meet the update conditions are identified from the multiple vehicle queues. An update operation is performed on the target vehicles to update at least one of the target vehicles' position, speed, and motion state. Among these, stationary vehicles in the stationary queue, except for those that can pass, are skipped. The stationary queue refers to a queue of vehicles that are in a stationary state.

[0069] The adjacent vehicle detection method divides each lane in the traffic simulation environment into multiple spatial grids. Each spatial grid is a fixed area with a preset parking space capacity and a corresponding vehicle information index, which indicates the vehicles contained in the spatial grid. The vehicle information index corresponding to the spatial grid is updated based on the update operation. When there is a first vehicle in the traffic simulation environment that has a lane change requirement, the adjacent spatial grids that meet the adjacent detection conditions are determined. Based on the vehicle information indexes corresponding to the adjacent spatial grids, the lane change situation of the first vehicle is determined.

[0070] In summary, the method provided in this application reduces the computational cost of vehicle state updates based on the stationary vehicle elimination method. For traffic intersections with a high number of vehicles, it can directly reduce vehicle update costs by 50%, fundamentally changing the situation where the computational cost increases quadratically with the number of vehicles in the traditional vehicle-by-vehicle update mode. It divides lane vehicles into alternating queues based on their dynamic / static states, updating only vehicles that meet the update conditions (all vehicles in the moving queue and vehicles at the head of the stopped queue that can pass), skipping invalid calculations for stationary vehicles not at the head of the stopped queue, significantly reducing the number of vehicles processed per simulation frame. It combines lane traffic light phases (passable / impassable) to formulate differentiated queue update logic, conforming to actual traffic vehicle driving patterns, improving update efficiency while ensuring the realism of traffic simulation, accurately restoring the dynamic characteristics of vehicle start-stop and driving. Using vehicle queues as the basic update unit simplifies the vehicle state judgment logic within the simulation frame, reducing redundant operations of vehicle-by-vehicle checks and judgments, improving the overall efficiency of vehicle state updates per frame, and ensuring the traffic simulation environment's need for high-frequency vehicle state updates. Based on the nearest-neighbor vehicle detection method, this approach completely changes the traditional lane change detection model, which requires checking every vehicle in the target lane one by one. It reduces the computational complexity of lane change detection from quadratic to linear, significantly decreasing unnecessary computation. By dividing the lane into spatial grids with fixed parking space capacity and configuring vehicle information indexes, lane change detection only needs to check the limited spatial grids adjacent to the grid of the vehicle to be changed into. The detection range is precise and controllable, avoiding redundant operations of full-lane scanning. Vehicle position updates and grid vehicle information index updates are performed synchronously, ensuring real-time consistency between index data and actual vehicle status. Vehicle information from adjacent grids is quickly retrieved based on the index, improving the efficiency and accuracy of lane change detection and effectively avoiding lane change collision risks. The detection complexity is only related to the number of vehicles to be changed into and the number of fixed adjacent grids, and is almost unaffected by the total number of vehicles in the lane. It maintains high detection efficiency even in large-scale traffic flow simulation scenarios, adapting to the needs of high-load traffic simulation.

[0071] This is illustrative; please refer to it. Figure 3 This document illustrates a flowchart of a vehicle state management method based on a simulation environment, provided in an exemplary embodiment of this application. This method can be executed by a terminal, a server, or both simultaneously. This embodiment uses the execution of the method by an environment simulator as an example for illustration. Figure 3 As shown, the method includes the following steps.

[0072] Step 310: Obtain the traffic simulation environment.

[0073] Traffic simulation environment refers to a virtual traffic environment created for training reinforcement learning agents. It is used to simulate traffic elements such as lanes, vehicles, and traffic lights at traffic intersections, as well as the dynamic changes of each element. It supports real-time updates of vehicle status and interactive feedback of traffic control actions, and serves as a virtual training platform for reinforcement learning agents to learn traffic control decisions.

[0074] The traffic simulation environment includes target lanes, which are lanes selected in the simulation environment for vehicle queue maintenance and vehicle status updates. They are the objects of vehicle status management and have clear driving directions, stop lines, and traffic light phase matching attributes.

[0075] The traffic simulation environment provided in this application provides a repeatable, intervention-friendly, and quantifiable experimental platform for traffic control strategy verification, algorithm model training, and traffic facility effectiveness evaluation by digitally modeling and dynamically simulating the road network topology, traffic facilities, vehicle behavior, and control rules of the regional traffic system.

[0076] Optionally, this traffic simulation environment can achieve multi-dimensional measurement, testing, analysis, and evaluation of traffic conditions, including but not limited to measuring individual driving data such as vehicle position coordinates, real-time speed, motion state, and distance to the vehicle in front / stop line in the simulation scenario, as well as group operation indicators such as vehicle queue length, average driving speed, average waiting time, and traffic flow in a single lane / the entire road network; testing traffic efficiency, congestion mitigation effect, and accident rate under different traffic control schemes such as traffic light phases, lane control rules, and lane changing strategies, and verifying the impact of various strategies on the overall operation of the traffic system; simulating complex working conditions such as peak hours, rain and snow, traffic accidents, and road construction, and testing the robustness, response speed, and decision rationality of various algorithm models in extreme or abnormal scenarios, providing data support for algorithm iteration and optimization; generating standardized environmental observation data to test the decision convergence speed, control effect stability, and policy generalization ability of reinforcement learning agents, and to evaluate the traffic control performance of agents in different traffic scenarios.

[0077] In some embodiments, a traffic simulation environment built for training enhanced agents is retrieved from a traffic simulator. This environment has completed the modeling of basic elements such as road network topology, lane attributes, traffic light relationships, and initial vehicle states.

[0078] The traffic simulation environment includes at least one target lane. The method provided in the embodiments of this application can be executed independently for a single lane, or the method provided in the embodiments of this application can be reused in parallel for multiple lanes.

[0079] Step 320: Maintain multiple vehicle queues connected before and after the target lane.

[0080] Each vehicle queue corresponds to the same type of motion state, and the motion states of two adjacent vehicle queues are different. The motion states include forward motion and stationary motion.

[0081] A vehicle queue is a collection of vehicles arranged in the order of their travel within a target lane. All vehicles in the same queue have the same motion state, and adjacent queues have different motion states. It is the basic unit for vehicle status classification and management.

[0082] Motion state refers to the driving state of a vehicle in the target lane in the simulation environment. It includes only two types: forward state and stationary state, and is the core basis for dividing vehicle queues.

[0083] The forward state is one of the movement states of a vehicle. Specifically, it is a state in which the vehicle speed is not zero, it is driving normally in the target lane, and can continue to move forward. The corresponding state is the marching queue. That is, the queue of vehicles in the forward state is the marching queue.

[0084] A stopped state is one of the motion states of a vehicle. Specifically, it means that the vehicle speed is zero and it is stationary in the target lane, unable to move forward. This corresponds to a stopped queue. In other words, a queue of vehicles in a stopped state is a stopped state.

[0085] Each vehicle queue is an independent vehicle status management unit, containing core information such as the total number of vehicles in the queue, the head vehicle (vehicle at the very front of the queue), the tail vehicle (vehicle at the very back of the queue), and the overall movement status of the queue. Queue and vehicle-related data can be retrieved quickly.

[0086] In some embodiments, regarding the definition of motion states, the motion states of all vehicles in the target lane are first determined. A forward motion state is defined as a vehicle with a non-zero speed that can move forward normally within the lane; a stopped motion state is defined as a vehicle with a zero speed that is stationary and queued within the lane, with no other intermediate motion states. Regarding queue division rules, vehicles with the same motion state are grouped into a queue according to their order of travel within the target lane (from the front of the vehicle closest to the stop line to the end of the lane), forming sequentially connected queues. Regarding queue state constraints, it is strictly ensured that the motion states of two adjacent queues are distinct; that is, a forward queue must be followed by a stopped queue, and vice versa, avoiding redundant division of queues with the same state and ensuring that the queue state matches the start-stop connection characteristics of the actual vehicle movement. Regarding queue dynamic maintenance, this step is a continuously executed operation. After each subsequent vehicle state update, if the vehicle's motion state changes (e.g., stopped → forward, forward → stopped), the vehicle queue will be dynamically adjusted in real time, including creating new queues, queue shifting, and vehicle transfer across queues, always maintaining the queue division rules and state constraints.

[0087] To illustrate, the vehicle's speed is measured by a state-aware unit in an environmental simulator to determine the vehicle's motion state.

[0088] The method provided in this application classifies vehicles in the target lane according to their motion state through stateful and structured queue division, allowing subsequent vehicle updates to be directly filtered by queue, reducing invalid vehicle state checks from the source and laying the foundation for targeted updates.

[0089] Step 330: Based on the traffic status corresponding to the target lane, determine the target vehicles that meet the update conditions from multiple vehicle queues.

[0090] Traffic status refers to the traffic permission status of the target lane based on traffic control elements. It is divided into passable and impassable status and is the core basis for determining vehicle renewal conditions. Its determination is directly related to the traffic light phase type matched with the target lane.

[0091] The passable state is one of the passable states of the target lane, that is, the traffic light matching the target lane is in the green phase, and the vehicles in the lane have the legal right to move forward and start.

[0092] The impassable state is one of the passage states of the target lane, that is, the traffic light corresponding to the target lane is in the red phase, and vehicles in the lane are prohibited from moving forward and must stop in front of the stop line.

[0093] Update conditions refer to the criteria for determining whether a vehicle needs to update its position, speed, or motion state in the simulation environment. These conditions are determined by the traffic status of the target lane, the order of the vehicle queue, and the current motion state. Vehicles that meet these criteria are considered target vehicles.

[0094] The target vehicle refers to a vehicle within the target lane that meets the update conditions and requires position, speed, and / or motion status update operations; it is the object of vehicle status update.

[0095] In some embodiments, the traffic status of a target lane is uniquely determined by its dedicated matching traffic light phase, and is divided into a passable state and a non-passable state. When the traffic light is green, the lane is passable and the vehicle has the legal right to move forward / start. When the traffic light is red, the lane is non-passable and the vehicle is prohibited from moving forward and must stop before the stop line.

[0096] Regarding the definition of update conditions, update conditions are the criteria for determining whether a vehicle needs to perform position, speed, or motion status updates. They are determined by the target lane's traffic status, the vehicle queue's order, and the queue's current motion status. The core is "whether the vehicle has the physical conditions and access rights to move its position, adjust its speed, or switch its motion status." Vehicles that meet these criteria are the target vehicles.

[0097] In some embodiments, vehicle queues are used as the basic unit, and each queue and the vehicles in each queue are determined one by one according to the lane traffic status to determine whether they meet the update conditions. Individual vehicles are not directly screened indiscriminately. When screening, the first queue at the front of the target lane is used as the core reference first, and then the subsequent queues are judged in turn. The screening results only retain vehicles that meet the update conditions and exclude stationary vehicles that do not need to be updated.

[0098] Based on access permissions and the actual status of vehicles, the target vehicles that need status updates can be accurately identified, avoiding ineffective checks on vehicles that do not need updates, thus achieving targeted screening and defining a precise implementation scope for subsequent update operations.

[0099] Step 340: Perform an update operation on the target vehicle.

[0100] The update operation is used to update at least one of the target vehicle's position, speed, and motion state, wherein stationary vehicles in the stop queue, excluding passable vehicles, are skipped. The stop queue refers to a queue of vehicles that are in a stopped state.

[0101] An update operation refers to a state adjustment action performed on a target vehicle. It is used to update at least one of the target vehicle's position, speed, and motion state. It is the core action for realizing dynamic updates of vehicle state and includes operations such as position refresh, speed adjustment, and motion state switching.

[0102] A stopped queue is a vehicle queue consisting of vehicles that are stopped. It is one of the types of vehicle queues in the target lane. During the update operation, only the passable vehicles in the queue are processed, and the remaining vehicles are skipped.

[0103] A passable vehicle is a vehicle in a stopped queue that has the conditions to move forward. It is usually the first vehicle in the stopped queue. The determination is based on the distance between it and the last vehicle in the queue of vehicles in front of it meeting the preset passing conditions. It is the only vehicle in the stopped queue that needs to perform an update operation.

[0104] In some embodiments, corresponding update actions are performed according to the actual state of the target vehicle. These actions can be performed individually or in combination, including but not limited to position update (calculating new position coordinates based on driving direction and speed to refresh the vehicle's physical position in the simulation environment), speed update (adjusting driving speed based on distance to the vehicle in front, lane speed limit, remaining distance to the stop line, etc., to refresh vehicle speed data), and motion state update (synchronously updating vehicle acceleration information when the vehicle switches from a stopped state to a forward state or from a forward state to a stopped state).

[0105] When performing update operations, stationary vehicles other than passable vehicles in the stop queue need to be removed, and update operations are only performed on target vehicles. The core removal rule is that when performing update operations on the stop queue, only passable vehicles are subject to the above update operations, and other stationary vehicles are skipped without any state checks, calculations, or updates. For the update rules of the moving queue, for the moving queue consisting of vehicles in a forward state, update operations are performed on all target vehicles in the queue in sequence to ensure real-time and continuous updates of vehicle status during movement, which is consistent with the realism of the simulation.

[0106] After the target vehicle is updated, the latest position, speed, and motion status data of the vehicle are synchronized to the vehicle status database of the traffic simulation environment. At the same time, the queue dynamic maintenance is triggered to adjust the queue structure according to the changes in vehicle status.

[0107] The system performs precise state updates on the selected target vehicles. At the same time, by skipping non-passable stationary vehicles in the stop queue, it completely eliminates invalid calculations in vehicle state updates, significantly reducing the computational cost of single-frame simulation and improving the efficiency of vehicle state updates. Meanwhile, it ensures that the updated vehicle state is consistent with the start-stop and driving characteristics of the actual traffic scene, without sacrificing the realism of the simulation.

[0108] In summary, the method provided in this application maintains alternating vehicle queues for the target lane according to its motion state, determines the target updated vehicle based on the lane's traffic status, and skips non-passable stationary vehicles in the stop queue. This eliminates invalid calculations in vehicle state updates, significantly reduces the computational cost of vehicle state updates in traffic simulation, improves the efficiency of vehicle state updates in simulation frames, and adapts to the high-frequency vehicle state update requirements of traffic simulation.

[0109] In some embodiments, the target vehicle is determined based on the traffic status corresponding to the target lane. Please refer to [reference needed]. Figure 4 , Figure 4 This is a flowchart illustrating a method for determining a target vehicle provided in an exemplary embodiment of this application. This method can be executed by a terminal, a server, or both simultaneously. This embodiment uses the execution of this method by an environment simulator as an example for illustration. Figure 4As shown, step 330 above includes the following steps (the method also includes, for example, the following steps). Figure 3 Steps 310, 320, and 340 shown have been... Figure 4 (As shown in the image).

[0110] Step 331: Obtain the phase type of the target traffic light corresponding to the target lane.

[0111] The target traffic light is a traffic signal control device in the traffic simulation environment that establishes a unique and exclusive matching association with the target lane. It belongs to the intersection traffic light phase control unit of the atomic mechanism layer of the simulation environment. The change of its phase state directly determines the passage permission of the matched target lane. It is the core traffic control element in the simulation environment that controls the driving and starting and stopping behavior of vehicles. It only has a control effect on the update of the vehicle queue status and the driving behavior of vehicles in the matched target lane.

[0112] Traffic light phase refers to the working status of traffic lights that control the passage permission of a target lane. It is divided into green light phase and red light phase, which is the direct basis for determining the passage status of the target lane. The green light phase corresponds to the passage status, and the red light phase corresponds to the passage status.

[0113] Phase type is the core working state classification of the target traffic light. It is the direct quantitative basis for determining the traffic status of the target lane in the simulation environment. It only includes two core types: green light phase and red light phase. It is stored in real time by the intersection traffic light phase control unit and can be retrieved by the vehicle status management unit. Different phase types form a unique mapping relationship with the traffic status of the target lane, which directly determines whether the vehicle in the target lane has the legal right to move forward and start. It is a key preliminary judgment element for screening target vehicles that meet the update conditions during the vehicle status update process.

[0114] In some embodiments, the entity executing this step is the intersection traffic light phase control unit of the atomic mechanism layer of the traffic simulator. This unit receives and stores the phase state of the traffic lights in real time, which can be retrieved in real time by the vehicle state management unit.

[0115] Indicatively, the vehicle status management unit sends a phase query request to the intersection traffic light phase control module, the request containing the unique identifier of the target lane; the intersection traffic light phase control unit retrieves the real-time phase type of the target traffic light matching the target lane in the current frame based on the lane identifier, and feeds it back to the vehicle status management module; in this embodiment, the phase type of the target traffic light only includes two core types: green light phase and red light phase, which correspond to the passability and non-passability permissions of the target lane, respectively, and no other transitional phases participate in the determination of vehicle update conditions.

[0116] The method provided in this application embodiment obtains the original quantitative basis for determining the traffic status of the target lane, provides basic data for the accurate determination of the subsequent traffic status, and ensures the consistency between the traffic status and traffic control rules (traffic light indication).

[0117] Step 332: Determine the traffic status corresponding to the target lane based on the phase type.

[0118] In some embodiments, when the current frame phase type of the target traffic light corresponding to the target lane has been successfully obtained, and the simulation environment has a preset rule for determining the correspondence between phase type and traffic status, the traffic status is determined by the vehicle status management unit of the traffic simulator.

[0119] In illustrative terms, this application embodiment uses a unique mapping determination logic without any other additional determination conditions. If the phase type of the target traffic light is green, the traffic status of the target lane is directly determined to be passable, meaning that vehicles in the target lane have the legal right to move forward, start, or continue moving. If the phase type of the target traffic light is red, the traffic status of the target lane is directly determined to be impassable, meaning that vehicles in the target lane are prohibited from moving forward and must remain stationary or gradually decelerate to a stop before the stop line.

[0120] The vehicle status management unit stores the determined traffic status (passable / inaccessible) of the target lane locally and uses it as the core input parameter for filtering target vehicles, while also associating it with the vehicle queue data of the target lane.

[0121] The method provided in this application converts the phase type of traffic lights into a passage permission determination result that can be directly used for vehicle status updates, and builds a logical bridge between traffic light phase and vehicle queue screening to ensure the matching of subsequent screening logic with traffic rules.

[0122] Step 333: Based on the traffic status, combined with the vehicle queue order and current movement status, filter target vehicles that meet the update conditions.

[0123] The order of arrangement refers to the sequence of vehicles in the queue from the front end near the stop line to the end of the lane. The current state of motion refers to whether the queue is currently moving forward or stopped.

[0124] In some embodiments, the vehicle state management unit of the traffic simulator determines the target vehicle. The core screening rules include queue-level screening priority, first queue core reference, state matching screening, and unified update conditions. For queue-level screening priority, the vehicle queue is used as the basic screening unit, rather than an individual vehicle. It first determines whether the queue has the possibility of being updated, and then filters the target vehicles in the queue, reducing the number of times a single vehicle is judged. For the first queue core reference, regardless of whether the target lane is passable or impassable, the first queue at the front of the target lane is used as the screening starting point, and then the subsequent queues are judged in turn, which conforms to the physical rule that the preceding vehicle determines the state of the following vehicle in the traffic scenario. For state matching screening, the passability state and the queue movement state are combined to only filter vehicles that have the physical conditions of position movement, speed adjustment, or movement state switching, and exclude stationary vehicles that have no possibility of being updated. For unified update conditions, the update condition of a vehicle is "whether it has both passability permission and physical movement conditions". If either one is missing, it is determined that it does not meet the update conditions and is skipped directly.

[0125] To illustrate, the process first retrieves the traffic status determination results of the target lane, as well as the order table and status information table of all vehicle queues within the target lane, constructing a queue-state association dataset. Starting with the first queue, differentiated queue filtering logic is executed based on the different traffic statuses (passable / impassable). For each queue to be filtered, its motion state and positional relationship with the preceding queue are considered to determine if any vehicles within the queue meet the update conditions: if it is a moving queue, all vehicles in the queue meet the update conditions; if it is a stopped queue, only the head vehicle may meet the update conditions (it is necessary to determine whether the distance to the preceding queue meets the passage conditions). Vehicles that meet the update conditions are marked, generating a target vehicle list containing the vehicle's unique identifier, its queue, and the update type to be performed (position / speed / motion state), providing a direct basis for subsequent update operations. Vehicles determined not to meet the update conditions (mainly stationary vehicles in the stopped queue excluding the head vehicle) are not marked and are directly added to the skip list.

[0126] By using both access permissions and queue status for dual filtering, the system accurately locates target vehicles in the target lane that require status updates, completely eliminating stationary vehicles that do not need updating. This lays the foundation for subsequent targeted updates, reduces invalid calculations in the filtering process, and improves the overall efficiency of vehicle status updates.

[0127] Next, we will explain the differentiated filtering logic for different passage statuses, which are divided into the following two cases based on the passage status.

[0128] The first scenario is that the target lane is passable.

[0129] If the target lane is passable, obtain the current movement state of the first queue in the target lane. The first queue is the first queue of vehicles arranged in the target lane according to the order of arrangement. Based on the current movement state of the first queue, determine the target vehicles that meet the update conditions.

[0130] The passability status of the target lane is uniquely determined by the green phase of the target traffic light. At this time, vehicles in the lane have the legal right to move forward and start passing. The core of the screening logic revolves around the movement status of the first queue, and the principle is to release vehicles that can start and update vehicles that are in normal operation, so as to accurately identify target vehicles that meet the update conditions.

[0131] In some embodiments, once the target lane is determined to be passable, the order table of all vehicle queues in the target lane is retrieved (the order of queues from the front end near the stop line to the end of the lane), the first queue (the first vehicle queue) at the very front is determined, and the current motion state (forward state / stopped state) of the first queue, as well as basic attribute information such as the head vehicle, tail vehicle, and queue spacing of the first queue and all subsequent queues are obtained.

[0132] In the passable state, the vehicle update conditions are having pass permission and having physical movement / state switching conditions. The filtering is based on the movement status of the first queue as the core judgment criterion. The filtering of subsequent queues is based on the processing results of the first queue and proceeds layer by layer. It always follows the rule of full update of the moving queue and only the head vehicle of the stopped queue is judged, skipping stationary vehicles that are not the head of the stopped queue.

[0133] Based on the current movement state of the first queue, two sub-scenes are further subdivided, and corresponding filtering operations are performed for each. Finally, vehicles that meet the conditions in each sub-scene are integrated to form a list of target vehicles.

[0134] Sub-scenario 1: The first queue is in a stopped state.

[0135] In some embodiments, if the first queue is a stopped queue, a moving queue is created as a new first queue, and the new first queue includes the head vehicle in the original first queue; the vehicle queues other than the new first queue are designated as subsequent queues; the subsequent target vehicles that meet the update conditions are determined based on the current motion state of the subsequent queues; and the vehicles in the new first queue and the subsequent target vehicles are designated as target vehicles.

[0136] Indicatively, all vehicles in the first queue are stationary, with only the head vehicle having the physical possibility of starting. The head vehicle in the first queue is first determined to be the target vehicle that meets the update conditions. At the same time, the queue dynamic adjustment rule is triggered to remove the head vehicle from the original stopped queue and create a new forward-moving first queue. The original stopped queue is moved to the subsequent queue. Then, the subsequent target vehicles in the moved subsequent queue are selected according to the rule of including all vehicles in the moving queue and only determining the head vehicle in the stopped queue.

[0137] The method provided in this application embodiment, for scenarios where the first queue is a stopped queue in a passable state, accurately locates the core update vehicle in this scenario by creating a new moving head queue and shifting the remaining queues. It updates only the movable head vehicle and subsequent vehicles that meet the conditions, further eliminating invalid vehicle processing in this specific scenario, thereby improving the vehicle status update efficiency in this scenario.

[0138] Sub-scenario 2: The first queue is in a forward state.

[0139] In some embodiments, when the first queue is a moving queue, vehicle queues other than the first queue are designated as subsequent queues, where a moving queue refers to a vehicle queue in a forward state; subsequent target vehicles that meet the update conditions are determined based on the current motion state of the subsequent queues; and vehicles in the first queue and subsequent target vehicles are designated as target vehicles.

[0140] Indicatively, all vehicles in the first queue are in motion and have the physical ability to move, so all vehicles in the first queue are determined to be target vehicles that meet the update conditions. Based on this, subsequent queues after the first queue are sequentially screened according to the rule of including all vehicles in the moving queue and only checking whether the head vehicle of the stopped queue meets the distance conditions, to determine the subsequent target vehicles.

[0141] The method provided in this application, for scenarios where the first queue is a moving queue under passable conditions, clarifies the rule of full update of the first queue + targeted screening of subsequent queues, making the target vehicle screening in this specific scenario more targeted, avoiding invalid judgments on the moving queue, and ensuring the efficiency and smoothness of vehicle status updates in this scenario.

[0142] Finally, all vehicles in the first and subsequent queues that meet the update conditions are integrated to generate a list of target vehicles in a passable state. The list clearly defines the update type of the vehicles (position / speed / motion status), providing a direct basis for subsequent update operations. All stationary vehicles in the stopped queues, except for the head vehicle, are directly included in the skip list without any further processing.

[0143] The method provided in this application, for scenarios where lanes are passable, uses the movement status of the first queue as the core screening criterion, simplifies the screening logic for target vehicles in passable states, reduces the judgment steps in the screening process, improves the efficiency of determining target vehicles in this scenario, and thus speeds up the overall process of vehicle status updates.

[0144] The second scenario is that the target lane is impassable.

[0145] If the target lane is impassable, obtain the current motion state of the first queue in the target lane, where the first queue is the first queue of vehicles arranged in the target lane according to the order of arrangement; based on at least one of the current motion state of the first queue and the condition of the vehicles in the first queue that are passable, determine the target vehicles that meet the update conditions.

[0146] The impassable state of the target lane is determined solely by the red phase of the target traffic light. At this time, vehicles in the lane are prohibited from moving forward and must remain stationary or gradually decelerate to a stop before the stop line. The core of the filtering logic revolves around the movement status of the first queue and the passable status of the first queue. The principle is to control the movement of vehicles to a stop and only check the connection status of stationary vehicles to accurately determine the target vehicles that meet the update conditions.

[0147] In some embodiments, if the target lane is determined to be impassable, the order table of all vehicle queues in the target lane is retrieved, the first queue at the front is determined, and its current motion state (forward state / stopped state) is obtained; at the same time, information such as the relative position of the first queue to the stop line, the remaining distance between the head vehicle of the first queue and the stop line, and the driving speed of the vehicles in the first queue are retrieved to provide a quantitative basis for determining the passability of the first queue (whether there are vehicles that still have the physical conditions to move forward).

[0148] In a state where passage is not possible, the vehicle update condition is that it only has the physical deceleration / stop condition and no right to move forward. The screening is based on the movement status of the first queue and combined with the situation of the vehicles in the first queue that can pass (whether there are vehicles that have not reached the stop line and can still move forward) for dual judgment. Priority is given to the deceleration / stop operation of the first queue. Subsequent queues only check the connection status between the vehicle at the head of the stopped queue and the vehicle in front, skipping vehicles that do not need to be updated as much as possible.

[0149] The passage status of the first queue refers to whether there are vehicles in the first queue that still have the physical ability to move forward when the queue is impassable. The determination is based on the remaining distance between the head vehicle of the first queue and the stop line and the vehicle's speed. If the head vehicle has not reached the stop line and its speed is not zero, then it is determined that there are vehicles in the first queue that can pass; if the head vehicle has reached the stop line or its speed is zero, then it is determined that there are no vehicles in the first queue that can pass.

[0150] The current motion state of the first queue is used as the primary criterion for classification. A secondary judgment is made based on the number of vehicles that can pass through the first queue. Different sub-scenes are subdivided and corresponding filtering operations are performed. Only vehicles that require physical deceleration / stopping / position adjustment are marked, while other vehicles are skipped directly.

[0151] Sub-scenario 1: The first queue is in a stopped state.

[0152] In some embodiments, if the first queue is a stopped queue, the first queue is skipped; the remaining vehicle queue starting from the second vehicle queue after the first queue is taken as the subsequent queue; the subsequent target vehicle that meets the update conditions is determined based on the current motion state of the subsequent queue; and the vehicles in the first vehicle queue after the first queue and the subsequent target vehicle are determined as the target vehicles.

[0153] Indicatively, vehicles in the first queue are stationary and there are no physical movements or state transitions. The entire first queue is skipped, and no vehicles are included in the target vehicle list. Starting from the second queue after the first queue, only the distance between the head vehicle of the stopped queue and the vehicle in front is checked to determine if there is a need for minor position adjustments. The moving queue is selected as the target vehicle according to the rule of decelerating to a stop.

[0154] The method provided in this application embodiment, for scenarios where the first queue is a stopped queue in an impassable state, eliminates invalid calculations in this specific scenario to the greatest extent by directly skipping the first queue and focusing on subsequent valid queues, simplifies the filtering steps, and greatly improves the efficiency of vehicle status updates in this scenario.

[0155] Sub-scenario 2: The first queue is in a forward state.

[0156] The vehicles in the first queue are still in motion, and further judgment needs to be made based on the number of vehicles that can pass through the first queue.

[0157] In some embodiments, when the first queue is a moving queue, the remaining distance between the head vehicle in the first queue and the stop line is obtained; the passable vehicle situation in the first queue is determined based on the remaining distance; and the target vehicle that meets the update conditions is determined based on the passable vehicle situation.

[0158] The stop line is a boundary line set within the target lane, which vehicles must stop and wait during the red light phase. It is a core reference marker for determining whether the head vehicle in the queue can pass when the lane is impassable.

[0159] The remaining distance refers to the actual distance between the head vehicle of the convoy and the stop line of the target lane when the convoy is impassable. It is a quantitative basis for determining whether the head vehicle has the conditions to continue moving forward.

[0160] The passable vehicles in the first queue can be categorized into two scenarios.

[0161] (1) There are passable vehicles in the first queue.

[0162] In some embodiments, if there are passable vehicles in the first queue, the remaining vehicle queue starting from the second vehicle queue after the first queue is taken as the subsequent queue; the subsequent target vehicle that meets the update conditions is determined based on the current movement state of the subsequent queue; and the vehicles in the first vehicle queue after the first queue and the subsequent target vehicle are determined as the target vehicles.

[0163] For illustration purposes, if there are passable vehicles in the first queue (the head vehicle has not reached the stop line), all vehicles in the first queue are determined to be target vehicles that meet the update conditions. Only deceleration adjustment and position update operations are performed on them, and the prohibition state is switched to forward. Subsequent queues only check the connection status of the head vehicle in the stop queue, and the remaining vehicles are skipped.

[0164] The method provided in this application embodiment, for scenarios where the first queue is a moving queue and there are passable vehicles in an impassable state, clarifies the screening range and rules of subsequent queues, accurately locates the target update vehicle in this specific scenario, eliminates the processing operation of invalid queues, and improves the vehicle status update efficiency in this scenario.

[0165] (2) There are no vehicles in the first queue that can pass.

[0166] In some embodiments, if there are no passable vehicles in the first queue, a stop queue is created as a new first queue, which includes the head vehicle from the original first queue; the remaining vehicle queues starting from the second vehicle queue after the new first queue are designated as subsequent queues; subsequent target vehicles that meet the update conditions are determined based on the current motion state of the subsequent queues; and vehicles in the first vehicle queue after the first queue and subsequent target vehicles are designated as target vehicles.

[0167] As an illustration, if there are no vehicles in the first queue (the first vehicle has reached the stop line), the first vehicle in the first queue is removed, a new first queue in a stopped state is created, and the original first queue is moved to the next queue. Only the first vehicle in the new stopped queue is included in the target vehicle (stop state switching + position update is performed). After the next queue is moved, only the connection status of the first vehicle in the stopped queue is checked, the moving queue decelerates to a stop, and the other vehicles skip.

[0168] The method provided in this application addresses the scenario where the first queue is a moving queue and there are no passing vehicles in a non-passable state. By creating a new stopped head queue and shifting the remaining queues, it standardizes queue management and target vehicle selection in this scenario, avoids invalid updates to stopped vehicles, and improves the accuracy and efficiency of vehicle status updates in this scenario.

[0169] The method provided in this application, for scenarios where the first queue is a moving queue in an impassable state, determines the passable vehicle situation based on the remaining distance between the head vehicle and the stop line. This provides a quantitative and accurate standard for determining passable vehicles in this scenario, avoids screening errors caused by subjective judgment, improves the accuracy of target vehicle determination, and ensures the realism of the simulation.

[0170] Finally, all vehicles in the first and subsequent queues that require deceleration / stopping / position adjustment are integrated to generate a list of target vehicles in an impassable state. It is determined that vehicles can only perform deceleration, position refresh, and stop state switching operations, and are prohibited from performing state updates related to forward movement. All stationary vehicles without any physical adjustment requirements are included in the skip list and are not subject to any further calculations.

[0171] The method provided in this application, for scenarios where lanes are impassable, combines the movement state of the first queue and the status of passable vehicles to filter target vehicles, accurately adapting to the movement characteristics of vehicles in impassable states, avoiding invalid processing of vehicles that do not need updating in this scenario, and improving the accuracy and update efficiency of target vehicle filtering in impassable states.

[0172] The filtering logic for the two traffic states mentioned above has both commonalities and differences in design. The core commonalities include the first queue as the core guide, queue-level filtering priority, the principle of removing stationary vehicles, and linkage with queue maintenance.

[0173] The "First Queue Core Guidance" means that both traffic states use the first queue as the starting point for screening, which conforms to the physical rule that the preceding vehicle determines the movement state of the following vehicle in a traffic scenario. This avoids invalid judgments that would occur if screening started from the rear queue, reducing the logical complexity of the screening process. "Queue-level Screening Priority" means that the vehicle queue is used as the basic screening unit, rather than individual vehicles. The system first determines whether the entire queue has the potential for updating before screening vehicles within the queue, significantly reducing the number of judgments for individual vehicles. "Stationary Vehicle Removal Principle" means that in both states, the system strictly adheres to the rule that only the head vehicle of the stopped queue is judged, and non-head vehicles are skipped. This completely eliminates invalid calculations for stationary vehicles during the screening process, ensuring screening efficiency. "Linkage with Queue Maintenance" means that if vehicle state changes (such as stop → forward, forward → stop) are involved during the screening process, the vehicle queue is dynamically adjusted synchronously to ensure that the queue state is consistent with the actual movement state of the vehicles, laying the foundation for screening and updating in the next frame.

[0174] The core differences include different judgment criteria, different update operation restrictions, and different screening principles.

[0175] The different criteria for judgment mean that in the passable state, only the movement status of the first queue is the core criterion, without considering the stop line factor; in the impassable state, the movement status of the first queue and the passable vehicle situation in the first queue are the dual criteria for judgment, and the remaining distance to the stop line is the key quantitative indicator.

[0176] The difference in update operation restrictions means that when the vehicle is passable, it can perform all types of updates, including position, speed, and motion status, and supports switching between stopped and forward states; when the vehicle is impassable, it can only perform updates to deceleration, position, and stop status, and any switching to forward motion is prohibited.

[0177] The different screening principles mean that the screening principle in the passable state is to release passage and update normally to maximize the matching of vehicle passage demand; while the screening principle in the impassable state is to control and stop, and only check the connection, strictly restricting the forward movement of vehicles and only meeting the necessary physical deceleration / stopping requirements.

[0178] In some embodiments, the stopping queue and the proceeding queue can be directly divided based on a static threshold of whether the speed is zero. Alternatively, a queue merging strategy can be adopted based on a dynamic state threshold of traffic conditions.

[0179] Indicatively, the state perception unit calculates the average vehicle speed and traffic density of the target lane in real time, dynamically adjusting the threshold for determining the forward state. For example, during congestion, a speed of 2 km / h or less is considered a near-stop state and added to the stop queue. A low-speed queue merging mechanism is used, and a new low-speed queue type is added. When a vehicle is detected traveling at extremely low speeds and the distance to the vehicle in front is less than a safe threshold, it is merged from the forward queue to the end of the preceding stop queue, becoming a candidate for passage. For the merged, extremely large stop queue, vehicles not at the head of the queue are directly marked as permanently skipped, further reducing the number of checks.

[0180] By introducing fuzzy state management that adapts to operating conditions, invalid decisions can be further reduced in extreme congestion scenarios, thus solving the efficiency loss problem caused by slow vehicles repeatedly jumping between queues in congested conditions.

[0181] In some embodiments, a hierarchical queuing mechanism for heterogeneous traffic participants can be introduced for vehicle-road cooperative scenarios.

[0182] Intuitively, entities in the simulation environment are divided into three layers: motor vehicles, non-motor vehicles, and pedestrians, with each layer maintaining its own queue. At the stop line area of ​​an intersection, an interactive mapping queue is established between layers. For example, the state of the head vehicle in the non-motor vehicle queue serves as one of the criteria for determining whether a vehicle in the motor vehicle stop queue can pass. If a non-motor vehicle is blocking the way, a motor vehicle cannot pass even if the light is green. Differentiated update rules are used for different queues: step-size updates are used for pedestrians, simplified dynamic updates are used for non-motor vehicles, and the fine-grained updates mentioned above are retained for motor vehicles, achieving efficient simulation of mixed traffic flow.

[0183] By upgrading single-body management to multi-body hierarchical collaborative management, it is possible to accurately simulate the interaction and impact of mixed traffic flows, enabling the simulation environment to adapt to more complex urban intersection scenarios.

[0184] In some embodiments, the method for determining subsequent target vehicles in the above-described differentiated screening logic includes: determining the stop queue and the moving queue from the subsequent queue based on the current motion state; determining the passable vehicles in the stop queue; and determining the passable vehicles in the stop queue and all vehicles in the moving queue as subsequent target vehicles.

[0185] The subsequent target vehicle refers to the target vehicle in the subsequent queue that meets the update conditions. It is determined by the current movement state of the subsequent queue, including all vehicles in the moving queue and passable vehicles in the stopped queue.

[0186] Specifically, the distance between the first vehicle in the stopped queue and the last vehicle in the preceding queue is obtained; if the distance meets the passage conditions, the first vehicle is determined to be a passable vehicle and is moved to the last vehicle in the preceding queue, until the distance between the first vehicle in the stopped queue no longer meets the passage conditions.

[0187] The head vehicle is moved to the end of the previous vehicle queue, that is, the head vehicle is moved to the end of the previous vehicle queue and returned to the previous vehicle queue as the tail vehicle of the previous vehicle queue.

[0188] Indicatively, passage conditions include vehicles reaching a preset distance threshold.

[0189] Optionally, the preset distance threshold can be dynamically determined based on the relative driving speed of the vehicles, the acceleration speed of the head vehicle in the stopped queue, etc., and this application embodiment does not limit it in this way.

[0190] The method provided in this application uses the distance between the head vehicle of the stopped queue and the tail vehicle of the previous queue as a quantitative standard to determine the passable vehicles, and simultaneously realizes the transfer of vehicles across queues, making the determination of passable vehicles more accurate and practical. At the same time, it dynamically optimizes the structure of the vehicle queue, ensures the consistency between queue management and the actual movement state of the vehicles, and lays an efficient queue foundation for updating the vehicle status in subsequent frames.

[0191] The method provided in this application standardizes the selection logic of target vehicles in subsequent queues under various scenarios. With "taking passable vehicles from the stopped queue and updating all vehicles in the moving queue" as the core rule, it reduces the logical differences in subsequent queue selection under different scenarios, improves the standardization and efficiency of determining target vehicles in subsequent queues, and further eliminates invalid calculations in subsequent queues.

[0192] Based on the removal of stationary vehicles, multiple vehicle queues are maintained for each feasible lane. All vehicles in each queue have the same motion state (either moving forward (speed not zero) or stopped (speed zero)). The states of the two queues are different. When the current traffic light phase of the target lane is green (indicating that vehicles in this lane can proceed), if the first queue is in motion, the position information of each vehicle in the queue is refreshed sequentially, and the vehicle speed information is refreshed according to the distance to the vehicle in front. Then, it checks whether the first vehicle in the second queue can proceed (whether the distance between it and the last vehicle in the first queue is large enough to move forward). If the queue can proceed, it starts processing from the head of the queue until the position of the first vehicle that cannot proceed (normally, only one head vehicle needs to be processed in each frame in the same stopped queue, and the status of the remaining vehicles in the queue remains unchanged). Then, it skips the queue and refreshes the position information of each vehicle in the third queue (in motion) sequentially, adjusting the speed according to the distance to the vehicle in front. If the vehicles in the first queue are stationary, the first vehicle in the queue is changed to a moving state, a new head queue is established, and the vehicle is added to the moving queue. The original first queue is set as the second queue (subsequent queues are shifted forward), and the subsequent vehicles in the queue are skipped. It directly processes each vehicle in the next queue, refreshes its position information, and adjusts its speed according to the distance to the vehicle in front. If the current lane signal is red (indicating that the lane cannot proceed), check the status of the first queue in that lane. If it is in motion and the first vehicle has not reached the stop line, update the positions of all vehicles and adjust their speeds according to the distance to the vehicle in front (or the distance to the stop line). If there is a stop queue following this queue, and the first vehicle reaches the stop line, create a new queue and move the first vehicle to the new queue. Set this queue as the first queue for that lane. The remaining queues move to the next queue in sequence, and update the positions and speeds of all vehicles in the second queue (one vehicle remaining in place). For the immediately following stop queue, check if the distance between the first vehicle and the vehicle in front (the last distance of the previous queue) meets the conditions for proceeding. If it does, update the speed and acceleration information of the first vehicle and add it to the end of the previous queue. Then, skip all vehicles in the original stop queue and enter the next moving queue for position and speed updates. For intersections with high vehicle traffic, where normally half of the vehicles are waiting and half are moving, this method can directly reduce vehicle update costs by 50%.

[0193] This is illustrative; please refer to it. Figure 5 , Figure 5 This is a flowchart of a stationary vehicle removal method provided in an exemplary embodiment of this application, as follows: Figure 5As shown, for the method of removing stationary vehicles, the following steps are executed sequentially: Step 510: Check if the lane is passable. If yes, proceed to step 511; otherwise, proceed to step 521. Step 511: Check if the first queue is a stopping queue. If yes, proceed to step 512; otherwise, proceed to step 514. Step 512: Create a moving queue as the first queue and move the first vehicle into it. Step 513: Move the remaining queues forward sequentially. Step 514: Update the position and speed of each vehicle. Step 521: Determine if the first queue is a stopping queue. If yes, proceed to step 522; otherwise, proceed to step 523. Step 522: Skip the stopping queue. Step 523: Determine if the first vehicle in the queue is far enough from the stop line. If yes, proceed to step 526; otherwise, proceed to step 52. 4; Step 524, create a stop queue as the first queue and move the first vehicle into it; Step 525, move the remaining queues forward sequentially; Step 526, update the position and speed of each vehicle in the next moving queue; Step 530, determine if the first vehicle in the next queue is far enough from the last vehicle in the preceding queue. If so, proceed to step 531; otherwise, proceed to step 532; Step 531, update the speed and move the vehicle to the end of the preceding queue; Step 532, skip this queue; Step 533, determine if this queue is the last one. If so, end the process; otherwise, proceed to step 534; Step 534, update the position and speed of each vehicle in the next moving queue; Step 535, determine if this queue is the last one. If so, end the process; otherwise, return to step 530.

[0194] This is illustrative, assuming the target lane is passable, including scenario 1: the first queue is a stopped queue, and the vehicles in the target lane are, in order: car 1 (stopped), car 2 (stopped), car 3 (stopped), car 4 (moving), and car 5 (moving). Cars 1, 2, and 3 form the stopped queue (first queue), and cars 4 and 5 form the moving queue. Within the stopped queue, cars 1, 2, and 3 are all stationary. Only car 1 (the head vehicle) meets the passability requirements and is passable. Cars 2 and 3 are stationary and cannot pass. Skip the update operation and only update car 1 (start → move forward), completely skip car 2 and car 3, and update car 4 and car 5 in the moving queue normally; Case 2: The first queue is the moving queue, which is a moving queue consisting of car 1 (moving) and car 2 (moving), followed by a stopping queue consisting of car 3 (stopped) and car 4 (stopped). There are no stationary vehicles in the moving queue, so all of them are updated. In the stopping queue, car 3 is the first passable vehicle and needs to be updated. Car 4 is a stationary vehicle and is skipped. When performing the update operation, update car 1, car 2, and car 3, and skip car 4.

[0195] When the target lane is impassable, including Case 1: The first queue is a stopped queue, with Car 1 (stopped), Car 2 (stopped), and Car 3 (stopped). The entire first queue consists of stationary vehicles. Under a red light, stationary vehicles do not have the right to pass. When performing an update operation, the entire queue is skipped; none of the vehicles are updated. Only vehicles in the following queues that need to slow down / stop are processed. Case 2: The first queue is a moving queue, including two sub-cases. Sub-case A: The first queue still has room to move (not yet at the stop line), for example, the first queue consists of Car 1 (moving), Car 2 (stopped), and Car 3 (stopped). 2 (line): There is still distance to the stop line. There are currently no stationary vehicles, only moving vehicles. However, at the red light, vehicles can only slow down and cannot continue moving. When performing the update operation, vehicles 1 and 2 will be updated to slow down without being skipped. Vehicles without stationary status need to be removed. Sub-case B: The first queue has reached the stop line and there is no room to pass. Vehicle 1 has reached the stop line and must stop. Vehicle 1 changes from moving to stationary. Vehicle 1 is the head of the new stop queue and its status needs to be updated. If vehicles 2 and 3 are stationary, they will be skipped directly. When performing the update operation, vehicle 1 will be updated (moving → stopped), and other stationary vehicles will be skipped.

[0196] Indicatively, in the stationary vehicle elimination method, in the stopped queue, except for the (head) passable vehicle, all other vehicles are considered stationary and the update is skipped. In the passable state, only the head vehicle of the stopped queue is allowed to start the update, and the other stationary vehicles do not perform any calculations. In the impassable state, the entire stopped queue is considered stationary and the entire queue skips the update. In the moving queue, there are no stationary vehicles, and the entire queue is updated, but only the deceleration update is performed when the light is red.

[0197] It is worth noting that the above-described method for eliminating stationary vehicles is merely an example, and the embodiments of this application do not limit it.

[0198] In summary, the method provided in this application associates lane traffic status with traffic light phase type, clarifies the specific criteria for determining traffic status, provides precise and feasible triggering conditions for target vehicle screening, ensures the standardization and consistency of target vehicle determination logic, further improves the accuracy of vehicle status updates, and avoids invalid screening operations.

[0199] In some embodiments, when there are too many vehicles in the current lane, lane switching may be considered to reduce queuing time. Before switching lanes, it is necessary to check whether the switching conditions are met; otherwise, it may lead to multiple vehicles traveling together and causing a collision. If all vehicles in adjacent lanes are detected, the complexity is multiple times the number of vehicles. This application provides an adjacent vehicle detection method to avoid checking and judging each vehicle one by one. Please refer to [reference needed]. Figure 6 , Figure 6This is a flowchart of a nearby vehicle detection method provided in an exemplary embodiment of this application. This method can be executed by a terminal, a server, or both simultaneously. This embodiment illustrates the method using an environmental simulator as an example. Figure 6 As shown, the method includes the following steps.

[0200] Step 610: Divide each lane in the traffic simulation environment into multiple spatial grids.

[0201] The space grid is a fixed area with a preset parking space capacity. The space grid has a corresponding vehicle information index, which is used to indicate the vehicles that are accommodated in the space grid.

[0202] A spatial grid refers to a fixed area formed by dividing each lane in a traffic simulation environment according to a preset parking capacity. Each spatial grid corresponds to a unique vehicle information index and is the basic spatial unit for realizing the detection of nearby vehicles.

[0203] The preset parking space capacity refers to the capacity standard for dividing the space into grids, that is, the number of vehicles that each grid can accommodate. It is a fixed value and serves as the unified basis for lane grid division.

[0204] The vehicle information index refers to the identification information that corresponds one-to-one with the spatial grid, indicating the actual vehicles contained in that spatial grid. It can be updated in real time with vehicle update operations and is the basis for quickly querying vehicle information within the spatial grid.

[0205] In some embodiments, after the road network topology modeling of the traffic simulation environment has been completed, the basic attributes such as the physical boundaries, driving directions, and lane widths of all lanes have been defined, and the relevant logic for vehicle state updates (stationary vehicle elimination method) has been enabled in the simulation environment, the division of spatial grids is performed. This is carried out collaboratively by the motion unit and vehicle movement unit of the atomic mechanism layer of the traffic simulator. The motion unit is responsible for spatial division, and the vehicle movement unit is responsible for index configuration.

[0206] Regarding the spatial grid division of lanes, a uniform spatial division rule is applied to all feasible lanes in the traffic simulation environment. Each lane is divided into multiple independent spatial grids at equal intervals along its length, according to the vehicle's travel direction. All spatial grids are fixed physical areas with uniform specifications, without differences in shape or size. A preset parking space capacity is defined, and the physical dimensions of each spatial grid are designed according to the preset parking space capacity, with the core being the ability to accommodate a fixed number of standard vehicles (preferably 3 parking spaces in this embodiment). This ensures that the grid division matches the actual spatial characteristics of vehicle driving and parking, without being too large or too small. Spatial redundancy; vehicle information index configuration, assigning a unique vehicle information index to each spatial cell. This index is structured identification information, bound one-to-one with the spatial cell. The index contains core information such as the unique identifier of the vehicle currently contained in the cell, the number of vehicles, the vehicle position (relative position within the cell), and the vehicle's movement status, enabling rapid association and retrieval of cell and vehicle information; global cell identifier, configuring a globally unique identifier for all lane spatial cells, such as "lane X - cell Y", clearly defining the lane to which the cell belongs and its arrangement order within the lane, providing a basis for subsequent determination of adjacent cells across lanes.

[0207] The traditional lane change detection is transformed into a grid-based area detection. The lane is physically divided into fixed-size spatial grids, and each grid is configured with a vehicle information index. This allows vehicle location queries to be retrieved quickly from the index instead of detecting each vehicle individually. This lays the foundation for improving the efficiency of lane change detection from the perspectives of spatial division and data management.

[0208] Step 620: Update the vehicle information index corresponding to the spatial cell based on the update operation.

[0209] In some embodiments, when the spatial grid division of all lanes and the initial configuration of the vehicle information index have been completed, and the vehicle status update operation is being performed / has been completed, and the position, number, and movement status of the vehicles are dynamically changing, the vehicle movement unit of the atomic mechanism layer of the traffic simulator will synchronously update the vehicle information index with the vehicle status update operation.

[0210] The update trigger timing is linked to every update operation of the vehicle status. After updating the position, speed, etc. of the target vehicle, the vehicle information index of the corresponding spatial grid is immediately triggered to ensure real-time synchronization between the index information and the actual vehicle status, without inter-frame delay.

[0211] The index update rule is to determine the change in the spatial grid to which the vehicle belongs based on the vehicle's actual location after the update, and then perform a targeted index update.

[0212] If a vehicle only moves within its original spatial cell, only its relative position within the cell is updated in the index, while other information remains unchanged. If a vehicle moves from its original spatial cell to the next spatial cell in the same lane, its identifier is deleted from the original cell index and added to the new cell index, and the number of vehicles and their relative positions in the new cell are updated. If a new vehicle enters the simulation environment and drives into a cell, or if a vehicle leaves the simulation environment / drives out of a cell, the corresponding vehicle identifier is added or deleted from the cell index, and the number of vehicles is updated.

[0213] In some embodiments, batch update optimization can be performed. For vehicles moving continuously in a batch in the marching queue, the grid index is updated in batches to avoid the repetitive operation of updating one vehicle at a time, and further reduce the computational cost of index update.

[0214] The method provided in this application update the vehicle information index based on the update operation, ensuring that the vehicle information index of the spatial grid is highly consistent with the actual state of the vehicle, making the index a precise data mirror of the vehicle's position and state, ensuring that the grid vehicle information retrieved in subsequent lane change detection is true and valid, and avoiding the extra calculation of separate index maintenance by updating the index in conjunction with the update operation.

[0215] Step 630: When there is a first vehicle in the traffic simulation environment that has a lane change requirement, determine the adjacent spatial grids that meet the adjacent detection conditions with the first vehicle.

[0216] In some embodiments, after lane grid division and real-time updating of vehicle information index have been completed, and the traffic simulator detects the presence of a first vehicle (a vehicle with lane change requirements) in the environment and has obtained basic information such as the current lane, spatial grid, and driving direction of the first vehicle, the rigid body collision unit of the atomic mechanism layer of the traffic simulator performs the detection. This unit provides collision detection and feasibility determination support for vehicle lane change.

[0217] The first vehicle refers to the vehicle in the traffic simulation environment that needs to switch to an adjacent lane due to traffic congestion or slow traffic in its current lane, and is the core object of lane change detection.

[0218] Lane change demand refers to the need for a vehicle in the simulation environment to switch to an adjacent lane due to traffic congestion in its current lane. It is a prerequisite for triggering the detection of nearby vehicles.

[0219] The lane change request is triggered by the vehicle behavior model of the simulator. When the queue length and driving speed of vehicles in the current lane reach the preset threshold, the lane change request is automatically generated for the vehicle.

[0220] Adjacent spatial cells refer to the spatial cells surrounding the cell where a vehicle with lane-changing needs is located, which meet the adjacent detection conditions. This is the only area where vehicle information needs to be queried during lane-change detection.

[0221] Adjacent detection conditions refer to the criteria for determining the spatial grid adjacent to a vehicle with lane-changing needs. Specifically, it's the range of surrounding spatial grids that require vehicle information detection, based on the grid where the vehicle is located. This serves as the standard for defining the lane-change detection range. For the first vehicle, adjacent detection conditions refer to the rules for determining the spatial range of cross-lane grids adjacent to its grid. These rules are fixed preset values, centered on the grid of the first vehicle, and only cover a limited area of ​​surrounding grids requiring detection, without redundant detection across the entire lane or road network.

[0222] Indicatively, the grid to which the first vehicle belongs is determined. Based on the real-time location of the first vehicle, its current target spatial grid (denoted as the core grid) is quickly determined through the vehicle information index, and the globally unique identifier of the core grid is obtained. Adjacent detection areas are determined. Based on the core grid, the spatial range for lane change detection is delineated according to the adjacent detection conditions. This range only includes a limited number of spatial grids in the adjacent lanes of the first vehicle that correspond to the position of the core grid (preferably 6 in this embodiment), and does not involve other grids in adjacent lanes, let alone non-adjacent lanes. Adjacent spatial grids are determined. From the delineated detection range, all spatial grids that meet the conditions are extracted as adjacent spatial grids, and a global identifier list of adjacent grids is generated to delineate a precise limited area for subsequent lane change detection and exclude all irrelevant grids.

[0223] The method provided in this application reduces the spatial range of lane change detection from the entire adjacent lane to a limited number of adjacent grids around the core grid, completely abandoning the traditional full-lane detection mode. This significantly reduces the number of vehicles that need to be verified from the perspective of the detection range, which is the core step in reducing the complexity of lane change detection.

[0224] Step 640: Determine the lane change status of the first vehicle based on the vehicle information index corresponding to the adjacent spatial grid.

[0225] Lane change scenarios refer to situations where a vehicle with a lane change requirement determines whether it is safe to change lanes based on the vehicle information index of adjacent spatial grids. These scenarios are categorized into two types: safe lane change and unsafe lane change.

[0226] In some embodiments, when the list of adjacent spatial grids of the first vehicle has been determined and the vehicle information indexes of all adjacent spatial grids are valid data that are updated in real time, the rigid body collision unit of the atomic mechanism layer of the traffic simulator completes the lane change feasibility determination based on the index information.

[0227] To illustrate, the index information is retrieved. Based on the global identifier list of adjacent spatial grids, the vehicle information index corresponding to all adjacent grids is retrieved in batches at once to obtain data such as the number of vehicles, vehicle identifiers, vehicle movement status, and relative positions of vehicles in the grid, without the need to scan each vehicle in the grid individually. For lane change feasibility determination, based on the retrieved index information, the lane change situation of the first vehicle is determined according to the preset safe lane change rules. The determination is only based on the vehicle information in adjacent grids, without other redundant determination conditions.

[0228] If there are no vehicles in the adjacent space grid, the first vehicle is deemed to be able to change lanes safely. If there are vehicles in the adjacent space grid, the actual distance between the first vehicle and the vehicles in the grid is calculated (based on the relative position of the vehicles in the grid index + the physical size of the grid). If the distance is greater than the preset safe lane-changing distance, the first vehicle is deemed to be able to change lanes safely. If the distance is less than or equal to the preset safe lane-changing distance, the first vehicle is deemed not to be able to change lanes (there is a risk of collision).

[0229] The final output is the lane change determination result, which is fed back to the behavior model of the first vehicle. If the lane change is determined to be safe, the first vehicle is triggered to change lanes and the subsequent vehicle status and grid index are updated. If the lane change is determined to be unsafe, the first vehicle cancels the lane change request and continues to drive in the original lane.

[0230] The method provided in this application embodiment, based on precise data of grid index, completes the lane change feasibility determination within a limited adjacent grid area, which not only ensures the accuracy and safety of lane change detection (no collision risk), but also avoids vehicle-by-vehicle verification of all vehicles in adjacent lanes, reducing the computational complexity of lane change detection from quadratic to linear.

[0231] When there are too many vehicles in a lane, lane switching is considered to reduce queuing time. Before switching lanes, it is necessary to check whether the switching conditions are met; otherwise, it may lead to multiple vehicles colliding. If all vehicles in adjacent lanes are detected, the complexity is a multiple of the number of vehicles. The adjacent vehicle detection method avoids checking and judging every vehicle one by one. Each lane is divided into two grids that can accommodate three parking spaces. Each grid has fixed index information. When refreshing the vehicle position according to the method in the previous section, the vehicle index in each grid is updated. When checking whether an adjacent lane can be switched, it is only necessary to check the status of the vehicles in the adjacent grids and their distance from the current lane to determine whether lane switching is possible. This method can directly reduce the quadratic complexity of adjacent vehicle detection to a linear multiple of the number of vehicles. For illustration, please refer to [reference needed]. Figure 7 , Figure 7 This is a schematic diagram of nearby vehicle detection provided in an exemplary embodiment of this application, such as... Figure 7As shown, each lane is divided into spatial cells, resulting in Cell1_1, Cell1_2, Cell1_3, Cell2_1, Cell2_2, Cell2_3, Cell3_1, Cell3_2, and Cell3_3. For vehicle 710 planning to change lanes, based on its spatial cell Cell2_2, the lane change inspection area corresponding to vehicle 710 includes 6 adjacent spatial cells, namely Cell1_1, Cell1_2, Cell1_3, Cell3_1, Cell3_2, and Cell3_3. It is only necessary to check the vehicle index in these six areas to complete the task, and the computational complexity is 6 times that of the number of vehicles to change lanes.

[0232] In some embodiments, proactive lane change risk pre-detection can also be performed based on spatiotemporal attention by combining vehicle trajectory prediction.

[0233] As an illustration, a trajectory feature dimension is added to the grid index, such as the lateral offset and speed change rate of three consecutive frames; the lane change detection module no longer scans all adjacent grids, but uses an attention mechanism to prioritize the detection of adjacent grids of vehicles with lane change intentions; for high-probability lane change vehicles, the target grid is marked as pre-occupied in advance, and the detection range is directly locked when the next frame is updated to avoid conflict judgments of multiple vehicles changing lanes at the same time.

[0234] By transforming passive detection triggered by demand into proactive prediction, and by combining spatiotemporal features to narrow the detection range, the collision detection conflict problem when multiple vehicles change lanes concurrently is solved, while the average response time of lane change detection is shortened.

[0235] In summary, the method provided in this application adds lane gridding and adjacent vehicle detection logic to the vehicle state management, reducing the scope of vehicle lane change detection from all vehicles in the lane to adjacent spatial grids, significantly reducing the computational complexity of lane change detection, avoiding invalid calculations in lane change detection, further improving the overall computational efficiency of traffic simulation, and ensuring the accuracy of vehicle lane change judgment.

[0236] In some embodiments, based on the above-described vehicle state management, the vehicle state update results are fed back to the traffic simulation environment to generate observation information, thereby achieving a connection between vehicle state updates and reinforcement learning agent training. Please refer to... Figure 8 , Figure 8 This is a flowchart of a reinforcement learning method based on a simulation environment, provided in an exemplary embodiment of this application. This method can be executed by a terminal, a server, or both simultaneously. This embodiment illustrates the method using an environment simulator as an example. Figure 8 As shown, the following steps are included after step 340 above.

[0237] Step 810: Based on the update operation, the updated vehicle status information is fed back to the status perception unit of the traffic simulation environment.

[0238] Vehicle status information refers to the latest position, speed, motion status, and other data obtained after the target vehicle has undergone an update operation. It is the core data reflecting the real-time status of the vehicle in the simulation environment.

[0239] The state perception unit refers to the functional unit used to collect and process vehicle state information in the simulation environment. It can generate observation information based on the updated vehicle state information and is the intermediate unit connecting vehicle state updates and reinforcement learning agent training.

[0240] In some embodiments, after the state update operation (single or combined update of position, speed, and motion state) of the target vehicle in the target lane has been completed, and the vehicle information index of the lane space grid has been synchronously updated, and a state perception unit has been deployed in the traffic simulation environment, the vehicle movement unit of the atomic mechanism layer of the traffic simulator executes step 810, which is completed synchronously with the vehicle state update operation and the grid index update operation, without inter-frame data delay. The state perception unit is a collaborative functional unit of the atomic mechanism layer and the interaction interface layer of the simulation environment, and has the ability to receive, collect, and preprocess vehicle state data.

[0241] Indicatively, regarding data extraction and encapsulation, the vehicle movement unit extracts the latest status information of all target vehicles involved in the current update operation from the vehicle status database of the simulation environment. This information is then encapsulated according to the lane-vehicle structured dimension, along with associated information such as the update timestamp, lane accessibility (passable / impassable), and traffic light phase type, forming a standardized vehicle status data message. For real-time data feedback, the encapsulated vehicle status data message is pushed to the status perception unit in real-time via the unit communication protocol within the simulation environment. This push process is synchronized with the vehicle status updates in the simulation frames, ensuring updates are performed on every frame. The state data can be captured instantly by the state perception unit. For full / incremental feedback adaptation, incremental or full feedback is flexibly adopted according to the range of vehicle state updates. When only a single lane or a small number of vehicles are updated, only incremental update data is fed back. When the global vehicle state in the simulation environment changes on a large scale, full state data feedback is performed, balancing data transmission efficiency and integrity. For data verification and completion, after receiving the data, the state perception unit performs preliminary verification of the data's integrity and validity, and automatically completes missing key fields (such as vehicle position and lane) to ensure that the data source for subsequent observation information generation is accurate.

[0242] The method provided in this application aggregates real-time vehicle status data scattered across the atomic mechanism layer into a unified state perception unit, breaking down data silos between modules and providing standardized, accurate, and real-time basic data for the generation of observation information, thus achieving preliminary connection between the physical state of the simulation environment and the perception data of the intelligent agent.

[0243] Step 820: The state perception unit generates observation information about the traffic simulation environment.

[0244] The observation information is used to provide the reinforcement learning agent with perceptible traffic state data, which in turn is used to learn traffic control decisions based on the traffic simulation environment.

[0245] Observational information refers to structured data generated by the state perception unit based on the updated vehicle state information. It is the perceptible traffic state data output by the simulation environment to the reinforcement learning agent, providing a direct basis for the agent to make traffic control decisions.

[0246] A reinforcement learning agent is a decision-making entity that can perceive the observation information output by the simulation environment, generate traffic control actions based on its own learning strategy, and continuously iterate and optimize its own decision-making strategy through reward information from the environment feedback in order to achieve the traffic control goal.

[0247] Traffic control decision refers to the decision instructions generated by a reinforcement learning agent based on observation information to control the phase switching of traffic lights. It is the core output of the reinforcement learning agent and can be applied to the simulation environment and affect the traffic status of lanes.

[0248] In some embodiments, observation information is generated after the state perception unit has successfully received and verified the updated vehicle state information, and the traffic simulation environment has preset the generation rules (such as observation range, data dimension, encoding format, etc.) for observation information through an environmental observation mapping strategy. This strategy is configured by the simulation environment strategy combination layer and can be flexibly adjusted as needed. The process is executed collaboratively by the state perception unit and the observation mapping unit of the atomic mechanism layer of the traffic simulator. The state perception unit is responsible for data integration, while the observation mapping unit is responsible for data filtering, encoding, and format conversion.

[0249] Regarding the process of generating observation information, the first step is to filter data according to the observation strategy.

[0250] The observation mapping module selects core data directly related to the traffic control decisions of the reinforcement learning agent from the full data of the state perception unit according to the preset environmental observation mapping strategy. The selection rules are adapted to the observation range requirements of the agent (configurable as global observation or local observation).

[0251] Global observation involves filtering all data in the simulation environment, including vehicle status, traffic light phase, and lane traffic status for all lanes. Local observation involves filtering only local data related to the target intersection / lane controlled by the agent, such as vehicle status (e.g., queue length, average speed, position of the first vehicle in the queue), traffic light phase, and lane control / accidents, while eliminating redundant data from irrelevant road networks.

[0252] Secondly, integrate data dimensions according to decision-making needs.

[0253] The core data after screening is integrated according to the key dimensions of traffic control decision-making to form a structured dataset. The integrated core dimensions include, but are not limited to: vehicle queue distribution of the target lane, number of vehicles in each queue, average vehicle speed, average queue length, distance of vehicles before the stop line, current phase and remaining duration of traffic lights, lane traffic status / control status, etc. All dimensions are the key basis for the agent to make decisions on traffic light phase and duration control.

[0254] Next, standardized coding processing is performed.

[0255] The integrated structured dataset is standardized and encoded to transform physical data (such as location coordinates, distance, and speed) in the traffic simulation environment into numerical / vector data that can be recognized by the reinforcement learning agent. This unifies the data units and value ranges, eliminates differences in data dimensions, and ensures efficient data processing by the agent. At the same time, unique identifiers are added to data of different dimensions to facilitate feature extraction by the agent.

[0256] Finally, the observation information is output according to the interaction protocol.

[0257] After encoding, the encoded data is encapsulated into standardized observation information according to the agent-environment interaction protocol preset in the simulation environment interaction interface layer. This observation information is in a structured data format, which is fully compatible with the decision input interface of the reinforcement learning agent and can be directly pushed to the agent.

[0258] The generated observation information is pushed to the reinforcement learning agent in real time through the communication module (shared memory / network communication) of the simulation environment's basic service layer, becoming the sole basis for the agent to perceive the state of the traffic simulation environment. After receiving the observation information, the reinforcement learning agent combines its learned traffic control strategies to generate corresponding traffic control decisions (such as traffic light phase switching, phase duration adjustment, etc.) and feeds these decisions back to the intersection traffic light phase control unit of the traffic simulation environment. Based on the agent's control decisions, the traffic simulation environment adjusts the traffic light phase and traffic status of the target lane, thereby triggering the vehicle state update in the next simulation frame, forming a reinforcement learning closed loop of vehicle state update → observation information generation → agent decision → traffic light state adjustment → vehicle state update again.

[0259] The method provided in this application transforms the raw and scattered vehicle physical state data in a traffic simulation environment into standardized observation information that can be perceived and utilized by a reinforcement learning agent. This enables precise data interaction between the simulation environment and the agent, providing a real-time and accurate data source for the agent to learn and iterate traffic control decisions. It ensures that the reinforcement learning agent can optimize strategies based on real traffic simulation states, ultimately achieving intelligent control of traffic lights.

[0260] The environmental simulator provided in this application automatically simulates and continuously measures (collects / tests) and processes multi-dimensional traffic state data during simulation operation. This traffic state data is not limited to a specific physical quantity, but covers various related variables such as the vehicle's position coordinates in the simulation environment, real-time driving speed, movement start / stop status, distance to the vehicle in front / stop line, vehicle queue length in the target lane, average waiting time, traffic light phase status, and lane-keeping permissions. The state perception unit processes these various types of data, transforming the original, scattered physical state information into structured perception data that can be directly used by the intelligent agent, providing comprehensive state-based information for traffic decision-making.

[0261] Based on general simulation data interaction rules, the embodiments of this application can flexibly adjust the perception range and data dimensions according to the needs of different training scenarios. It can measure the full state information of the global road network or focus only on the local state data of the target intersection or target lane. In the vehicle state update and lane change detection stages, it will also complete the vehicle state adjustment and safe lane change judgment based on the collaborative judgment of multiple state variables. The whole process does not rely on a dedicated measurement device for a single variable, but achieves efficient processing and transmission of multiple types of data through a unified state management mechanism.

[0262] The simulation test scenario constructed in this application embodiment can be adapted to various complex working conditions such as peak hours, rain and snow, traffic control, and traffic accidents. It is used to verify / test the vehicle behavior patterns and control strategy effects under different traffic conditions. Using a traffic simulator as a carrier, it realizes the collaborative perception (measurement), data conversion, and strategy verification (testing) of multiple types of traffic state variables, providing a stable and efficient environment for the training of reinforcement learning agents.

[0263] Indicative, such as Figure 2 Taking the environmental simulator shown as an example, it can be applied to traffic simulation scenarios such as intelligent transportation projects. The user side can perceive the speed of sample production and strategy learning by comparing it with the actual simulator.

[0264] Please refer to Figure 9 , Figure 9 This is a schematic diagram illustrating a traffic simulation environment application provided in an exemplary embodiment of this application, such as... Figure 9As shown, traffic simulation environments can be used to publish traffic-related tasks. Users can complete relevant model tasks using reinforcement learning agents. The system can display a competition introduction (910), single-round simulation training results (920), and a multi-task training summary (930). The competition introduction (910) describes the competition task. The single-round simulation training results (920) displays the game identifier, algorithm type, training mode, core score, and key evaluation indicators such as average vehicle delay, waiting time, and queue length for that round of training. It also marks complex environmental conditions such as peak hours, snowy weather, traffic control, and traffic accidents, demonstrating that the simulation environment supported by this patented technology can accurately simulate complex traffic scenarios and output multi-dimensional quantitative evaluation data, providing accurate feedback for reinforcement learning agent training. The multi-task training summary (930) presents the overall configuration of the training task, training duration, training steps, and average core indicators of multiple rounds of training. It also provides detailed operation access points such as downloading logs, recording videos, and quick review, confirming that the simulation environment built by this patented technology has high stability and efficiency, can support large-scale and batch training of reinforcement learning agents, and the training process is reproducible and debuggable.

[0265] According to the method provided in the embodiments of this application, the simulation efficiency of traffic intersections is significantly improved, and the sample generation efficiency of online reinforcement learning is greatly enhanced. Please refer to [link / reference] for the changes in performance before and after using the method provided in the embodiments of this application. Figure 10 , Figure 10 This is a comparison diagram of the effects provided by an exemplary embodiment of this application, such as... Figure 10 As shown, without this method, the corresponding step function performance 1, when the number of vehicles reaches 1000, requires 3000-6000 milliseconds to generate samples for a single step of reinforcement learning; with this method, the corresponding step function performance 2, when there are 1000 vehicles, reduces the sample generation time required for a single step of reinforcement learning to 600-1200 milliseconds.

[0266] In summary, the method provided in this application feeds back the vehicle state update results to the simulation environment and generates observation information, thereby achieving an effective connection between vehicle state updates and reinforcement learning agent training. This allows the updated vehicle state data to be quickly transformed into training data that the agent can perceive, ensuring that the reinforcement learning agent can obtain accurate and real-time traffic state data and improving the learning efficiency and accuracy of the agent's traffic control decisions.

[0267] Figure 11 This is a structural block diagram of a vehicle state management device based on a simulation environment provided in an exemplary embodiment of this application, such as... Figure 11 As shown, the device includes the following parts: The acquisition module 1110 is used to acquire a traffic simulation environment, the traffic simulation environment including the target lane; Management module 1120 is used to maintain multiple vehicle queues connected to each other for the target lane. The vehicles in each vehicle queue correspond to the same type of movement state, and the movement states of two adjacent vehicle queues are different. The movement states include forward state and stopped state. The determination module 1130 is used to determine the target vehicle that meets the update conditions in the multiple vehicle queues based on the traffic status corresponding to the target lane. The management module 1120 is further configured to perform an update operation on the target vehicle. The update operation is used to update at least one of the target vehicle's position, speed, and motion state, wherein stationary vehicles other than passable vehicles in the stop queue are skipped. The stop queue refers to a queue of vehicles in the stopped state.

[0268] In some embodiments, the acquisition module 1110 is further configured to acquire the phase type of the target traffic light corresponding to the target lane; The determining module 1130 is further configured to determine the traffic status corresponding to the target lane based on the phase type; The determining module 1130 is further configured to, based on the traffic status, and in conjunction with the arrangement order of the vehicle queue and the current movement status, filter the target vehicles that meet the update conditions.

[0269] In some embodiments, the acquisition module 1110 is further configured to acquire the movement state of the first queue in the target lane when the target lane is in a passable state, wherein the first queue is the first queue of vehicles arranged in the target lane according to the arrangement order. The determining module 1130 is further configured to determine the target vehicle that meets the update conditions based on the current motion state of the first queue.

[0270] In some embodiments, the determining module 1130 is further configured to: If the first queue belongs to the stopped queue, a moving queue is created as a new first queue. The moving queue refers to the queue of vehicles in the forward state. The new first queue includes the first vehicle in the original first queue. Vehicle queues other than the new first queue will be designated as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the new first queue and the subsequent target vehicles are identified as the target vehicles.

[0271] In some embodiments, the determining module 1130 is further configured to: If the first queue is a moving queue, then the vehicle queues other than the first queue are designated as subsequent queues, where the moving queue refers to the vehicle queues in the forward state. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first queue and the subsequent target vehicles are identified as the target vehicles.

[0272] In some embodiments, the acquisition module 1110 is further configured to acquire the current movement state of the first queue in the target lane when the target lane is in an impassable state, wherein the first queue is the first queue of vehicles arranged in the target lane according to the arrangement order. The determining module 1130 is further configured to determine the target vehicle that meets the update conditions based on at least one of the current motion state of the first queue and the passable vehicle situation in the first queue.

[0273] In some embodiments, the determining module 1130 is further configured to: If the first queue belongs to the stop queue, skip the first queue; The remaining vehicle queues, starting from the second vehicle queue after the first queue, are designated as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

[0274] In some embodiments, the acquisition module 1110 is further configured to acquire the remaining distance between the head vehicle in the first queue and the stop line when the first queue belongs to a marching queue; The determining module 1130 is further configured to determine the passable vehicle situation in the first queue based on the remaining distance; The determining module 1130 is further configured to determine the target vehicle that meets the update conditions based on the passable vehicle situation.

[0275] In some embodiments, the determining module 1130 is further configured to: If there are passable vehicles in the first queue, the remaining vehicle queue starting from the second vehicle queue after the first queue shall be used as the subsequent queue. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

[0276] In some embodiments, the determining module 1130 is further configured to: If there are no passable vehicles in the first queue, a stop queue is created as a new first queue, and the new first queue includes the first vehicle in the original first queue. The remaining vehicle queues, starting from the second vehicle queue after the new first queue, are used as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

[0277] In some embodiments, the determining module 1130 is further configured to: The stopping queue and the moving queue are determined from the subsequent queues based on the current motion state; Determine the passable vehicles in the stop queue of the subsequent queue; The passable vehicles in the stopped queue and all vehicles in the moving queue are identified as the subsequent target vehicles.

[0278] In some embodiments, the acquisition module 1110 is further configured to acquire the vehicle distance between the head vehicle in the stop queue and the tail vehicle in the previous vehicle queue. The determining module 1130 is further configured to, when the vehicle distance meets the passage conditions, determine the head vehicle as the passable vehicle, and move the head vehicle to the tail of the previous vehicle queue, until the vehicle distance corresponding to the head vehicle in the stopped queue does not meet the passage conditions.

[0279] In some embodiments, the management module 1120 is further configured to divide each lane in the traffic simulation environment into multiple spatial grids, each spatial grid being a fixed area with a preset parking space capacity, each spatial grid corresponding to a vehicle information index, the vehicle information index being used to indicate the vehicles accommodated in the spatial grid; The management module 1120 is also used to update the vehicle information index corresponding to the spatial grid based on the update operation; The determining module 1130 is further configured to determine adjacent spatial grids that meet the adjacent detection conditions with respect to the first vehicle when there is a first vehicle in the traffic simulation environment that has a lane change requirement. The determining module 1130 is further configured to determine the reversible lane status of the first vehicle based on the vehicle information index corresponding to the adjacent spatial grid.

[0280] Please refer to Figure 12 , Figure 12 This is a structural block diagram of a vehicle state management device module based on a simulation environment provided in an exemplary embodiment of this application, as shown below. Figure 12 As shown, in some embodiments, the device further includes a feedback module 1140 and a generation module 1150; The feedback module 1140 is used to feed back the updated vehicle status information to the status perception unit of the traffic simulation environment based on the update operation. The generation module 1150 is used to generate observation information of the traffic simulation environment by the state perception unit. The observation information is used to provide perceptible traffic state data to the reinforcement learning agent, which is used to learn traffic control decisions based on the traffic simulation environment.

[0281] In summary, the apparatus provided in this application maintains alternating vehicle queues for the target lane according to its motion state, determines the target updated vehicle based on the lane's traffic status, and skips non-passable stationary vehicles in the stop queue. This eliminates invalid calculations in vehicle state updates, significantly reduces the computational cost of vehicle state updates in traffic simulation, improves the efficiency of vehicle state updates in simulation frames, and adapts to the high-frequency vehicle state update requirements of traffic simulation.

[0282] It should be noted that the vehicle status management device based on the simulation environment provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0283] Figure 13 This illustration shows a structural block diagram of a terminal 1300 provided in an exemplary embodiment of this application. The terminal 1300 may be a smartphone, tablet computer, MP3 player, MP4 player, laptop computer, or desktop computer. The terminal 1300 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0284] Typically, terminal 1300 includes a processor 1301 and a memory 1302.

[0285] Processor 1301 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1301 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1301 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1301 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1301 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0286] The memory 1302 may include one or more computer-readable storage media, which may be non-transitory. The memory 1302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1302 is used to store at least one instruction, which is executed by the processor 1301 to implement the vehicle state management method based on a simulation environment provided in the method embodiments of this application.

[0287] In some embodiments, the terminal 1300 also includes other components 1303, as those skilled in the art will understand. Figure 13 The structure shown does not constitute a limitation on terminal 1300 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0288] Embodiments of this application also provide a computer device that can be implemented as follows: Figure 1 The terminal or server shown. The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the vehicle state management method based on the simulation environment provided in the above-described method embodiments.

[0289] The embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the vehicle state management method based on the simulation environment provided in the above-described method embodiments.

[0290] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the vehicle state management method based on a simulation environment provided in the above-described method embodiments.

[0291] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0292] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0293] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A vehicle state management method based on a simulation environment, characterized in that, The method includes: Acquire a traffic simulation environment, which includes the target lane; For the target lane maintenance, there are multiple vehicle queues connected before and after, wherein the vehicles in each vehicle queue correspond to the same type of movement state, and the movement states of two adjacent vehicle queues are different, including forward state and stopped state. Based on the traffic status corresponding to the target lane, target vehicles that meet the update conditions are determined from the multiple vehicle queues. This includes obtaining the phase type of the target traffic light corresponding to the target lane; determining the traffic status corresponding to the target lane based on the phase type; and filtering the target vehicles that meet the update conditions based on the traffic status, combined with the arrangement order of the vehicle queues and the current movement status. An update operation is performed on the target vehicle. The update operation is used to update at least one of the target vehicle's position, speed, and motion state. Specifically, for a moving queue consisting of vehicles in a forward state, the update operation is performed on all vehicles in the queue sequentially. For a stopped queue consisting of vehicles in a stopped state, stationary vehicles in the stopped queue are skipped except for vehicles that can pass. A vehicle that can pass refers to a vehicle in the stopped queue that has the conditions to move forward. It is usually the head vehicle of the stopped queue. The determination criteria is that the distance between the head vehicle and the tail vehicle of the preceding vehicle queue meets the preset passage conditions. It is the only vehicle in the stopped queue that needs to have the update operation performed.

2. The method according to claim 1, characterized in that, The step of filtering target vehicles that meet the update conditions based on the traffic status, combined with the vehicle queue order and current movement status, includes: When the target lane is passable, the movement state of the first queue in the target lane is obtained, where the first queue is the first queue of vehicles arranged in the target lane according to the arrangement order. Based on the current motion state of the first queue, the target vehicle that meets the update conditions is determined.

3. The method according to claim 2, characterized in that, The step of determining the target vehicle that meets the update conditions based on the current motion state of the first queue includes: If the first queue belongs to the stopped queue, a moving queue is created as a new first queue. The moving queue refers to the queue of vehicles in the forward state. The new first queue includes the first vehicle in the original first queue. Vehicle queues other than the new first queue will be designated as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the new first queue and the subsequent target vehicles are identified as the target vehicles.

4. The method according to claim 2, characterized in that, The step of determining the target vehicle that meets the update conditions based on the current motion state of the first queue includes: If the first queue is a moving queue, then the vehicle queues other than the first queue are designated as subsequent queues, where the moving queue refers to the vehicle queues in the forward state. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first queue and the subsequent target vehicles are identified as the target vehicles.

5. The method according to claim 1, characterized in that, The step of filtering target vehicles that meet the update conditions based on the traffic status, combined with the vehicle queue order and current movement status, includes: When the target lane is impassable, the current movement state of the first queue in the target lane is obtained, where the first queue is the first queue of vehicles arranged in the target lane according to the arrangement order. Based on at least one of the current movement state of the first queue and the passable vehicle situation in the first queue, the target vehicle that meets the update conditions is determined.

6. The method according to claim 5, characterized in that, The step of determining the target vehicle that meets the update conditions based on at least one of the current movement state of the first queue and the passable vehicle situation in the first queue includes: If the first queue belongs to the stop queue, skip the first queue; The remaining vehicle queues, starting from the second vehicle queue after the first queue, are designated as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

7. The method according to claim 5, characterized in that, The step of determining the target vehicle that meets the update conditions based on at least one of the current movement state of the first queue and the passable vehicle situation in the first queue includes: If the first queue is a moving queue, obtain the remaining distance between the first vehicle in the first queue and the stop line; The number of passable vehicles in the first queue is determined based on the remaining distance. The target vehicle that meets the update conditions is determined based on the passable vehicle situation.

8. The method according to claim 7, characterized in that, The step of determining the target vehicle that meets the update conditions based on the passable vehicle situation includes: If there are passable vehicles in the first queue, the remaining vehicle queue starting from the second vehicle queue after the first queue shall be used as the subsequent queue. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

9. The method according to claim 7, characterized in that, The step of determining the target vehicle that meets the update conditions based on the passable vehicle situation includes: If there are no passable vehicles in the first queue, a stop queue is created as a new first queue, and the new first queue includes the first vehicle in the original first queue. The remaining vehicle queues, starting from the second vehicle queue after the new first queue, are used as subsequent queues. Based on the current motion state of the subsequent queue, determine the subsequent target vehicles that meet the update conditions; The vehicles in the first vehicle queue after the first queue and the subsequent target vehicles are identified as the target vehicles.

10. The method according to claim 3, characterized in that, The step of determining subsequent target vehicles that meet the update conditions based on the current motion state of the subsequent queue includes: The stopping queue and the moving queue are determined from the subsequent queues based on the current motion state; Determine the passable vehicles in the stop queue of the subsequent queue; The passable vehicles in the stopped queue and all vehicles in the moving queue are identified as the subsequent target vehicles.

11. The method according to claim 10, characterized in that, Determining the passable vehicles in the stopped queue of the subsequent queue includes: Obtain the distance between the first vehicle in the stopped queue and the last vehicle in the previous queue; If the distance between the vehicles meets the passage conditions, the leading vehicle is determined to be the passable vehicle, and the leading vehicle is moved to the tail of the previous vehicle queue, until the distance between the leading vehicle in the stopped queue does not meet the passage conditions.

12. The method according to claim 1, characterized in that, The method further includes: Each lane in the traffic simulation environment is divided into multiple spatial grids. Each spatial grid is a fixed area with a preset parking space capacity. Each spatial grid corresponds to a vehicle information index, which is used to indicate the vehicles accommodated in the spatial grid. The vehicle information index corresponding to the spatial grid is updated based on the update operation; When a first vehicle in the traffic simulation environment has a lane-changing requirement, the adjacent spatial grids that meet the adjacent detection conditions with the first vehicle are determined. Based on the vehicle information index corresponding to the adjacent spatial grid, the lane change status of the first vehicle is determined.

13. The method according to claim 1, characterized in that, After performing the update operation on the target vehicle, the method further includes: Based on the update operation, the updated vehicle status information is fed back to the status perception unit of the traffic simulation environment. The state perception unit generates observation information about the traffic simulation environment. This observation information is used to provide perceptible traffic state data to the reinforcement learning agent, which then learns traffic control decisions based on the traffic simulation environment.

14. A vehicle state management device based on a simulation environment, characterized in that, The device includes: An acquisition module is used to acquire a traffic simulation environment, the traffic simulation environment including the target lane; The management module is used to maintain multiple vehicle queues connected to each other for the target lane. The vehicles in each vehicle queue correspond to the same type of movement state, and the movement states of two adjacent vehicle queues are different. The movement states include forward state and stopped state. The determination module is used to determine target vehicles that meet the update conditions from the plurality of vehicle queues based on the traffic status corresponding to the target lane, including obtaining the phase type of the target traffic light corresponding to the target lane; determining the traffic status corresponding to the target lane based on the phase type; and filtering the target vehicles that meet the update conditions based on the traffic status, combined with the arrangement order of the vehicle queues and the current movement status. The management module is also used to perform an update operation on the target vehicle. The update operation is used to update at least one of the target vehicle's position, speed, and motion state. Specifically, for a moving queue consisting of vehicles in a forward state, the update operation is performed sequentially on all vehicles in the queue. For a stopped queue consisting of vehicles in a stopped state, stationary vehicles in the stopped queue, except for vehicles that can pass, are skipped. A vehicle that can pass refers to a vehicle in the stopped queue that has the conditions to move forward. It is usually the head vehicle of the stopped queue, and its determination is based on the fact that its distance from the tail vehicle of the preceding vehicle queue meets the preset passage conditions. It is the only vehicle in the stopped queue that needs to have the update operation performed.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the vehicle state management method based on a simulation environment as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer program, which is loaded and executed by a processor to implement the vehicle state management method based on a simulation environment as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the vehicle state management method based on a simulation environment as described in any one of claims 1 to 13.

Images