A non-line-of-sight traffic participant perception method based on multi-source topology alignment

Abstract

The application discloses a non-line-of-sight traffic participant perception method based on multi-source topology alignment, wherein mobile terminals in a region periodically collect motion states and coordinate information of the mobile terminals, and perform distributed safety beacon broadcasting; adjacent mobile terminals in the region communicate with each other, and the mobile terminals can integrate safety beacon packets with each other to form a 'cooperative perception list'; in a vehicle alignment and identification stage, the vehicle side communicates with the mobile terminals in the region to obtain the 'cooperative perception list'; meanwhile, the vehicle side uses vehicle-mounted sensors to obtain feature points and three-dimensional space coordinates of all pedestrians in a surrounding visual visible range; the latitude and longitude coordinates or relative positions in the communication list are subjected to space transformation with the relative coordinates in the vehicle-mounted sensor perception list, and are aligned into a unified local coordinate system of the vehicle; and a virtual-real list alignment algorithm is used to detect whether there is a 'hidden target'.

Description

Technical Field

[0001] This invention relates to the fields of intelligent driving, vehicle-to-everything (V2X) and active safety technologies. Specifically, it relates to a method for predicting and locating hidden targets in advance in occluded scenarios where the sensor line of sight is obstructed (such as "ghost peek" scenarios), by aligning mobile terminal network collaboration with vehicle-mounted perception data. Background Technology

[0002] With the development of autonomous driving technology, perception systems relying on onboard sensors (such as LiDAR, cameras, and millimeter-wave radar) can now accurately identify obstacles in open environments. However, in urban traffic environments, there are numerous blind spots created by large vehicles (such as buses), corner buildings, and green belts.

[0003] Current solutions primarily rely on onboard sensors for automatic emergency braking (AEB). However, due to physical obstructions, by the time the sensors detect a target, the vehicle is often already within the collision limit distance, resulting in insufficient braking. Although existing technologies have proposed vehicle-to-pedestrian (V2P) direct communication warnings, the limited transmission power of mobile devices carried by people and the severe attenuation and multipath effects of electromagnetic waves when passing through concrete walls or large metal vehicles mean that vehicles often cannot reliably receive direct broadcast signals from pedestrians in the "deepest blind spots" where warnings are most needed.

[0004] In addition, existing V2X warning systems have difficulty distinguishing between pedestrians and passengers in vehicles, which can easily lead to false alarms and cause vehicles to frequently trigger brakes unnecessarily, affecting traffic efficiency.

[0005] Therefore, how to utilize the distributed collaborative capabilities of pedestrian-accompanied devices and accurately identify high-risk targets in visual blind spots through logical inference at the vehicle end is a technical challenge that urgently needs to be solved in the field of active safety for intelligent driving. Summary of the Invention

[0006] This invention aims to address the problems in existing technologies, such as the perception lag of single-vehicle perception systems in occluded scenarios, and the ease with which single-point V2P communication loses signals and fails to accurately identify hidden targets in visual blind spots under NLOS environments. Therefore, this invention discloses a non-line-of-sight traffic participant perception method based on multi-source topology alignment. This method significantly improves the ability of autonomous vehicles to identify hidden targets in occluded blind spot scenarios by logically aligning the results of short-range collaborative networking (mesh network) between mobile terminals and the physical perception results of onboard sensors. It enables the anticipation of pedestrian intentions behind obstructions and facilitates defensive driving.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A non-line-of-sight traffic participant perception method based on multi-source topology alignment, comprising: a mobile terminal collaboration stage and a vehicle alignment recognition stage;

[0009] During the mobile terminal collaboration phase, mobile terminals within the area periodically collect their own motion status and coordinate information and broadcast distributed safety beacons. Adjacent mobile terminals within the area communicate with each other, and any mobile terminal integrates its own information with the safety broadcast beacon packets of adjacent mobile terminals to form a "collaboration awareness list" and broadcasts it outwards in a unified manner.

[0010] During the vehicle-side alignment and recognition phase, the vehicle communicates with mobile terminals within the area to obtain a "cooperative perception list." Simultaneously, the vehicle uses onboard sensors to acquire feature points and three-dimensional spatial coordinates of all pedestrians within the visually visible range. The latitude and longitude coordinates or relative positions in the communication list are spatially transformed with the relative coordinates in the onboard sensor perception list to align with the vehicle's unified local coordinate system. The virtual-real list alignment algorithm is then used to detect the presence of "hidden targets."

[0011] Furthermore, when a terminal node A detects that it is in a weak signal environment and determines that the signal-to-noise ratio of its direct communication link with the vehicle or infrastructure is lower than a preset threshold, it will automatically request to establish a relay link with the nearby mobile terminal node B. As a relay node, node B will aggregate the safety beacon information received from node A with its own information to form a "cooperative perception list" containing multiple target information.

[0012] Furthermore, a first-order kinematic model is used to predict and compensate for the location of the mobile terminal ID.

[0013] Furthermore, the hidden target state identified using the virtual-real list alignment algorithm is analyzed in the following steps:

[0014] S1. Visible Target Verification: If the coordinates of a certain terminal ID coincide with the coordinates of a physical entity sensed by the sensor within a preset error threshold δ, then the target is determined to be a "visible target" and the normal tracking strategy is executed.

[0015] S2. Hidden Target Locking: If a terminal ID exists in the communication list, but no physical entity is detected in the corresponding spatial coordinate area of ​​the sensor list, the system determines that the target is a "hidden target", that is, a potential "ghost peek" object in the visual blind spot.

[0016] Furthermore, a probabilistic discrimination model for pedestrians and passengers in vehicles is established based on a Bayesian inference classifier. This model is used to determine the identified hidden targets, avoiding interference from passengers in the vehicle. The probabilistic discrimination model is denoted as:

[0017]

[0018] Where H represents pedestrians and M represents motor vehicle passengers. Likelihood estimation is performed based on a typical pedestrian gait frequency motion model. Let be the prior probability that the target in the scene is a pedestrian. Let represent the prior probability that the target in the scenario is a passenger in a motor vehicle. To determine the conditional probability of observing a feature vector F, assuming the target is identified as a pedestrian. Given the observed feature vector F, the posterior probability that the concealed target is ultimately identified as a pedestrian.

[0019] Furthermore, a dynamically evolving risk potential field is generated for the identified concealed target. , denoted as:

[0020]

[0021] Where x and y are the horizontal and vertical grid coordinates in the vehicle's local cost map coordinate system, respectively; xm and ym are the predicted center position coordinates of the hidden target in the vehicle's local coordinate system, respectively; and σx and σy are the position uncertainties (standard deviations) of the hidden target in the x and y directions, respectively. This standard deviation is determined by the positioning error matrix of the mobile terminal and the delay error of multi-hop communication; the higher the uncertainty, the larger the coverage area of ​​the potential field.

[0022] Furthermore, based on the identified concealed target's state and its movement intentions, the autonomous vehicle automatically triggers defensive braking or evasive decisions, including:

[0023] Pre-charge pressure: Controls the braking system to enter a pre-pressurized state, shortening the braking response time;

[0024] Decelerate and avoid: Actively reduce the vehicle's target cruising speed and control the vehicle to veer laterally away from the blind spot;

[0025] Emergency braking: If a target enters the vehicle's protection radius in a very short time, the system triggers an emergency braking command to bring the vehicle to a stop before a collision occurs.

[0026] Furthermore, the motion state information includes, but is not limited to, geographic location coordinates, movement speed vector, acceleration, and a unique temporary anonymous ID. The device encapsulates this data into a secure broadcast beacon packet containing a timestamp and a unique temporary identifier for subsequent identity alignment.

[0027] Furthermore, the vehicle-side communication unit obtains in real time a list of cooperative awareness data sent by all surrounding mobile terminals via direct broadcast or Mesh relay through the V2X receiving link.

[0028] Furthermore, a deep learning target detection algorithm is used to identify all dynamic targets within the field of view and obtain their three-dimensional spatial coordinates relative to the vehicle's center point.

[0029] Beneficial effects

[0030] Compared with existing technologies, this invention achieves the following significant beneficial effects through its innovative technical solution:

[0031] 1. Breaking the Limits of Physical Perception: This invention establishes a collaborative grid among mobile terminals, enabling traffic participants in visual blind spots and whose signals are physically blocked to transmit their motion vectors to vehicles via relay. This effectively solves the perception lag problem of single-vehicle sensors when facing large vehicles or building obstructions, achieving early warning capabilities beyond line-of-sight.

[0032] 2. Reduced False Alarm Rate: This invention, through in-depth analysis of the motion trajectory characteristics of mobile terminals, can effectively identify and eliminate non-dangerous targets inside public transportation vehicles or high-speed motor vehicles. This mechanism avoids the problem of frequent false alarms triggered by traditional radio warning systems at busy traffic intersections, ensuring the smoothness of decision-making in autonomous driving systems.

[0033] 3. Enhanced Localization Robustness: This invention employs a virtual-real list alignment algorithm. By performing a difference operation between the communication list and the sensor list, it can accurately locate concealed targets hidden behind obstructions. Even if the vehicle's sensors cannot capture any image or point cloud of the target, the system can still reconstruct the target's dynamic distribution in a local coordinate system, providing reliable data support for the vehicle's defensive driving strategies.

[0034] 4. Optimized Response Speed: This invention utilizes short-range direct communication technology to construct a collaborative network, eliminating the need for relays via public base stations and significantly reducing end-to-end communication latency. Combined with a millisecond-level alignment algorithm on the vehicle side, it ensures sufficient braking redundancy distance for vehicles before the risk of a "ghost pedestrian" incident occurs. Attached Figure Description

[0035] Figure 1 is a schematic diagram of the overall architecture of a sensing system based on multi-source topology alignment provided by the present invention.

[0036] Figure 2 is a schematic diagram of the algorithm flow for vehicle-side execution of virtual-real list alignment and hidden target recognition in an embodiment of the present invention.

[0037] Figure 3 is a flowchart illustrating the logic of a vehicle making an advance avoidance decision in a typical occlusion scenario according to an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the following description will be provided in conjunction with the appendix. Figure 1 , 2 The present invention will be further described in detail below with reference to sections 3 and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0039] A non-line-of-sight traffic participant perception method based on multi-source topology alignment, comprising: a mobile terminal collaboration stage and a vehicle alignment recognition stage;

[0040] During the mobile terminal collaboration phase, mobile terminals within the area periodically collect their own motion status and coordinate information and broadcast distributed safety beacons. Adjacent mobile terminals within the area communicate with each other, and any mobile terminal integrates its own information with the safety broadcast beacon packets of adjacent mobile terminals to form a "collaboration awareness list" and broadcasts it outwards in a unified manner.

[0041] More specifically, mobile terminals include, but are not limited to, smartphones, smartwatches, or various wearable electronic devices with wireless communication capabilities. When these devices are in traffic activity, they use built-in positioning chips, accelerometers, and gyroscopes to extract their current physical position and movement vector in real time.

[0042] More specifically, each mobile device within the area acts as a node, periodically collecting its own motion status and coordinate information. This motion status information includes, but is not limited to, geographic location coordinates, movement velocity vector, acceleration, and a unique temporary anonymous ID. The device encapsulates this data into a secure broadcast beacon packet containing a timestamp and a unique temporary identifier for subsequent identity alignment.

[0043] More specifically, mobile terminals can use short-range communication technologies such as UWB, BLE, or Wi-Fi Aware to search for nearby nodes, that is, continuously monitor the signal quality of surrounding neighboring nodes.

[0044] More specifically, when a terminal node A detects that it is in a weak signal environment (such as being near a large obstacle that restricts communication with a base station or vehicle), and determines that the signal-to-noise ratio of its direct communication link with the vehicle or infrastructure is below a preset threshold, it will automatically request to establish a relay link with a nearby mobile terminal node B to establish a Mesh relay link. Node B, acting as the relay node, aggregates the received security beacon information from node A with its own information to form a "cooperative awareness list" containing multiple target information. Node B then broadcasts this list externally with high transmission power. In this way, even if mobile terminal node A is in the physical shadow of the vehicle's sensors and radio receiver, its critical location information can still be relayed to the target vehicle by node B, thus ensuring that target information in signal-blocked areas can be delivered to the target vehicle via a multi-hop method.

[0045] During the vehicle-side alignment and recognition phase, the vehicle communicates with mobile terminals within the area to obtain a "cooperative perception list." Simultaneously, the vehicle uses onboard sensors to acquire feature points and three-dimensional spatial coordinates of all pedestrians within the visually visible range. The latitude and longitude coordinates or relative positions in the communication list are spatially transformed with the relative coordinates in the onboard sensor perception list to align with the vehicle's unified local coordinate system. The virtual-real list alignment algorithm is then used to detect the presence of "hidden targets."

[0046] More specifically, the vehicle-mounted wireless communication unit continuously monitors broadcast signals sent by surrounding mobile terminal nodes. Upon receiving the cooperative sensing list forwarded through the cooperative network, the system extracts multiple mobile terminal IDs contained in the list, along with their corresponding latitude and longitude coordinates, speed, orientation, and timestamp information. This set is defined as the communication sensing list.

[0047] More specifically, the vehicle-side communication unit acquires in real-time a cooperative awareness list (external data packets) sent by all surrounding mobile terminals via direct broadcast or Mesh relay through a V2X receiving link. The system parses, deduplicates, and formats the received multiple cooperative awareness lists to construct a unified communication awareness list within the vehicle. More specifically, this communication awareness list contains unique identifiers, real-time motion parameters (speed, acceleration, heading angle), and spatial coordinates of all active traffic participants within the area.

[0048] More specifically, the vehicle utilizes onboard sensors (such as LiDAR, monocular or binocular cameras, and millimeter-wave radar) for real-time physical perception. Specifically, the system uses the deep learning object detection algorithm and multi-sensor data fusion module of the onboard computing platform to extract the bounding boxes and feature points of all pedestrians and obstacles within the visually visible range. It then uses the sensor extrinsic parameter matrix to transform these into point cloud clusters or entity coordinate sets in a three-dimensional spatial coordinate system with the vehicle as the origin, forming a physical perception list.

[0049] More specifically, a deep learning object detection algorithm is used to identify all dynamic targets within the field of view and obtain their three-dimensional spatial coordinates relative to the vehicle's center point.

[0050] More specifically, due to transmission losses and processing latency caused by mobile terminals forwarding data through the mesh network, the directly obtained coordinates are delayed. The system uses a first-order kinematic model to predict and compensate for the location of the mobile terminal ID.

[0051] set up The original coordinates sent by the i-th mobile terminal at time i are The instantaneous velocity vector it reports is ; They are The coordinates of the mobile terminal on the x-axis, y-axis, and z-axis at any given time.

[0052] When the vehicle is Calculate the total latency when processing this data. .

[0053] Compensated terminal prediction location The calculation formula is as follows:

[0054]

[0055] in, The compensation acceleration vector is fed back from the accelerometer built into the mobile terminal.

[0056] Then, using the vehicle's current global navigation coordinates First, it is converted into global planar coordinates using the Gauss-Krüger projection. Similarly, the predicted and compensated terminal global navigation coordinates are also projected and converted into global plane coordinates. Finally, considering the vehicle's current heading angle The terminal coordinates are transformed to a local Cartesian coordinate system with the vehicle's center of mass as the origin. The calculation formula is as follows:

[0057]

[0058] in, The vehicle's current global navigation longitude coordinates. The vehicle's current global navigation latitude and longitude coordinates. The horizontal and vertical coordinates of the vehicle after Gauss-Kruger projection are given in the global plane coordinate system (world coordinate system). The horizontal and vertical coordinates of the mobile terminal are projected onto the global plane coordinate system. To finally obtain the relative horizontal and vertical coordinates of the mobile terminal in a local Cartesian coordinate system (i.e., the vehicle coordinate system) with the vehicle's center of mass as the origin and the vehicle's direction of travel as the front, This is the vehicle's current heading angle.

[0059] More specifically, the hidden target state identified using the virtual-real list alignment algorithm involves the following steps:

[0060] S1. Visible Target Verification: If the coordinates of a certain terminal ID coincide with the coordinates of a physical entity sensed by the sensor within a preset error threshold δ, then the target is determined to be a "visible target" and the normal tracking strategy is executed.

[0061] S2. Hidden Target Locking (Ghost Peek Identification): If a terminal ID exists in the communication list, but no physical entity is detected in the corresponding spatial coordinate area of ​​the sensor list (i.e., the sensor field of view is empty, but the communication signal is present), the system determines that the target is a "hidden target", that is, a potential "ghost peek" object in the visual blind spot.

[0062] More preferably, to address the issue of interference from in-vehicle passengers, a probabilistic discrimination model for pedestrians and in-vehicle passengers is established based on a Bayesian inference classifier. Let event H represent a pedestrian and event M represent a motor vehicle passenger. The system observes feature vector F (including speed smoothness, trajectory-to-lane overlap, and multipath index of surrounding wireless channels).

[0063] Its posterior probability is calculated as follows:

[0064]

[0065] Where H represents pedestrians and M represents motor vehicle passengers. Likelihood estimation is performed based on a typical pedestrian gait frequency motion model. Let be the prior probability that the target in the scene is a pedestrian. Let represent the prior probability that the target in the scenario is a passenger in a motor vehicle. To determine the conditional probability of observing a feature vector F, assuming the target is identified as a pedestrian. Given the observed feature vector F, this is the posterior probability that the concealed target is ultimately identified as a pedestrian.

[0066] More preferably, for a locked, concealed target, the system does not simply set up hard obstacles at the planning layer, but instead generates a dynamically evolving risk potential field. :

[0067]

[0068] The peak potential field A is inversely proportional to the time-to-collision (TCC) of the hidden target, and the variance σ increases with the positioning uncertainty. The vehicle's local path planner automatically decelerates and avoids lateral movements in advance by solving for the minimum cost path under this potential field. Thus, before the pedestrian is visually detected, the vehicle's state has been adjusted to within the safe envelope by the control algorithm.

[0069] More preferably, this embodiment aims to automatically trigger defensive braking or avoidance decisions of the autonomous vehicle based on the identified concealed target's state and its movement intention. Specifically:

[0070] After sensing virtual dynamic obstacles, the trajectory planning module of the autonomous vehicle recalculates the optimal path. If it detects that a hidden target is moving from the blind spot towards the lane centerline, which matches the characteristics of a "ghost peek" (a person suddenly appearing out of nowhere), the system will immediately perform the following actions:

[0071] Pre-charge pressure: Controls the braking system to enter a pre-pressurized state, shortening the braking response time.

[0072] Deceleration and avoidance: Actively reduce the vehicle's target cruising speed and control the vehicle to veer laterally away from the blind spot.

[0073] Emergency braking: If a target enters the vehicle's protection radius in a very short time, the system triggers an emergency braking command to bring the vehicle to a stop before a collision occurs.

[0074] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A non-line-of-sight traffic participant perception method based on multi-source topology alignment, characterized in that, The method includes: a mobile terminal collaboration stage and a vehicle-side alignment and recognition stage; During the mobile terminal collaboration phase, mobile terminals within the area periodically collect their own motion status and coordinate information and broadcast distributed safety beacons. Adjacent mobile terminals within the area communicate with each other, and any mobile terminal integrates its own information with the safety broadcast beacon packets of adjacent mobile terminals to form a "collaboration awareness list" and broadcasts it outwards in a unified manner. During the vehicle-side alignment and recognition phase, the vehicle communicates with mobile terminals within the area to obtain a "cooperative perception list." Simultaneously, the vehicle uses onboard sensors to acquire feature points and three-dimensional spatial coordinates of all pedestrians within the visually visible range. The latitude and longitude coordinates or relative positions in the communication list are spatially transformed with the relative coordinates in the onboard sensor perception list to align with the vehicle's unified local coordinate system. The virtual-real list alignment algorithm is then used to detect the presence of "hidden targets." 2. The non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 1, characterized in that, When a terminal node A detects that it is in a weak signal environment and determines that the signal-to-noise ratio of its direct communication link with the vehicle or infrastructure is lower than a preset threshold, it will automatically request to establish a relay link with the nearby mobile terminal node B. Node B, as the relay node, will aggregate the safety beacon information received from node A with its own information to form a "cooperative awareness list" containing multiple target information.

3. The non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 1, characterized in that, A first-order kinematic model is used to predict and compensate for the location of the mobile terminal ID.

4. The non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 1, characterized in that, The specific steps for identifying the hidden target state using the virtual-real list alignment algorithm are as follows: S1. Visible Target Verification: If the coordinates of a certain terminal ID coincide with the coordinates of a physical entity sensed by the sensor within a preset error threshold δ, then the target is determined to be a "visible target" and the normal tracking strategy is executed. S2. Hidden Target Locking: If a terminal ID exists in the communication list, but no physical entity is detected in the corresponding spatial coordinate area of ​​the sensor list, the system determines that the target is a "hidden target", that is, a potential "ghost peek" object in the visual blind spot.

5. The non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 1, characterized in that, A probabilistic discrimination model for pedestrians and passengers in vehicles is established based on a classifier using Bayesian inference. This probabilistic discrimination model is used to determine the locked hidden targets and avoid interference from passengers in vehicles. The probabilistic discriminant model is denoted as: Where H represents pedestrians and M represents motor vehicle passengers. Likelihood estimation is performed based on a typical pedestrian gait frequency motion model. Let be the prior probability that the target in the scene is a pedestrian. Let represent the prior probability that the target in the scenario is a passenger in a motor vehicle. To determine the conditional probability of observing a feature vector F, assuming the target is identified as a pedestrian. Given the observed feature vector F, this is the posterior probability that the concealed target is ultimately identified as a pedestrian.

6. The non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 5, characterized in that, For the locked, concealed target, a dynamically evolving risk potential field is generated. , denoted as: Where x and y are the horizontal and vertical grid coordinates in the vehicle's local cost map coordinate system, respectively; xm and ym are the predicted center position coordinates of the hidden target in the vehicle's local coordinate system, respectively; and σx and σy are the position uncertainties of the hidden target in the x and y directions, respectively. This standard deviation is determined by the positioning error matrix of the mobile terminal and the delay error of multi-hop communication; the higher the uncertainty, the larger the coverage area of ​​the potential field.

7. A non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 6, characterized in that, Based on the identified state of the concealed target and its movement intentions. Automatically triggering defensive braking or evasive decisions for autonomous vehicles, including: Pre-charge pressure: Controls the braking system to enter a pre-pressurized state, shortening the braking response time; Decelerate and avoid: Actively reduce the vehicle's target cruising speed and control the vehicle to veer laterally away from the blind spot; Emergency braking: If a target enters the vehicle's protection radius in a very short time, the system triggers an emergency braking command to bring the vehicle to a stop before a collision occurs.

8. The non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 1, characterized in that, The motion state information includes, but is not limited to, geographic location coordinates, movement speed vector, acceleration, and a unique temporary anonymous ID. The device encapsulates this data into a secure broadcast beacon packet containing a timestamp and a unique temporary identifier for subsequent identity alignment.

9. A non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 1, characterized in that, The vehicle-side communication unit obtains a list of cooperative awareness data sent in real time by all surrounding mobile terminals via direct broadcast or Mesh relay through the V2X receiving link.

10. A non-line-of-sight traffic participant perception method based on multi-source topology alignment according to claim 1, characterized in that, The deep learning target detection algorithm identifies all dynamic targets within the field of view and obtains their three-dimensional spatial coordinates relative to the vehicle's center point.

Images