A traffic situation awareness method and system

Abstract

The application relates to the technical field of mobile robots, in particular to a traffic situation perception method and system, which comprises the following steps: in the process of a body-side roadside robot inspection, real-time acquisition of chassis encoder pulse data and multi-source perception data, construction of a traffic semantic vector based on the multi-source perception data and a global three-dimensional Gaussian road map; local incremental reconstruction updating of the global three-dimensional Gaussian road map based on the multi-source perception data, the chassis encoder pulse data and the traffic semantic vector, construction of a local perception blind area heat map based on the updated global three-dimensional Gaussian road map; construction of a semantic cost map based on the local perception blind area heat map, the multi-source perception data and the updated global three-dimensional Gaussian road map; construction of a local inspection path based on the local semantic cost map, movement along the local inspection path, and construction of a traffic situation perception result. The application improves the safety of robot autonomous movement and can output fine road perception information.

Description

Technical Field

[0001] This application relates to the field of mobile robot technology, and in particular to a traffic situation perception method and system. Background Technology

[0002] With the rapid development of embodied intelligence and smart transportation technologies, roadside perception has become an important technical means for urban traffic management, traffic incident monitoring, and traffic flow analysis. Traditional fixed roadside perception mainly uses fixed cameras and fixed LiDAR equipment, which have limited coverage and fixed deployment points, and cannot flexibly adapt to dynamic traffic scenarios such as intersection congestion, temporary construction sections, and key road section inspections. Although some mobile inspection robots are used for road inspection, they mostly use single-sensor perception, have insufficient positioning accuracy, low mapping efficiency, and weak navigation and obstacle avoidance capabilities, making it difficult to operate safely and stably in complex traffic and non-motorized vehicle mixed environments. Summary of the Invention

[0003] (a) Technical problems to be solved In view of the above-mentioned shortcomings and deficiencies of the prior art, this application provides a traffic situation perception method and system, which solves the technical problems that traditional fixed roadside perception cannot flexibly adapt to dynamic traffic scenarios such as intersection congestion, temporary construction sections, and key road section inspections, as well as the technical problems that traditional mobile inspection robots have difficulty operating safely and stably in complex traffic flow and non-motorized vehicle mixed traffic environments.

[0004] (II) Technical Solution To achieve the above objectives, the main technical solutions adopted in this application include: In a first aspect, embodiments of this application provide a traffic situation perception method, the method being based on an embodied roadside robot, the method comprising: During the inspection of the target area according to the pre-set inspection instructions, the embodied roadside robot acquires chassis encoder pulse data and multi-source perception data within the current perception range in real time, and constructs traffic semantic vectors based on the multi-source perception data and the pre-deployed global three-dimensional Gaussian road map. The embodied roadside robot performs local incremental reconstruction and updates of the global 3D Gaussian road map based on multi-source perception data, chassis encoder pulse data, and traffic semantic vectors. Based on the updated global 3D Gaussian road map, it constructs a local perception blind zone heat map. The local perception blind zone heat map is a gridded occlusion index distribution map within the current perception range of the embodied roadside robot. Based on local perception blind spot heatmaps, multi-source perception data, and updated global 3D Gaussian road maps, the embodied roadside robot constructs a local semantic cost map to quantify the travel cost of the embodied roadside robot. The embodied roadside robot constructs a local inspection path to guide its movement based on a local semantic cost map and a pre-set path planning algorithm, and obtains traffic situation perception results based on multi-source perception data and local perception blind spot heat maps.

[0005] Optionally, the multi-source sensing data includes: depth images, 3D laser point clouds, and satellite positioning data; The embodied roadside robot constructs traffic semantic vectors based on depth images, 3D laser point clouds, satellite positioning data, and a pre-deployed global 3D Gaussian road map, including: The embodied roadside robot performs pixel-by-pixel classification of depth images based on a pre-set semantic segmentation network, obtaining binary mask data corresponding to each pixel in the depth image. Based on the binary mask data corresponding to all pixels in the depth image, a basic occlusion rate is constructed for the depth image. The basic occlusion rate is then corrected based on 3D laser point cloud to obtain the actual occlusion rate. The binary mask data is used to distinguish whether a pixel is an occluded area or an effective road surface area. The basic occlusion rate is the ratio of the number of pixels in the depth image whose binary mask data indicates an occluded area to the total number of pixels. The embodied roadside robot inputs depth images into a pre-deployed target detection model to detect targets, obtains all detected targets in the depth image and the confidence score corresponding to each detected target in the depth image, and obtains the average confidence score based on the confidence score corresponding to each detected target in the depth image; the detected targets include motor vehicles and non-motor vehicles; The embodied roadside robot obtains the effective perceived road area within the current perception range based on satellite positioning data, depth images, 3D laser point clouds, and a pre-deployed global 3D Gaussian road map. Based on the effective perceived road area within the current perception range and all detected targets in the depth image, it obtains the corresponding traffic flow density. The traffic flow density is the ratio of the number of detected targets in the depth image to the effective perceived road area within the current perception range. The traffic semantic vector includes the actual occlusion rate, traffic density, and average confidence level.

[0006] Optionally, the multi-source sensing data further includes IMU pose data; The embodied roadside robot performs local incremental reconstruction and updates of the global 3D Gaussian road map based on multi-source perception data, chassis encoder pulse data, and traffic semantic vectors, including: The embodied roadside robot constructs coordinate transformation matrices based on chassis encoder pulse data and IMU pose data to map depth images and 3D laser point clouds to a global 3D Gaussian road map, respectively. The embodied roadside robot performs global coordinate mapping on satellite positioning data, depth images, and 3D laser point clouds based on a coordinate transformation matrix, so as to align the satellite positioning data, depth images, and 3D laser point clouds with the global three-dimensional Gaussian road map in data space. The embodied roadside robot performs local incremental reconstruction and updates of the global 3D Gaussian road map based on traffic semantic vectors, as well as satellite positioning data, depth images, and 3D laser point clouds after data spatial alignment.

[0007] Optionally, the chassis encoder pulse data includes chassis linear velocity and chassis angular velocity; The embodied roadside robot constructs coordinate transformation matrices based on chassis encoder pulse data and IMU pose data to map depth images and 3D laser point clouds to a global 3D Gaussian road map, including: The embodied roadside robot obtains its original displacement increment based on the chassis linear velocity and chassis angular velocity, as well as the pre-set chassis encoder sampling period and the pre-set global heading angle of the previous moment; and constructs the inertial displacement increment based on the IMU pose data. The embodied roadside robot performs a weighted summation of the original displacement increment and the inertial displacement increment to obtain the optimal displacement increment, and obtains the optimal global pose at the current moment based on the pre-set global pose and optimal displacement increment of the previous moment. Based on the optimal global pose at the current moment, the embodied roadside robot constructs coordinate transformation matrices to map the depth image and 3D laser point cloud to the global 3D Gaussian road map, respectively.

[0008] Optionally, the embodied roadside robot performs local incremental reconstruction and updates of a pre-deployed global 3D Gaussian road map based on traffic semantic vectors, as well as spatially aligned satellite positioning data, depth images, and 3D laser point clouds. This includes: Based on spatially aligned satellite positioning data, depth images, and 3D laser point clouds, the embodied roadside robot selects ROI areas to be incrementally updated from a pre-deployed global 3D Gaussian road map. The embodied roadside robot constructs a multimodal fusion weight based on the actual occlusion rate, traffic density, and average confidence level, and dynamically filters the spatially aligned depth image and 3D laser point cloud based on the multimodal fusion weight. The embodied roadside robot incrementally reconstructs and updates the ROI region in the global 3D Gaussian road map based on dynamically filtered depth images and 3D laser point clouds.

[0009] Optionally, based on the updated global 3D Gaussian road map, a heatmap of local perception blind spots is constructed, including: Based on the spatially aligned depth image and the binary mask data corresponding to each pixel in the depth image, the embodied roadside robot determines whether each 3D Gaussian point in the ROI region of the global 3D Gaussian road map has a depth observation binary label and whether it is an occluded region. The embodied roadside robot uses spatially aligned satellite positioning data as the center and performs uniform grid subdivision of the ROI region in the global 3D Gaussian road map according to a pre-set grid size to obtain multiple grid cells. Based on the updated global 3D Gaussian road map, it obtains the map coordinates and grid cell matrix corresponding to each grid cell. The grid cell matrix corresponding to each grid cell includes all 3D Gaussian points within the grid cell, as well as the depth observation binary label and occlusion binary label corresponding to each 3D Gaussian point within the grid cell. The embodied roadside robot constructs the obstacle space ratio and depth observation missing ratio for each grid cell based on the depth observation binary label and occlusion binary label corresponding to each three-dimensional Gaussian point in each grid cell. The embodied roadside robot constructs an occlusion index for each grid cell based on the proportion of obstacle space and the proportion of missing depth observations for each grid cell. The embodied roadside robot visualizes and renders each grid cell based on the occlusion index corresponding to each grid cell and a pre-set color rendering strategy. Based on the map coordinates corresponding to each grid cell, it integrates all visualized and rendered grid cells to obtain a heat map of local perception blind spots.

[0010] Optionally, the local sensing blind zone heatmap includes multiple grid units, and each grid unit has a corresponding occlusion index; the multi-source sensing data includes depth images; Based on local perception blind spot heatmaps, multi-source perception data, and an updated global 3D Gaussian road map, the embodied roadside robot constructs a local semantic cost map to quantify the robot's travel costs, including: The embodied roadside robot determines the access label corresponding to each grid cell based on the local perception blind zone heat map and the updated global 3D Gaussian road map, and constructs a static obstacle cost subgraph based on the access label corresponding to each grid cell. The embodied roadside robot inputs the depth image into a pre-deployed target detection model to detect targets, obtaining all detected targets in the depth image and the instantaneous velocity corresponding to each detected target; The embodied roadside robot maps each detected target in the depth image to the corresponding grid cell, and based on the instantaneous velocity of each detected target, it dilates the reachable grid of each detected target within a pre-set time window to construct a dynamic object cost subgraph. The embodied roadside robot constructs a blind spot penalty cost subgraph based on the occlusion index corresponding to each grid cell in the local perception blind spot heatmap. The embodied roadside robot linearly superimposes the static obstacle cost subgraph, the dynamic object cost subgraph, and the blind spot penalty cost subgraph based on pre-set cost subgraph weighting coefficients to obtain a local semantic cost map.

[0011] Optionally, the multi-source sensing data includes depth images; Based on multi-source sensing data and local sensing blind spot heat maps, traffic situation perception results are obtained, including: The embodied roadside robot inputs the depth image from the multi-source perception data into a pre-deployed target detection model to detect targets, and obtains the target position, target category and instantaneous velocity of each detected target within the current perception range; The embodied roadside robot obtains traffic situation perception results based on the target position, target category, and instantaneous speed of each detected target within the current perception range, as well as the heat map of local perception blind spots.

[0012] Optionally, the local sensing blind zone heat map includes multiple grid units; The method further includes: After obtaining the traffic situation perception results, the embodied roadside robot constructs a perception backflow vector based on the traffic situation perception results and the local perception blind zone heat map; wherein, the perception backflow vector is used to indicate the grid cells in the local perception blind zone heat map that need to be incrementally reconstructed and updated, and the expected depth observation density corresponding to each grid cell. The embodied roadside robot performs secondary local incremental reconstruction and update of the updated global 3D Gaussian road map based on the perception return vector, and optimizes the local inspection path based on the secondary updated global 3D Gaussian road map to guide the roadside robot's movement.

[0013] Secondly, embodiments of this application provide a traffic situation awareness system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-described traffic situation awareness method.

[0014] (III) Beneficial Effects This application discloses a traffic situation perception method that uses an embodied roadside robot as a mobile perception carrier, enabling it to flexibly enter congested intersections, temporary construction areas, and key controlled road sections for mobile inspections. By collecting chassis encoder pulse data and multi-source perception data, a global 3D Gaussian road map is locally incrementally reconstructed and updated. The updated global 3D Gaussian road map is used to generate a local perception blind spot heat map in the form of a rasterized occlusion index. By combining the local perception blind spot heat map, multi-source perception data, and the updated global 3D Gaussian road map, a local semantic cost map is constructed, enabling the quantitative calculation of traffic costs. This not only improves the safety of the robot's autonomous movement but also outputs refined road perception information. Attached Figure Description

[0015] Figure 1 This is a schematic flowchart of a traffic situation perception method provided in an embodiment of this application; Figure 2 A flowchart for constructing traffic semantic vectors provided in this application embodiment; Figure 3 A flowchart of local incremental reconstruction and update of a global 3D Gaussian road map provided in this application embodiment; Figure 4 A flowchart illustrating the construction of a local sensing blind zone heatmap provided in this application embodiment; Figure 5 A flowchart for constructing a local semantic cost map provided in this application embodiment; Figure 6 A flowchart illustrating the process of obtaining traffic situation perception results provided in this application embodiment. Detailed Implementation

[0016] To better explain and facilitate understanding of this application, the following detailed description of the application is provided in conjunction with the accompanying drawings and specific embodiments.

[0017] With the continuous development of embodied intelligence and smart transportation, roadside perception has become a key means for urban traffic management, event monitoring, and traffic flow analysis. Traditional fixed cameras and LiDAR sensors have fixed locations and limited coverage, making it difficult to meet the dynamic inspection needs of congested intersections and temporary construction zones. Most existing mobile inspection robots are equipped with only a single sensor, resulting in poor localization and mapping performance, insufficient navigation and obstacle avoidance capabilities, and an inability to operate stably in complex road conditions with mixed pedestrian and vehicle traffic.

[0018] Therefore, this application provides a traffic situation perception method that relies on an embodied roadside robot to achieve mobile patrol, effectively making up for the lack of flexibility of fixed perception equipment; and by combining encoder pulse data and multi-source perception data to perform local incremental updates on the global three-dimensional Gaussian road map, a gridded occlusion index blind spot heat map is generated, and a semantic cost map that quantifies the passage cost is further constructed, so as to ensure the safety of the robot's autonomous driving while outputting refined road traffic perception data.

[0019] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application can be understood more clearly and thoroughly, and that the scope of this application can be fully conveyed to those skilled in the art.

[0020] This application provides a traffic situation perception method based on an embodied roadside robot, as follows: Figure 1 As shown, it includes: S1. During the inspection of the target area according to the pre-set inspection instructions, the embodied roadside robot acquires the chassis encoder pulse data and multi-source perception data within the current perception range in real time, and constructs traffic semantic vectors based on the multi-source perception data and the pre-deployed global three-dimensional Gaussian road map. S2. The embodied roadside robot performs local incremental reconstruction and updates of the global 3D Gaussian road map based on multi-source perception data, chassis encoder pulse data, and traffic semantic vectors. Based on the updated global 3D Gaussian road map, it constructs a local perception blind zone heat map. The local perception blind zone heat map is a gridded occlusion index distribution map within the current perception range of the embodied roadside robot. S3. Based on the local perception blind zone heat map, multi-source perception data and the updated global 3D Gaussian road map, the embodied roadside robot constructs a local semantic cost map to quantify the passage cost of the embodied roadside robot. S4. The embodied roadside robot constructs a local inspection path to guide the movement of the roadside robot based on a local semantic cost map and a pre-set path planning algorithm, and obtains traffic situation perception results based on multi-source perception data and local perception blind spot heat map.

[0021] This application provides a traffic situation perception method that uses an embodied roadside robot as a mobile perception carrier, enabling it to flexibly enter congested intersections, temporary construction areas, and key controlled road sections for mobile inspections. By collecting chassis encoder pulse data and multi-source perception data, a global 3D Gaussian road map is locally incrementally reconstructed and updated. The updated global 3D Gaussian road map is used to generate a local perception blind spot heat map in the form of a rasterized occlusion index. The local perception blind spot heat map, multi-source perception data, and the updated global 3D Gaussian road map are combined to construct a local semantic cost map, realizing the quantitative calculation of traffic costs. This not only improves the safety of the robot's autonomous movement but also outputs refined road perception information.

[0022] Furthermore, taking the XY Road intersection in a certain district of a city as an example, it includes four entrances and four exits (3 lanes in each direction, 1 non-motorized vehicle lane, and 1 pedestrian lane), with a geometric dimension of approximately 60m × 60m and a total inspection section length of approximately 480m. The calculation time is 18:00:00 on XX / XX / 20XX (evening rush hour). The weather on that day was moderate rain, with a rainfall of R=4.2mm / h and a visibility of V=3.8km. The embodied roadside robot ROBOT-XX travels from north to south along the inspection task TASK-20XX-XXXX-NWXXXX to the vicinity of the central safety island at the intersection. Currently, there are complex conditions within the perception range, such as large buses obstructing the view (visual occlusion rate 0.37) and mixed pedestrian and vehicle traffic (23 motor vehicles, 5 non-motorized vehicles, including 2 pedestrians crossing the road).

[0023] The embodied roadside robot is equipped with a 3D LiDAR, an IMU inertial measurement unit, an RGB-D depth camera, an RTK-GNSS satellite positioning sensor, a chassis encoder, and an edge computing module.

[0024] Optionally, in one embodiment, the global three-dimensional Gaussian road map covers the XY road intersection and a 100m range at each entrance (approximately 180,000m in total). 2 The road surface area is represented using three-dimensional Gaussian points to represent its geometry and semantics; the initial number of three-dimensional Gaussian points is 4.18 × 10⁻⁶. 6 Each point contains its position (x, y, z), covariance Σ, opacity α, and view-dependent spherical harmonic coefficient SH. 16 It also includes semantic attributes such as lane number, lane line type, and pedestrian crossing sign.

[0025] Optionally, in a specific embodiment, the inspection instructions include, but are not limited to, the inspection section (XY intersection and 100m of each entrance), the driving speed limit (0.8m / s during evening peak hours), fixed monitoring points (center of the intersection, 8m behind the stop line at the four entrances, and the bus stop on the north side), and traffic incident identification rules (illegal parking / motor vehicles driving against traffic / non-motor vehicles running red lights / pedestrians crossing the road / traffic accidents).

[0026] Optionally, in one specific embodiment, the embodied roadside robot constructs traffic semantic vectors based on depth images, 3D laser point clouds, satellite positioning data, and a pre-deployed global 3D Gaussian road map, such as... Figure 2 As shown, it includes: S11. The embodied roadside robot performs pixel-by-pixel classification on the depth image based on a pre-set semantic segmentation network, obtaining the binary mask data corresponding to each pixel in the depth image. Based on the binary mask data corresponding to all pixels in the depth image, a basic occlusion rate corresponding to the depth image is constructed, and the basic occlusion rate is corrected based on 3D laser point cloud to obtain the actual occlusion rate. Among them, the binary mask data is used to distinguish whether a pixel is an occluded area or an effective road surface area, and the basic occlusion rate is the ratio of the number of pixels in the depth image whose binary mask data is an occluded area to the total number of pixels. S12. The embodied roadside robot inputs the depth image into a pre-deployed target detection model to perform target detection, obtains all detected targets in the depth image and the confidence level corresponding to each detected target in the depth image, and obtains the average confidence level based on the confidence level corresponding to each detected target in the depth image; among which, the detected targets include motor vehicles and non-motor vehicles; S13. The embodied roadside robot obtains the effective perceived road area within the current perception range based on satellite positioning data, depth images, 3D laser point clouds, and a pre-deployed global 3D Gaussian road map. Based on the effective perceived road area within the current perception range and all detected targets in the depth image, it obtains the corresponding traffic flow density. The traffic flow density is the ratio of the number of detected targets in the depth image to the effective perceived road area within the current perception range. Traffic semantic vectors include actual occlusion rate, traffic density, and average confidence level.

[0027] Specifically, the robot synchronously acquired depth images within its perception range at 18:00:00. d 3D laser point cloud P L and satellite positioning data G R The depth image is 1920×1080 pixels, containing RGB and depth channels, and the 3D laser point cloud averages 216,000 points / frame.

[0028] The embodied roadside robot will use depth images I d The input is a pre-deployed semantic segmentation network (in this embodiment, a lightweight dual-branch network based on U-Net is used, with an input of 1920×1080 and an output binary mask image) for pixel-by-pixel classification to obtain a depth image I. d The binary mask data M(u,v) ∈ {0,1} for each pixel (1 = occluded area / 0 = effective road surface area).

[0029] This period's split results: Total number of pixels = 1920 × 1080 = 2073600, number of pixels marked as occlusion (M=1) = 766232 (mainly from a 12m long large bus in front and 3 parked vehicles on the side of the road), number of pixels marked as effective road surface (M=0) = 1307368.

[0030] Therefore, the base occlusion rate is calculated as follows: S OCC,0 =766232 / 2073600=0.3696.

[0031] Further correction of the basic occlusion rate based on 3D laser point cloud: The 3D laser point cloud P... L Projecting onto the depth image coordinate system, the geometric occlusion state of each pixel (u,v) is verified (depth images are prone to false occlusion due to raindrop reflections, requiring LiDAR geometric verification). This period's LiDAR verification results: After review, the number of pixels determined to be true occlusion was 769,232 (of which 3,000 pixels that were not occluded were reclassified as occluded; the raindrop reflective area was verified by LiDAR to be the back of the large vehicle), and the actual occlusion rate was S. OCC =769232 / 2073600=0.371.

[0032] The embodied roadside robot will use depth images I d Input a pre-deployed object detection model (in this embodiment, a lightweight network based on YOLOv8-Lite with 12ms inference per frame) to perform object detection and obtain a depth image I. d All detection targets and their corresponding confidence levels. This period's detection results: Motor vehicles, number 23, with a confidence interval of 0.62–0.95 and a mean confidence level of 0.83; non-motor vehicles, number 5, with a confidence interval of 0.55–0.91 and a mean confidence level of 0.71; total, number N=28, with a confidence interval of 0.55–0.95 and a mean confidence level λ. avg It is 0.81.

[0033] Embossed roadside robots based on satellite positioning data G R Locate the current position (lon0, lat0, h0) in the global 3D Gaussian road map. Combine this with the effective depth range of the depth image (4m high-precision area, 8m medium-precision area) and the effective observation sector of the 3D laser point cloud (±60° in front). Select a set of Gaussian points from the global 3D Gaussian road map that are both effectively observed by the depth image or laser point cloud and belong to road surface features. Calculate their actual projected area within the current RGB-D field of view, i.e., the effectively perceived road area: A eff=45.16m 2 .

[0034] Therefore, the traffic flow density D is constructed. veh =28 / 45.16=0.62 vehicles / m 2 .

[0035] Therefore, the traffic semantic vector Λ=(S OCC D veh ,λ avg =(0.37,0.62,0.81).

[0036] Additional explanation: Effectively perceived road area A eff The projection area of ​​the robot's close-range high-precision observation window only covers a narrow section of the road in front of the robot and is not the actual total area of ​​the entire lane. In close-range congestion scenarios, vehicle projections may overlap. The traffic density value is only used for the adaptive calculation of modal weights in this application and does not represent the macroscopic traffic density of the entire road.

[0037] This traffic semantic vector serves as the core input for subsequent local incremental reconstruction, blind spot heatmap construction, and perception backflow.

[0038] This embodiment first generates occlusion masks pixel by pixel using a lightweight semantic segmentation network, calculates the basic occlusion rate by statistically analyzing the pixel percentages, and then uses 3D laser point clouds to perform geometric verification, eliminating false occlusion errors caused by visual reflections and significantly improving the accuracy of occlusion rate calculation. A lightweight target detection model is used to quickly identify motor vehicles and non-motor vehicles and calculate the average confidence level, objectively quantifying the reliability of target recognition. Combining satellite positioning, depth observation range, and LiDAR field of view, the effective perceived road area is accurately counted within a global 3D Gaussian road map, and traffic flow density is calculated accordingly. Finally, a traffic semantic vector containing three quantitative indicators—actual occlusion rate, traffic flow density, and average confidence level—is formed. This provides a unified and quantifiable semantic input for subsequent map incremental updates and blind spot heatmap generation, enabling visual observation error correction, traffic target quantification statistics, and precise constraints on the perception space, ensuring the stability and reliability of subsequent mapping and risk assessment results.

[0039] Optionally, in one specific embodiment, the embodied roadside robot performs local incremental reconstruction and updates of the global 3D Gaussian road map based on multi-source perception data, chassis encoder pulse data, and traffic semantic vectors, such as... Figure 3 As shown, it includes: S21. The embodied roadside robot constructs coordinate transformation matrices based on chassis encoder pulse data and IMU pose data to map depth images and 3D laser point clouds to a global three-dimensional Gaussian road map, respectively. S22, the embodied roadside robot performs global coordinate mapping on satellite positioning data, depth images and 3D laser point clouds based on coordinate transformation matrix, so as to align the satellite positioning data, depth images and 3D laser point clouds with the global three-dimensional Gaussian road map in data space. S23, the embodied roadside robot, uses traffic semantic vectors, as well as satellite positioning data, depth images, and 3D laser point clouds aligned with data space, to perform local incremental reconstruction and update of the global 3D Gaussian road map.

[0040] Specifically, based on depth images, 3D laser point clouds, and satellite positioning data, this embodiment further introduces IMU pose data IM (triaxial acceleration a and triaxial angular velocity ω sampled at 200Hz). The mean value of this periodic IMU sampling window (the first 100ms, with a total of 20 sampling points) is: triaxial acceleration a = (0.018, -0.124, 9.787) m / s². 2 The triaxial angular velocity ω = (0.002, 0.001, 0.131) rad / s.

[0041] Embossed roadside robots based on chassis encoder pulse data (v chassis =0.62m / s,ω chassis =0.13rad / s) and IMU pose data IM are used to construct coordinate transformation matrices for mapping depth images and 3D laser point clouds to a global 3D Gaussian road map, respectively. Generally, the coordinate transformation matrix includes two matrices for mapping depth images and 3D laser point clouds, respectively.

[0042] Based on the coordinate transformation matrix, global coordinate mapping is performed on the satellite positioning data, depth image and 3D laser point cloud for the current period: satellite positioning data is directly mapped to the local ENU coordinate system of the global map according to the ECEF calculated by RTK; depth image, each pixel (u,v,d) is mapped to the world coordinate system (X,Y,Z) through the camera intrinsic parameter K and the camera extrinsic parameter (coordinate transformation matrix). 3D laser point cloud, each laser point is mapped to the world coordinate system through LiDAR extrinsic parameters (coordinate transformation matrix).

[0043] After mapping, the three types of data are spatially aligned in the same world coordinate system defined by the global 3D Gaussian road map. The spatial alignment residuals for this period are as follows: 1.9 cm for depth image and 3D laser point cloud, 3.1 cm for 3D laser point cloud and global map, and 4.7 cm for satellite positioning and global map, all meeting the alignment accuracy requirement of 5 cm.

[0044] Embossed roadside robots are based on traffic semantic vectors Λ=(S OCC D veh ,λavg The data is calculated as follows: (0.37, 0.62, 0.81), along with the spatially aligned satellite positioning, depth image, and 3D laser point cloud. These are used to perform local incremental reconstruction and update of the global 3D Gaussian road map.

[0045] Furthermore, the embodied roadside robot constructs coordinate transformation matrices based on chassis encoder pulse data and IMU pose data to map depth images and 3D laser point clouds to a global 3D Gaussian road map, including: The embodied roadside robot obtains its original displacement increment based on the chassis linear velocity and chassis angular velocity, as well as the pre-set chassis encoder sampling period and the pre-set global heading angle of the previous moment; and constructs the inertial displacement increment based on the IMU pose data. The embodied roadside robot performs a weighted summation of the original displacement increment and the inertial displacement increment to obtain the optimal displacement increment, and obtains the optimal global pose at the current moment based on the pre-set global pose and optimal displacement increment of the previous moment. Based on the optimal global pose at the current moment, the embodied roadside robot constructs coordinate transformation matrices to map the depth image and 3D laser point cloud to the global 3D Gaussian road map, respectively.

[0046] Specifically, in this embodiment, the chassis encoder output consists of two parts: the chassis linear velocity v. chassis =0.62m / s (calculated from the average pulse rate of the left and right drive wheel encoders using a chassis radius r=0.15m), chassis angular velocity ω chassis =0.13rad / s (calculated from the difference in pulse rates between the left and right drive wheel encoders using a wheelbase of b=0.62m).

[0047] Preset chassis encoder sampling period △T enc =10ms (100Hz), the pre-set heading angle ψ of the previous moment k-1 =87.42° (derived from the optimal global pose at the previous moment), the original displacement increment ΔP is calculated according to the differential kinematics model. enc : ; The embodied roadside robot constructs inertial displacement increments based on IMU pose data (three-axis acceleration a and three-axis angular velocity ω) through second-order integration. Within a time window ΔT = 10 ms, the mean equivalent horizontal acceleration after deducting the gravitational component is ã = (0.018, -0.124, 0) m / s². 2 The speed of ENU at the previous moment v k-1 =(0.028,0.62,0)m / s, then: ≈[ ]; Furthermore, △p imu For inertial displacement increment, rotation increment Δθ imu =ω·△T=0.131×0.01=1.31×10 -3 rad.

[0048] The embodied roadside robot performs a weighted summation of the original displacement increment and the inertial displacement increment. The weight coefficients are adaptive according to the current chassis slippage discrimination factor ξ∈[0,1]. In rainy weather and station entry / exit scenarios, the chassis is prone to slippage. At this moment, ξ=0.4, and the weight of the original displacement increment is 0.6, and the weight of the inertial displacement increment is 0.4. Similarly, when severe slippage occurs, ξ=1, and the inertial displacement increment is fully trusted. When there is no slippage, ξ=0, and the original displacement increment is fully trusted.

[0049] Furthermore, in this embodiment, the slippage discrimination factor ξ is calculated as follows: ξ=||△p enc -△p imu ||2 / T slide , among which, T slide The pre-set slip threshold is 0.008m in this embodiment, ||·| |2 It is an L2 norm.

[0050] The weighted summation result is the optimal displacement increment Δp. opt for: △p opt =α enc ·△p enc +α imu ·△p imu ; Where, α enc Assign weights to the original shift increment, α imu For the inertial displacement increment weight, α imu =ξ,α enc =1-ξ.

[0051] Preset the global pose L from the previous time step k-1 =(x k-1 ,y k-1 ,z k-1 ,ψ k-1 The optimal displacement increment is superimposed with the global pose from the previous time step, resulting in the optimal global pose L at the current time step. k =L k-1 +△p opt .

[0052] After 60 seconds of accumulation, the optimal global pose using weighted summation relative to the unweighted scheme decreased from 21.3cm to 4.9cm, a reduction of approximately 77%.

[0053] Based on the current optimal global pose L k By combining the pre-calibrated sensor extrinsic parameters, a coordinate transformation matrix is ​​constructed: H d→W =H robot→W (T k )· H L→W =H robot→W (T k )· ; Among them, H robot→W (T k The transformation from the physical roadside robot body to the global map world coordinate system is generated by the robot's optimal global pose at time T. For the pre-set inverse matrix of extrinsic parameters of the depth camera relative to the chassis of the roadside robot, The inverse matrix of the external parameters of the pre-set lidar relative to the chassis of the roadside robot.

[0054] H d→W H is the transformation matrix from depth image pixels to global world coordinates. L→W The transformation matrix from 3D laser point cloud to global world coordinates is used to map the depth image and 3D laser point cloud from their respective sensor coordinate systems to the world coordinate system defined by the global 3D Gaussian road map.

[0055] Furthermore, the embodied roadside robot, based on traffic semantic vectors and spatially aligned satellite positioning data, depth images, and 3D laser point clouds, performs local incremental reconstruction and updates of a pre-deployed global 3D Gaussian road map, including: Based on spatially aligned satellite positioning data, depth images, and 3D laser point clouds, the embodied roadside robot selects ROI areas to be incrementally updated from a pre-deployed global 3D Gaussian road map. The embodied roadside robot constructs multimodal fusion weights based on actual occlusion rate, traffic density, and average confidence level, and dynamically filters spatially aligned depth images and 3D laser point clouds based on these weights. The embodied roadside robot incrementally reconstructs and updates the ROI region in the global 3D Gaussian road map based on dynamically filtered depth images and 3D laser point clouds.

[0056] Specifically, the embodied roadside robot uses spatially aligned satellite positioning data as anchor points and combines this with the actual effective observation range of depth images and 3D laser point clouds (depth camera at close range of 4m in front of LiDAR ±60° × 20m). It then selects Regions of Interest (ROIs) from a pre-deployed global 3D Gaussian road map that are both within its current perception range and may require updating due to new observations. Each ROI is a rectangle centered on the embodied roadside robot's current position, extending 16m in front, 8m to the left and right, and 3m above. The ROI contains approximately 27,800 3D Gaussian points (accounting for 4.18 × 10^3 points on the global 3D Gaussian road map). 6 (0.66% of the total); the projected area of ​​the ROI region is 384m². 2 .

[0057] Embossed roadside robots are based on traffic semantic vectors Λ=(S OCC D veh ,λ avg Construct multimodal fusion weights (w) using (0.37, 0.62, 0.81). d ,w L ): w d =β1·(1-S OCC )·λ avg =0.5103·β1;w L =β2·[(1-β3·D veh )+β3·S OCC ] = 0.925·β2; Among them, the calibration coefficients for traffic semantic modality fusion are β1=0.9, β2=1.0, and β3=0.3.

[0058] Then the multimodal fusion weights (w) are obtained. d ,w L = (0.332, 0.668), which means that the current period has a high occlusion rate and a large traffic density. The radar mode weight increases to 0.668, and the depth image mode weight is compressed to 0.332. This is consistent with the physical intuition that LiDAR's geometric stability is better than RGB-D under moderate rain and bus occlusion conditions.

[0059] The embodied roadside robot dynamically filters spatially aligned depth images and 3D laser point clouds based on multimodal fusion weights. Depth image, press w d =0.332, 33.2% of pixels (approximately 688,000 pixels) were sampled from 1920×1080 pixels according to gradient intensity and semantic importance and used for reconstruction; 3D laser point cloud, according to w L=0.668, from 216,000 points, 66.8% of the points (about 144,000 points) were selected for reconstruction by binning and stratifying by distance and semantic importance sampling.

[0060] The total amount of observation data involved in the reconstruction was reduced to about 49.6% of the original data, and the computational cost of a single-cycle reconstruction was reduced by about 50%.

[0061] Based on dynamically filtered depth images and 3D laser point clouds, incremental reconstruction and updates are performed on the ROI regions in the global 3D Gaussian road map. The update method is as follows: For each 3D Gaussian point g within the ROI region j The position, covariance, opacity, and SH coefficient are updated in reverse using the composite loss composed of the filtered depth image photometric residual and the 3D laser point cloud geometric residual. Gaussian points without corresponding observations within the ROI region retain their original parameters (without updating). For locations with strong observations but no corresponding Gaussian points (i.e., newly appearing obstacles / road repair areas), three-dimensional Gaussian points are added according to a pre-set densification strategy.

[0062] Based on the above rules, the ROI region is incrementally reconstructed and updated.

[0063] This embodiment weights and fuses chassis encoder mileage information with IMU inertial observations, and adaptively allocates the weights of the two based on slippage factors, significantly suppressing pose accumulation drift during long-term operation and obtaining a high-precision optimal global pose. This is used to solve the transformation matrix from the sensor to the world coordinate system, achieving accurate spatial registration of depth images, laser point clouds, and the global map. Based on traffic semantic vectors, multimodal fusion weights for visual and laser modes are adaptively generated. Under conditions of severe occlusion and dense traffic flow, the weight of LiDAR observations is automatically increased. At the same time, observation data is dynamically sampled and simplified according to the weights, reducing the computational load by nearly half while ensuring the reliability of mapping. Only local ROI areas within the robot's current perception range are extracted for iterative optimization of 3D Gaussian point parameters. Only local areas are updated while retaining the static parameters of the global map. This avoids the huge computational overhead of global reconstruction and can iteratively optimize Gaussian point parameters based on photometric and geometric composite losses. Gaussian points are also added for newly appearing traffic targets. Ultimately, it balances positioning accuracy, multi-source observation reliability, real-time computing power constraints at the edge, and map update accuracy in dynamic scenarios, effectively adapting to the operational needs of mobile inspection at mixed-traffic intersections.

[0064] Based on this, the offline pre-construction process for the global 3D Gaussian road map is as follows: Collect raw data for the entire target area, including but not limited to RTK high-precision point cloud data, aerial orthophoto data, lane marking maps, and road attribute ledgers; The target intersection road area is subjected to grid-based scraping and semantic labeling of static elements such as roads, guardrails, sidewalks, and bus stops, and three types of basic semantic labels are distinguished: passable road surfaces, physical obstacles, and restricted areas. A Gaussian point rendering framework is used to discretize the continuous road geometry of the target area into a three-dimensional Gaussian point group, and the coordinates, covariance, opacity and SH16 spherical harmonic coefficients of each three-dimensional Gaussian point in the three-dimensional Gaussian point group are recorded. For each 3D Gaussian point within a target area, match the corresponding raster annotation semantic information and add structured semantic attribute fields, including the lane number, lane direction, lane line type, pedestrian crossing sign, bus stop mark, and entity obstacle category, etc. The system performs registration and alignment, deduplication of redundant points, sparse downsampling in long-distance areas, and dense densification in core areas near intersections on the global 3D Gaussian point group; it also removes noisy Gaussian points with elevation anomalies and coordinate drift to ensure that the geometric accuracy of the point cloud meets the error standard of less than 5cm. The optimized 3D Gaussian point group with complete semantic attributes is packaged and encapsulated into a global 3D Gaussian road map file and stored in the local edge computing module of the embodied roadside robot. When the robot starts its inspection, it directly loads the pre-built global 3D Gaussian road map. During the inspection, only the ROI area within the current observation range of the sensor is reconstructed and updated locally, without the need to reconstruct the global 3D Gaussian road map.

[0065] Optionally, in a specific embodiment, a local perception blind spot heatmap is constructed based on the updated global 3D Gaussian road map, such as... Figure 4 As shown, it includes: S24. Based on the depth image after data space alignment and the binary mask data corresponding to each pixel in the depth image, the embodied roadside robot determines whether each three-dimensional Gaussian point in the ROI region of the global three-dimensional Gaussian road map has a depth observation binary label and whether it is an occlusion region. S25. The embodied roadside robot uses the spatially aligned satellite positioning data as the center and performs uniform grid subdivision of the ROI region in the global 3D Gaussian road map according to the pre-set grid size to obtain multiple grid cells. Based on the updated global 3D Gaussian road map, it obtains the map coordinates and grid cell matrix corresponding to each grid cell. The grid cell matrix corresponding to each grid cell includes all 3D Gaussian points in the grid cell, as well as the depth observation binary label and occlusion binary label corresponding to each 3D Gaussian point in the grid cell. S26. The embodied roadside robot constructs the obstacle space ratio and depth observation missing ratio for each grid cell based on the depth observation binary label and occlusion binary label corresponding to each three-dimensional Gaussian point in each grid cell. S27. The embodied roadside robot constructs an occlusion index for each grid cell based on the proportion of obstacle space and the proportion of missing depth observations for each grid cell. S28, the embodied roadside robot performs visualization rendering of each grid cell based on the occlusion index corresponding to each grid cell and the pre-set color rendering strategy, and integrates all the visualized grid cells based on the map coordinates corresponding to each grid cell to obtain a local perception blind spot heat map.

[0066] Specifically, based on the spatially aligned depth image and the binary mask data M(u,v) corresponding to each pixel, for each 3D Gaussian point g within the ROI region of the global 3D Gaussian road map... j Determine two binary labels: Deep observation binary label b j,obs ∈{0,1}, when the projected pixel (u) on the depth image j ,v j There are valid depth observations (depth values ​​are not missing and are consistent with g). j When the true depth residual is ≤5cm, b j,obs =1, otherwise b j,obs =0; Occlusion of binary labels, b j,occ ∈{0,1}, when g j When the projected pixel is located in the binary mask occlusion area, b j,occ =1, otherwise b j,occ =0.

[0067] The labels of 28,051 three-dimensional Gaussian points in the ROI region were statistically analyzed.

[0068] Centered on the satellite positioning data projection points after data spatial alignment, the ROI region is uniformly divided into grids according to the pre-set grid size of 0.5m×0.5m, resulting in a total of 32×32=1024 grid cells.

[0069] The world coordinates of the center point of a raster cell are the map coordinates of that raster cell. The raster cell matrix G k It includes the set of all 3D Gaussian points within the raster cell, as well as the depth observation binary label and occlusion binary label for each point.

[0070] For each 3D Gaussian point within each grid cell, the proportions of the two items are calculated according to the following formula: Obstacle space ratio The percentage of missing in-depth observations .

[0071] After obtaining the obstacle space ratio and depth observation ratio within each grid cell, the grid occlusion index S is constructed. k S k =r1· +r2· The proportionality coefficients are r1=0.6 and r2=0.4.

[0072] Then, the embodied roadside robot performs visualization rendering on each grid cell according to a pre-set color rendering strategy (using JET color bands, with occlusion index [0,1] mapped to RGB [dark blue, cyan, green, yellow, red]).

[0073] Based on the map coordinates of each grid cell, the 1024 visualized and rendered grid cells are stitched together to form a local perception blind spot heatmap H. blind .

[0074] This embodiment relies on aligned depth image pixel masks to accurately label each 3D Gaussian point within the ROI region with binary labels for both depth observation and occlusion, achieving point-by-point quantization of observation and occlusion states. A 0.5-meter grid is created centered on satellite positioning coordinates, discretizing the continuous space into grid units. Each grid is then bound to a set of Gaussian points and corresponding labels to form a grid unit matrix. Based on the labeled Gaussian points within the grid, the proportion of obstacle space and the proportion of missing depth observations are statistically analyzed. A grid-level occlusion index is then calculated using fixed weights, avoiding the inability to distinguish local blind zone differences due to a globally uniform occlusion rate. Finally, grid visualization images are generated in batches using color-band rendering rules and stitched together to form a blind zone heatmap. This allows for a detailed characterization of the strength levels of perception blind zones at different locations, quantifying the perception risks caused by missing observations and obstacle occlusion. This provides refined grid risk factors for subsequent semantic cost maps, ensuring that inspection path planning can accurately avoid high blind zone sections.

[0075] Optionally, in one specific embodiment, the embodied roadside robot constructs a local semantic cost map to quantify the travel cost of the embodied roadside robot based on a local perception blind spot heatmap, multi-source perception data, and an updated global 3D Gaussian road map, such as... Figure 5 As shown, it includes: S31. The embodied roadside robot determines the access label corresponding to each grid cell based on the local perception blind zone heat map and the updated global 3D Gaussian road map, and constructs a static obstacle cost sub-graph based on the access label corresponding to each grid cell. S32. The embodied roadside robot inputs the depth image into a pre-deployed target detection model to detect targets, and obtains all the detected targets in the depth image and the instantaneous velocity corresponding to each detected target. S33. The embodied roadside robot maps each detected target in the depth image to the corresponding grid cell, and based on the instantaneous velocity of each detected target, it dilates the reachable grid of each detected target within a pre-set time window to construct a dynamic object cost subgraph. S34. The embodied roadside robot constructs a blind spot penalty cost subgraph based on the occlusion index corresponding to each grid unit in the local perception blind spot heatmap. S35. The embodied roadside robot linearly superimposes the static obstacle cost subgraph, dynamic object cost subgraph, and blind spot penalty cost subgraph based on the pre-set cost subgraph weighting coefficients to obtain a local semantic cost map.

[0076] Specifically, the embodied roadside robot determines a passage label P for each grid cell based on a local perception blind spot heatmap and an updated global 3D Gaussian road map. K ∈{0,0.5,1} (The pass label represents 0 = passable, 0.5 = restricted passable, and 1 = impassable).

[0077] If the percentage of Gaussian points with a depth observation binary label of 1 within a raster cell is greater than 0.7, and their semantics are entity obstacles (vehicles, guardrails, curbs, etc.), then P K =1; If the proportion of Gaussian points with a depth observation binary label of 1 within a raster cell is in the range of [0.3, 0.7], or the semantics are non-motorized vehicle lane / temporary occupancy, then P K =0.5; The rest of P K =0.

[0078] Construct a static obstacle cost subgraph C based on access labels. static :C static (C k )=μ static ·P K ; Among them, C k For the raster cell with index k, μ static In this embodiment, μ is a pre-set weighting coefficient. static =1.

[0079] The embodied roadside robot inputs depth images into a pre-deployed target detection model for target detection (consistent with the above), obtaining all detected targets and their corresponding instantaneous velocities. The instantaneous velocities of the 28 detected targets in this period are calculated using pixel-level optical flow tracking and depth backprojection. Motor vehicles (23 vehicles) had an instantaneous speed range of -0.8 to 3.6 m / s and an average instantaneous speed of 1.42 m / s. Non-motor vehicles (5 vehicles) had an instantaneous speed range of 0.3 to 2.8 m / s and an average instantaneous speed of 1.18 m / s.

[0080] Each detected target i is mapped to its corresponding grid cell (based on the determination that its center point in the world coordinate system falls into the grid cell), and the instantaneous velocity v of the target is used as the basis for the mapping. i Along its direction of motion, within a pre-set time window △T pred =Forward reachable grid R within 2.0s i Perform expansion treatment: ; Among them, (X) k ,Y k (X) represents the planar coordinates of the raster Ck in the global world coordinate system. i ,Y i Let be the global planar coordinates of the i-th traffic target at the current time, and v be the coordinates of the target. i Let v be the instantaneous linear velocity of the i-th traffic target. i,p Let be the velocity direction vector of the i-th traffic target.

[0081] The expansion direction is along v i A 60° forward sector in the vector direction, with the sector radius proportional to the instantaneous velocity.

[0082] Constructing the cost subgraph of dynamic objects C dynamic : ; In this embodiment, μ is taken. dyn =1.2, σ v =0.6m+0.4v i .

[0083] The embodied roadside robot directly constructs a blind spot penalty cost subgraph C based on the occlusion index corresponding to each grid cell in the local perception blind spot heatmap. blind C blind (C k )=1.5·S k .

[0084] Linear superposition is performed according to the pre-set cost subgraph weighting coefficients. Generally, the weight coefficient of the blind zone penalty cost subgraph is 0.4, and the weight coefficient of the dynamic object cost subgraph and the static obstacle cost subgraph is 0.3.

[0085] The cost map creates a low-cost passage corridor near the central safety island at the intersection, a high-cost avoidance restricted area behind the bus stop, and a medium-cost dynamic avoidance zone in the forward expansion area of ​​motor vehicles, which is consistent with the intuitive human driving decision-making in the complex evening rush hour scenario at this time.

[0086] This embodiment first divides traffic into three levels based on the proportion of Gaussian points of obstacles within the grid and their semantic attributes, thereby generating a static obstacle cost sub-map to accurately define prohibited and slow-moving areas caused by physical obstacles such as guardrails and stationary vehicles. Combining target detection and optical flow velocity measurement, the instantaneous speed of traffic targets is obtained, and a fan-shaped grid expansion is performed along the direction of movement. An exponential decay cost is set according to the distance to generate a dynamic object cost sub-map, predicting the intrusion areas of motor vehicles and non-motor vehicles in the next two seconds. The grid occlusion index of the blind spot heatmap is directly reused to generate a blind spot penalty cost sub-map, transforming the risk of missing perception into a traffic penalty item. Then, the three-layer cost sub-maps are linearly fused using fixed weights, simultaneously considering the risks of static physical obstacles, dynamic vehicle intrusion, and potential blind spot hazards. The resulting semantic cost map not only conforms to the real driving decision-making logic under mixed traffic conditions at intersections but also provides multi-dimensional quantitative costs for path planning, effectively ensuring that the inspection robot actively avoids obstacles and stays away from perception blind spots, significantly improving the safety of autonomous navigation in complex traffic scenarios.

[0087] Furthermore, the embodied roadside robot constructs a local inspection path to guide its movement based on a local semantic cost map and a pre-set path planning algorithm (in this embodiment, the DLite incremental heuristic search algorithm is used). Starting point: Current robot pose (12.314, 8.762); End point: Next fixed monitoring point (central safety island at the intersection, world coordinates (15.020, 24.187)) or a pre-set displacement distance; Number of path nodes: 47 (discretely sampled along the low-cost corridor, with a spacing of 0.5m between adjacent nodes); Total path length: 16.2m; Average path cost: 0.142 (far lower than the greedy straight path cost of 0.487); Critical avoidance point: 0.6m to the right near (12.8, 14.5) to avoid the high-cost area behind the bus stop; Path generation time: 14ms.

[0088] Optionally, in a specific embodiment, traffic situation perception results are obtained based on multi-source sensing data and local sensing blind spot heat maps, such as... Figure 6 As shown, it includes: S41. The embodied roadside robot inputs the depth image from the multi-source perception data into a pre-deployed target detection model to detect targets and obtain the target position, target category and instantaneous velocity of each detected target within the current perception range. S42. The embodied roadside robot obtains traffic situation perception results based on the target position, target category, and instantaneous speed of each detected target within the current perception range, as well as the heat map of the local perception blind area.

[0089] Specifically, the embodied roadside robot inputs the depth image from the multi-source perception data into a pre-deployed target detection model to perform target detection and obtain a quadruple for each detected target i within the current perception range. That is, the target position in the two-dimensional plane coordinates of the global three-dimensional Gaussian road map world coordinate system, obtained by depth back projection; the target category (motor vehicle / non-motor vehicle / pedestrian, etc.), which is output by the target detection network classification; the instantaneous velocity, which is calculated by pixel-level optical flow and depth back projection; and the confidence level.

[0090] The embodied roadside robot obtains traffic situation perception results based on the target position, target category, instantaneous velocity of each detected target within the current perception range, as well as a heatmap of local perception blind spots. The traffic situation perception results in this embodiment consist of the following fields: Detected target list: 28 targets (ID, category, target location, instantaneous velocity, confidence level); Current perception range: 16m×16m projection envelope centered on the robot; Blind spot distribution: occlusion index thermal distribution of 1024 grids, maximum value 0.83 (behind the bus stop), average value 0.21; High-density traffic area: near the central safety island at the intersection, left-turn waiting area (peak density of dynamic objects); Key event indication: OBJ-019 and OBJ-020 have a spacing of <0.8m, accompanied by obvious speed matching, marked as a suspected rear-end collision event, frames to be reviewed; Timestamp.

[0091] Traffic situation awareness results are transmitted to the upper-level business management module via a 5G communication module at a frequency of 5Hz, and visualized in real time via a web interface.

[0092] This embodiment relies on a target detection model to output the location, category, instantaneous speed, and confidence level information of each traffic target. Simultaneously, it introduces a local perception blind spot heatmap to supplement risk information in areas lacking perception. By combining dynamic vehicle target information with grid blind spot distribution information, a multi-dimensional traffic situation perception result is formed, including a target list, perception range, blind spot heatmap, high-density traffic areas, and traffic emergencies. The data structure is standardized and complete, enabling accurate identification of potential traffic accidents such as vehicles driving close together. This data is then transmitted back to the management platform in real time via high-frequency wireless communication and visualized at the front end. This achieves both refined monitoring of dynamic traffic targets and incorporates potential blind spot hazards into the situation assessment scope, effectively improving the completeness and predictive capability of traffic event identification at intersections with mixed traffic, meeting the business needs of urban road mobile inspection and traffic management.

[0093] Optionally, in one specific embodiment, the method further includes: After obtaining traffic situation perception results, the embodied roadside robot constructs a perception backflow vector based on the traffic situation perception results and the local perception blind zone heat map. The perception backflow vector is used to indicate the grid cells in the local perception blind zone heat map that need to be incrementally reconstructed and updated, as well as the expected depth observation density corresponding to each grid cell. The embodied roadside robot performs secondary local incremental reconstruction and update of the updated global 3D Gaussian road map based on the perception return vector, and optimizes the local inspection path based on the secondary updated global 3D Gaussian road map to guide the roadside robot's movement.

[0094] Specifically, after acquiring traffic situation perception results, the embodied roadside robot constructs a perception return vector V based on the traffic situation perception results and a heat map of local perception blind spots. back The logic is as follows: The occlusion index S was selected from the local perception blind zone heat map. k All high-occlusion grids with a value >0.5, totaling 36 in this cycle; From the traffic situation perception results, detection targets located in the adjacent area of ​​high-obscured grids or marked as suspected rear-end collisions, missed detections, or critical confidence levels were selected. In this cycle, a total of 4 targets were associated with 18 grid cells. After merging and deduplication, the sensing return vector V for this cycle is... back A total of 18 raster cells are designated for priority incremental reconstruction; For each grid cell, the desired depth observation density ρ is calculated based on its current occlusion index and target density using the following formula. k (Unit: observation points / grid): ; Among them, l gl This is the correlation event index. It is 1 when there are associated hazardous events such as rear-end collision risk, target miss detection, or traffic conflict within the grid, and 0 when there are no traffic risk events within the grid. In this embodiment, ρ0 = 60, so the grid cell C behind the bus stop... 712 (its S) 712 =0.78, associated with suspected missed detection events): =60·3.95=237 observation points / grid.

[0095] The original observation density was only 38 observation points / grid (the depth image was obstructed by public transportation).

[0096] The embodied roadside robot performs a secondary local incremental reconstruction update on the updated global 3D Gaussian road map based on the perceived return vector: For the 18 high-priority grids indicated in the perception return vector, the body-based blind spot filling is triggered: the lifting column is raised to +0.85m, translated laterally by 0.4m, and actively rotated by 28°, so that the main axis of the RGB-D depth camera is directly facing the blind spot; Multiple frames of blind spot depth images and 3D laser point clouds were acquired from a new perspective. Secondary reconstruction of the three-dimensional Gaussian points in 18 high-priority grids was performed, with 145-239 new depth observation points added to each grid. After the second reconstruction, the average occlusion index of the 18 grids decreased from 0.71 to 0.19, a reduction of 73%.

[0097] Based on the updated global 3D Gaussian road map, the robot reconstructs the local perception blind spot heat map and the local semantic cost map, and then uses the traffic situation perception results to incrementally update (only incremental correction is made to the affected grid, and the whole map replanning takes 4 ms) to regenerate the local inspection path.

[0098] This embodiment combines traffic situational awareness results with blind spot heatmaps to generate perception return vectors. It automatically filters highly occluded grids and risk areas such as missed target detections and nearby vehicles, accurately locating a limited number of grids that need blind spot reconstruction. Based on the occlusion index and traffic events, it dynamically calculates the expected observation density of each grid, quantifying the blind spot acquisition index. The robot actively adjusts its pose and sensor posture according to the return vectors to complete the viewpoint blind spot acquisition. It only performs secondary incremental reconstruction on a small number of high-priority grids, significantly reducing the computational cost of global reconstruction, effectively filling the observation gaps caused by large vehicle occlusion, and significantly reducing the grid occlusion index. Subsequently, it only incrementally updates the blind spot heatmap and cost map for the affected areas, rapidly iterating and optimizing the inspection path, continuously reducing the risk of missed detections caused by perception blind spots, and continuously improving map accuracy and path planning safety, adapting to autonomous inspection operations in complex occlusion scenarios at intersections.

[0099] In addition, this application provides a traffic situation awareness system, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the above-described traffic situation awareness method.

[0100] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0101] In the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0102] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make modifications, alterations, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A traffic situation perception method, characterized in that, The method is based on a embodied roadside robot, and the method includes: During the inspection of the target area according to the pre-set inspection instructions, the embodied roadside robot acquires chassis encoder pulse data and multi-source perception data within the current perception range in real time, and constructs traffic semantic vectors based on the multi-source perception data and the pre-deployed global three-dimensional Gaussian road map. The embodied roadside robot performs local incremental reconstruction and updates of the global 3D Gaussian road map based on multi-source perception data, chassis encoder pulse data, and traffic semantic vectors. Based on the updated global 3D Gaussian road map, it constructs a local perception blind zone heat map. The local perception blind zone heat map is a gridded occlusion index distribution map within the current perception range of the embodied roadside robot. Based on local perception blind spot heatmaps, multi-source perception data, and updated global 3D Gaussian road maps, the embodied roadside robot constructs a local semantic cost map to quantify the travel cost of the embodied roadside robot. The embodied roadside robot constructs a local inspection path to guide its movement based on a local semantic cost map and a pre-set path planning algorithm, and obtains traffic situation perception results based on multi-source perception data and local perception blind spot heat maps.

2. The traffic situation perception method according to claim 1, characterized in that, The multi-source sensing data includes: depth images, 3D laser point clouds, and satellite positioning data; The embodied roadside robot constructs traffic semantic vectors based on depth images, 3D laser point clouds, satellite positioning data, and a pre-deployed global 3D Gaussian road map, including: The embodied roadside robot performs pixel-by-pixel classification of depth images based on a pre-set semantic segmentation network, obtaining binary mask data corresponding to each pixel in the depth image. Based on the binary mask data corresponding to all pixels in the depth image, a basic occlusion rate is constructed for the depth image. The basic occlusion rate is then corrected based on 3D laser point cloud to obtain the actual occlusion rate. The binary mask data is used to distinguish whether a pixel is an occluded area or an effective road surface area. The basic occlusion rate is the ratio of the number of pixels in the depth image whose binary mask data indicates an occluded area to the total number of pixels. The embodied roadside robot inputs depth images into a pre-deployed target detection model to detect targets, obtains all detected targets in the depth image and the confidence score corresponding to each detected target in the depth image, and obtains the average confidence score based on the confidence score corresponding to each detected target in the depth image; the detected targets include motor vehicles and non-motor vehicles; The embodied roadside robot obtains the effective perceived road area within the current perception range based on satellite positioning data, depth images, 3D laser point clouds, and a pre-deployed global 3D Gaussian road map. Based on the effective perceived road area within the current perception range and all detected targets in the depth image, it obtains the corresponding traffic flow density. The traffic flow density is the ratio of the number of detected targets in the depth image to the effective perceived road area within the current perception range. The traffic semantic vector includes the actual occlusion rate, traffic density, and average confidence level.

3. The traffic situation perception method according to claim 2, characterized in that, The multi-source sensing data also includes IMU pose data; The embodied roadside robot performs local incremental reconstruction and updates of the global 3D Gaussian road map based on multi-source perception data, chassis encoder pulse data, and traffic semantic vectors, including: The embodied roadside robot constructs coordinate transformation matrices based on chassis encoder pulse data and IMU pose data to map depth images and 3D laser point clouds to a global 3D Gaussian road map, respectively. The embodied roadside robot performs global coordinate mapping on satellite positioning data, depth images, and 3D laser point clouds based on a coordinate transformation matrix, so as to align the satellite positioning data, depth images, and 3D laser point clouds with the global three-dimensional Gaussian road map in data space. The embodied roadside robot performs local incremental reconstruction and updates of the global 3D Gaussian road map based on traffic semantic vectors, as well as satellite positioning data, depth images, and 3D laser point clouds after data spatial alignment.

4. The traffic situation perception method according to claim 3, characterized in that, The chassis encoder pulse data includes chassis linear velocity and chassis angular velocity; The embodied roadside robot constructs coordinate transformation matrices based on chassis encoder pulse data and IMU pose data to map depth images and 3D laser point clouds to a global 3D Gaussian road map, including: The embodied roadside robot obtains its original displacement increment based on the chassis linear velocity and chassis angular velocity, as well as the pre-set chassis encoder sampling period and the pre-set global heading angle of the previous moment; and constructs the inertial displacement increment based on the IMU pose data. The embodied roadside robot performs a weighted summation of the original displacement increment and the inertial displacement increment to obtain the optimal displacement increment, and obtains the optimal global pose at the current moment based on the pre-set global pose and optimal displacement increment of the previous moment. Based on the optimal global pose at the current moment, the embodied roadside robot constructs coordinate transformation matrices to map the depth image and 3D laser point cloud to the global 3D Gaussian road map, respectively.

5. The traffic situation perception method according to claim 3, characterized in that, The embodied roadside robot, based on traffic semantic vectors and spatially aligned satellite positioning data, depth images, and 3D laser point clouds, performs local incremental reconstruction and updates of a pre-deployed global 3D Gaussian road map, including: Based on spatially aligned satellite positioning data, depth images, and 3D laser point clouds, the embodied roadside robot selects ROI areas to be incrementally updated from a pre-deployed global 3D Gaussian road map. The embodied roadside robot constructs a multimodal fusion weight based on the actual occlusion rate, traffic density, and average confidence level, and dynamically filters the spatially aligned depth image and 3D laser point cloud based on the multimodal fusion weight. The embodied roadside robot incrementally reconstructs and updates the ROI region in the global 3D Gaussian road map based on dynamically filtered depth images and 3D laser point clouds.

6. The traffic situation perception method according to claim 5, characterized in that, Based on the updated global 3D Gaussian road map, a heatmap of local perception blind spots is constructed, including: Based on the spatially aligned depth image and the binary mask data corresponding to each pixel in the depth image, the embodied roadside robot determines whether each 3D Gaussian point in the ROI region of the global 3D Gaussian road map has a depth observation binary label and whether it is an occluded region. The embodied roadside robot uses spatially aligned satellite positioning data as the center and performs uniform grid subdivision of the ROI region in the global 3D Gaussian road map according to a pre-set grid size to obtain multiple grid cells. Based on the updated global 3D Gaussian road map, it obtains the map coordinates and grid cell matrix corresponding to each grid cell. The grid cell matrix corresponding to each grid cell includes all 3D Gaussian points within the grid cell, as well as the depth observation binary label and occlusion binary label corresponding to each 3D Gaussian point within the grid cell. The embodied roadside robot constructs the obstacle space ratio and depth observation missing ratio for each grid cell based on the depth observation binary label and occlusion binary label corresponding to each three-dimensional Gaussian point in each grid cell. The embodied roadside robot constructs an occlusion index for each grid cell based on the proportion of obstacle space and the proportion of missing depth observations for each grid cell. The embodied roadside robot visualizes and renders each grid cell based on the occlusion index corresponding to each grid cell and a pre-set color rendering strategy. Based on the map coordinates corresponding to each grid cell, it integrates all visualized and rendered grid cells to obtain a heat map of local perception blind spots.

7. The traffic situation perception method according to claim 1, characterized in that, The local sensing blind zone heatmap includes multiple grid units, and each grid unit has a corresponding occlusion index; the multi-source sensing data includes depth images; Based on local perception blind spot heatmaps, multi-source perception data, and an updated global 3D Gaussian road map, the embodied roadside robot constructs a local semantic cost map to quantify the robot's travel costs, including: The embodied roadside robot determines the access label corresponding to each grid cell based on the local perception blind zone heat map and the updated global 3D Gaussian road map, and constructs a static obstacle cost subgraph based on the access label corresponding to each grid cell. The embodied roadside robot inputs the depth image into a pre-deployed target detection model to detect targets, obtaining all detected targets in the depth image and the instantaneous velocity corresponding to each detected target; The embodied roadside robot maps each detected target in the depth image to the corresponding grid cell, and based on the instantaneous velocity of each detected target, it dilates the reachable grid of each detected target within a pre-set time window to construct a dynamic object cost subgraph. The embodied roadside robot constructs a blind spot penalty cost subgraph based on the occlusion index corresponding to each grid cell in the local perception blind spot heatmap. The embodied roadside robot linearly superimposes the static obstacle cost subgraph, the dynamic object cost subgraph, and the blind spot penalty cost subgraph based on pre-set cost subgraph weighting coefficients to obtain a local semantic cost map.

8. The traffic situation perception method according to claim 1, characterized in that, The multi-source sensing data includes depth images; Based on multi-source sensing data and local sensing blind spot heat maps, traffic situation perception results are obtained, including: The embodied roadside robot inputs the depth image from the multi-source perception data into a pre-deployed target detection model to detect targets, and obtains the target position, target category and instantaneous velocity of each detected target within the current perception range; The embodied roadside robot obtains traffic situation perception results based on the target position, target category, and instantaneous speed of each detected target within the current perception range, as well as the heat map of local perception blind spots.

9. The traffic situation perception method according to claim 1, characterized in that, The local sensing blind zone heat map includes multiple grid units; The method further includes: After obtaining the traffic situation perception results, the embodied roadside robot constructs a perception backflow vector based on the traffic situation perception results and the local perception blind zone heat map; wherein, the perception backflow vector is used to indicate the grid cells in the local perception blind zone heat map that need to be incrementally reconstructed and updated, and the expected depth observation density corresponding to each grid cell. The embodied roadside robot performs secondary local incremental reconstruction and update of the updated global 3D Gaussian road map based on the perception return vector, and optimizes the local inspection path based on the secondary updated global 3D Gaussian road map to guide the roadside robot's movement.

10. A traffic situation awareness system, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the traffic situation awareness method according to any one of claims 1-9.

Images