Optimization Layout Method of Road-Side Sensing Devices Integrating Real-Scene Point Cloud and Virtual Simulation

Abstract

The present invention discloses an optimized layout method for roadside perception devices that integrates real - world point clouds and virtual simulation, which relates to the technical field of optimized layout of perception - type sensors. It includes: replicating the three - dimensional space of active road infrastructure with three - dimensional real - world point cloud data, creating a virtual - reality integrated traffic scene with traffic individual models as the link, establishing a sensing model for roadside perception devices based on spherical coordinate projection, obtaining depth perception maps of static road environments and traffic individual triangular mesh models, establishing a collaborative perception efficiency evaluation, and optimizing the layout of collaborative perception devices; it can simultaneously take into account the accurate representation of static road environments and the advantages of the flexible control of traffic flow by virtual simulation, providing a more realistic and diverse traffic scene for the optimized layout of roadside collaborative perception devices.

Description

Technical Field

[0001] The present invention relates to the technical field of optimized layout of perception sensors, and specifically to an optimized layout method for roadside perception devices that integrates real-scene point clouds and virtual simulations. Background Art

[0002] Roadside collaborative perception devices are an important part of future intelligent transportation systems. On the one hand, the roadside perception system can perceive the traffic road scene flow through the collaborative work of multiple sensors, obtain high-precision traffic operation information, and provide basic data for real-time traffic risk prediction and control. On the other hand, in the vehicle-road collaborative system, the roadside perception system can effectively supplement the vehicle perception blind area and provide more complete scene information for the decision-making of intelligent vehicles.

[0003] Different from traditional video surveillance devices, roadside collaborative perception devices can achieve data interconnection through wireless communication technologies, such as C-V2X based on 5G cellular networks, dedicated short-range communication DSRC, etc., and the data obtained by sensors can be unified into the same coordinate space to achieve holographic perception of traffic scenes. Currently, roadside collaborative perception devices are mainly radar-vision fusion all-in-one machines or lidar. Although roadside collaborative perception devices can improve the accuracy of traffic perception through the fusion of multi-view data, the cost of roadside perception devices is relatively high, and as the number of perception devices increases, the data throughput will also increase, and the requirements for computing power resources are also higher. Against this background, the optimized layout of roadside collaborative perception devices has become one of the emerging technical issues in the current intelligent transportation field.

[0004] The existing optimized layout technologies of perception sensors in the traffic field mainly focus on the macroscopic road network scale, and the accuracy of the sensor layout position is in the order of 100 meters to 1000 meters. The main purpose is to optimize the position deployment of sensors at the macroscopic scale to achieve the observation and inference of macroscopic road network traffic information. Different from this, roadside collaborative perception mainly faces relatively important road intersections or sections. Therefore, the optimized layout of roadside collaborative perception devices needs to solve the problems of the layout position of perception devices and the attitude angle of sensors at the local space scale. The layout position accuracy is in the order of 1 centimeter to 1 meter, and the attitude angle setting accuracy is in the order of 1 degree to 5 degrees. The main goal is to comprehensively and accurately obtain the microscopic traffic information in the key area by optimizing parameters such as the position and angle of perception devices within the local space range. The present invention is directed to the optimized layout of perception sensors for microscopic traffic monitoring.

[0005] At the microscopic space scale, the main factor affecting the working effect of roadside perception devices is the occlusion of obstacles. Specifically, static obstacles (such as building facades) and dynamic obstacles (bus bodies) on the roadside will both cause perception blind areas to the roadside perception system, thereby affecting the monitoring of the dynamic traffic scene by roadside perception devices.

[0006] The existing technical solutions can be divided into two categories: those based on virtual simulation and those based on real-scene data-driven.

[0007] Traffic parameters can be configured in the virtual simulation platform to simulate different traffic flow operation scenarios. And through the combination of simulation platforms (such as the combination of Carla [1] and Sumo [2]), the perception of traffic flow by roadside perception devices can be simulated. Currently, the technical solutions based on virtual simulation can be further divided into two categories: direct methods and indirect methods. The direct method takes the layout parameters of roadside perception devices as decision variables, directly obtains the perception effect indicators under specific parameters by calling the sensor model in the simulation platform, and takes improving the perception indicators as the objective function to promote the optimal layout of roadside perception devices [3]. The indirect method is to establish a relationship model between the layout parameters of roadside perception devices and the evaluation perception efficiency indicators under different traffic operation scenario conditions through a large number of simulations [4]. Based on the relationship model, the optimal configuration of roadside perception devices is promoted with the goal of improving the perception efficiency.

[0008] The main disadvantage of the simulation-based technical solution is the difference between the simulation scenario and the real scenario, resulting in the fact that the optimal layout result of the roadside perception device cannot well respond to the actual needs. And it takes a lot of manpower and time to reproduce the real road traffic environment on the virtual simulation platform, with a high cost.

[0009] Therefore, the technical solution based on real-scene data-driven proposes to use three-dimensional real-scene data as the digital representation of the road environment, and on this basis, carry out the optimal layout of roadside perception devices. Ma et al. used the three-dimensional point cloud data of real roads as the background, regarded the road surface area as the target monitoring area, constructed a sensor model based on the Unet neural network, and took 100% coverage of the target monitoring area under the condition of collaborative perception by multiple roadside perception devices as the constraint and minimizing the number of sensors as the goal to realize the optimal layout of roadside perception devices. However, in their technical solution, traffic individuals were not considered, nor was the occlusion effect of moving vehicles on roadside perception considered [5], and the layout result was too ideal. Ma et al. combined the three-dimensional point cloud data of real roads with the measured trajectories to realize the reconstruction of the dynamic traffic three-dimensional scene, and at the same time considered the adverse effects of static and dynamic obstacles in the road environment, and took the average recall rate as the evaluation index of perception efficiency to realize the optimal layout of roadside perception devices at intersection positions [6]. However, the construction of the traffic scene in their technical solution strongly depends on the actual trajectory data collection and cannot adjust the traffic scene according to the needs, resulting in the fact that the technical solution cannot flexibly cope with the layout of roadside perception devices in other traffic scenarios.

[0010] In summary, the existing technical solutions cannot flexibly optimize the layout of roadside perception devices under different traffic flow operation characteristics on the basis of accurately mapping the three-dimensional environment of real roads. Summary of the Invention

[0011] To make up for the deficiencies of the existing technical problems, the purpose of the present invention is to provide an optimized layout method for roadside perception devices that integrates real-scene point cloud and virtual simulation. By integrating three-dimensional real-scene point cloud data and traffic virtual simulation technology, on the basis of accurately reconstructing the current road environment with real-scene point cloud data, traffic operation parameters are controlled through traffic simulation software, and a three-dimensional traffic individual model library is established to create a virtual-real fusion traffic scene. Based on the virtual-real fusion scene, a sensor simulation model for roadside perception is established, and the multi-time-step average intersection over union of the original bounding box of the traffic individual model and the bounding box under roadside perception conditions is calculated as the perception efficiency index. Taking the layout parameters of the roadside perception device as the decision variable and the improvement of the perception efficiency index as the objective function, the technical solution of the present invention is constructed. Compared with the existing technical solutions, the present invention retains the advantage of customizing and adjusting traffic parameters through traffic virtual simulation on the basis of mapping the real road space, and is more suitable for the application of roadside collaborative perception devices in the intelligent transformation of roads.

[0012] To achieve the above object, the present invention provides the following technical solutions:

[0013] An optimized layout method for roadside perception devices that integrates real-scene point cloud and virtual simulation, comprising the following steps:

[0014] (1) Create a file that meets the requirements of traffic simulation software according to the basic road geometric information in the three-dimensional real-scene point cloud data; establish a traffic individual model database, and obtain its original bounding box according to the size of each traffic individual triangular network model in the traffic individual model database;

[0015] (2) Create a virtual-real fusion traffic scene

[0016] Import the basic road geometric information file into the traffic simulation software, establish a road network model consistent with the coordinate space of the real-scene point cloud data, link the traffic individual model database based on the traffic individual movement trajectory, and use the three-dimensional real scene as the static road environment to create a virtual-real fusion traffic scene;

[0017] (3) Establish a sensing model for roadside perception devices based on spherical coordinate projection

[0018] Taking the center of a single roadside perception sensor as the origin, a local coordinate system of the sensor is established. Through coordinate transformation, both the static road environment and the traffic individual triangular network model in the virtual-real fusion scene in the global coordinate system are mapped into the local coordinate system of the sensor. The sensor is used to perform three-dimensional perception on the static road environment and the traffic individual in the local coordinate system of the sensor respectively, obtaining the depth perception maps of the static road environment and the traffic individual triangular mesh model respectively, and fusing the two depth perception maps.

[0019] (4) Cooperative perception efficiency evaluation

[0020] The fused depth perception map is re-projected back into the global coordinate system to obtain the perception data acquisition result of the roadside sensor for the traffic individual triangular mesh model. For the modeling of the cooperative perception process of multiple roadside perception sensors, the roadside perception device sensor model is applied to each roadside perception sensor respectively, and the traffic individual data points are combined in the global coordinate space to obtain the data acquisition result under the condition of cooperative perception.

[0021] The bounding box under the roadside perception condition is obtained according to the perception data acquisition result, and its intersection over union (IoU) parameter is calculated by comparing it with the original bounding box. The average IoU parameter of all traffic individuals is used as the evaluation index of the roadside cooperative perception efficiency.

[0022] (5) Optimal layout of cooperative perception devices

[0023] Taking the configuration parameter sets of multiple roadside perception devices as decision variables, the perception efficiency index is obtained through continuous virtual-real fusion simulation and applying the sensor simulation model. With the goal of improving the perception efficiency, the layout parameters of the roadside perception devices are continuously iterated by a Bayesian optimizer.

[0024] The basic road geometric information includes the lane center line, the road boundary line, and the lane width.

[0025] The establishment of the traffic individual model database is as follows:

[0026] The triangular mesh models of different traffic individuals are obtained through an open-source network database, and the geometric dimension information of each model is recorded, forming a traffic individual model database together with the original mesh model. The triangular mesh model is defined by the vertex coordinates and the vertex index matrix of the triangles.

[0027] Mapping the static road environment and the traffic individual model in the virtual-real fusion scene into the spherical coordinate space is as follows:

[0028] Let C s be the coordinates of the center of a single roadside perception sensor, α, β, and γ be the pitch angle, roll angle, and azimuth angle of the roadside perception device respectively, where the azimuth angle is based on the due north direction, and let P eFor the 3D real - world point cloud data of the road environment, let P a be the vertex coordinate data that constitutes the 3D model of traffic individuals, and let VI a be the index N of the triangular mesh a ×3 matrix, where N a is the number of vertices of the triangular mesh model;

[0029] Through coordinate transformation, both the static road environment and the traffic individual model in the virtual - real fusion scenario are mapped to the spherical coordinate space. The formulas are shown in Equations (1) to (3):

[0030]

[0031] Where: M 3×3 (α,β,γ)·(P e -C s ) is to convert the global coordinates to the local coordinates of the sensor;

[0032] C2S(.) is the function to convert Cartesian coordinates to spherical coordinates;

[0033] are the spherical coordinates of the static road 3D environment after conversion;

[0034] are the spherical coordinates of the traffic individual model after conversion;

[0035] φ, r are the coordinate components of the horizontal angle, pitch angle, and distance in the spherical coordinates respectively.

[0036] Use the sensor to perform 3D perception on the static road environment and traffic individuals respectively, as follows:

[0037] (5.1) Set and [φ min , φ max to be the horizontal angle range and vertical angle range of the sensing field of view of the roadside sensing device respectively; establish a new Cartesian coordinate system with φ, r as the coordinate components; project the road environment data and traffic individual model data after coordinate transformation onto the plane, and only retain the data within the sensing field of view of the roadside sensing device

[0038] (5.2) On the plane, rasterize the projected static road environment data . The pixel sizes along the direction and along the φ direction are and ε φ respectively, and obtain the depth perception map I e of the static road environment;

[0039] (5.3) In the plane, the ray - triangle intersection solution method is used to collect traffic individual data by the sensor, and the depth perception map I of the traffic individual triangular grid model is obtained a .

[0040] The three - dimensional perception process of traffic individuals using the ray - triangle intersection solution method is as follows:

[0041] (6.1) In the plane, rasterization processing consistent with the road static environment data is performed on the projected traffic individual triangular grid vertex data , and the pixel size is also kept consistent; after rasterization, each pixel point corresponds to a virtual ray;

[0042] (6.2) An envelope box of triangle k is established for the triangle vertices after rasterization of traffic individuals. Encoding processing is performed on any pixel or grid point in the envelope box and concatenated. At the same time, in order to retain the information of triangle index k, a one - dimensional array with the same length as the concatenated encoding is established, and the grid points in the two - dimensional envelope box are converted into a 1×m k ×2 matrix, where m k is the number of grid points in the envelope box;

[0043] Let (m,n) be the matrix position of a certain pixel in the envelope box, and linear encoding is performed on it using Equation (4);

[0044]

[0045] where:

[0046]

[0047] where mod(.) and rem(.) are the modulo and remainder calculation functions respectively; M is the number of rows of the rasterized matrix; is the linear encoding value;

[0048] (6.3) A null matrix of n f ×m k_max ×2 is pre - established on the GPU, where n f and m k_max are the number of triangle units and the m k value corresponding to the largest triangle respectively. All triangles are converted into matrices through parallel computing; then the layers with depth 1 and depth 2 in the n f ×m k_max ×2 matrix are concatenated respectively to obtain (n f·m k_max ) × 1 linear encoding matrix and triangular index matrix;

[0049] (6.4) Reorder and organize the linear encoding. Obtain multiple triangular indices corresponding to the same linear encoding value through sorting operations; through decoding calculations, convert to the position (m, n) of the pixel point, and there is a one-to-one mapping relationship between (m, n) and the virtual ray, then obtain the pair of virtual ray (ray) and triangle unit (triangle) that may intersect;

[0050] (6.5) Use the spatial geometry analysis method to solve the intersection points of the paired virtual rays and triangular units. For the case where the same ray corresponds to multiple triangles, only retain the nearest data points according to the perception principle; obtain the depth perception map I of the traffic individual triangular grid model a .

[0051] During the fusion process of the two depth perception maps, the occlusion effect sensed by the sensor needs to be considered, that is, the roadside sensing device cannot obtain the data of the occluded area:

[0052] Set Ω s to represent the configuration parameter set {C s , α, β, γ} of a single roadside sensing device; execute step 3 to obtain I s and I e under the condition of Ω a respectively; fuse I e and I a to obtain the depth perception map of the traffic individual under static and dynamic occlusion conditions, and the formula is as shown in Equation (6):

[0053]

[0054] where: I a (m, n) and I e (m, n) are the pixel points in the m-th row and n-th column of I a and I e respectively;

[0055] I' a is the depth perception map of the traffic individual after fusion.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] 1. The present invention replicates the three-dimensional space of the existing road infrastructure through three-dimensional real-scene point cloud data, and uses traffic individual models as a link. Through traffic simulation technology, traffic flow parameters are controlled to create a virtual-real fusion traffic scene, which can take into account both the accurate representation of the static road environment and the advantage of the flexible control of traffic flow by virtual simulation, providing a more realistic and diverse traffic scene for the optimal layout of roadside cooperative perception devices. Moreover, the technical idea of integrating real-scene point cloud data with traffic simulation is introduced into the field of optimal layout of roadside cooperative perception devices for the first time, having good commercial value.

[0058] 2. The present invention proposes a sensor modeling method based on spherical coordinate projection for the traffic scene of the fusion of real-scene point cloud and triangular mesh model. Especially for the perception process simulation of the traffic individual triangular mesh model, a technical process of "encoding - reorganization - decoding" is proposed to realize the search for the combination of "virtual ray - triangle unit" that may have intersection points, thereby reducing the number of virtual ray calculations and improving the operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flowchart of the method of the present invention.

[0060] Figure 2 It is a schematic diagram of the data format of the triangular mesh model in the present invention.

[0061] Figure 3 It is a schematic diagram of the creation process of the virtual-real fusion traffic scene in the present invention.

[0062] Figure 4 It is a schematic diagram of coordinate transformation in the present invention.

[0063] Figure 5 It is a schematic diagram of the projection of the traffic individual grid model in the present invention.

[0064] Figure 6 It is a schematic diagram of virtual rays in different coordinate systems in the present invention.

[0065] Figure 7 It is a schematic diagram of encoding in the "encoding - reorganization - decoding" in the present invention.

[0066] Figure 8 It is a schematic diagram of reorganization in the "encoding - reorganization - decoding" in the present invention.

[0067] Figure 9 It is a schematic diagram of decoding in the "encoding - reorganization - decoding" in the present invention.

[0068] Figure 10 It is a comparison chart of the running time between the present invention and the traditional GPU calculation method.

[0069] Figure 11This is the schematic diagram of perception data fusion in the present invention.

[0070] Figure 12 This is the schematic diagram of the optimization layout process of roadside perception devices in the present invention. Detailed implementation manners

[0071] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0072] As Figures 1-12 shown, the method for optimizing the layout of roadside perception devices that integrates real-scene point cloud and virtual simulation disclosed in the present invention is specifically as follows:

[0073] 1. Technical overview

[0074] The present invention proposes a technical framework for optimizing the layout of roadside perception devices that integrates real-scene point cloud and virtual simulation, as Figure 1 shown. Based on the three-dimensional real-scene point cloud data, first configure the road network model in the simulation platform to ensure that the simulation coordinate system is consistent with the real-scene point cloud data. Then establish a three-dimensional grid model library of traffic individuals and link it with the traffic simulation scene to integrate the real-scene point cloud and traffic simulation to create a three-dimensional traffic scene. Establish a sensor simulation model of roadside perception devices based on spherical coordinate projection, and through virtual-real fusion simulation, compare the original envelope box of traffic individuals with the envelope box under the condition of roadside joint perception to obtain the perception efficiency evaluation index. Finally, use the layout parameters of the perception devices as decision variables and the perception efficiency as the objective function, and apply the Bayesian optimization algorithm to complete the optimization layout of the roadside perception devices. The following will detail each step in the technical solution of the present invention.

[0075] 2. Creation of virtual-real fusion traffic scene

[0076] The creation of the virtual-real fusion traffic scene in the present invention is divided into the following three basic steps:

[0077] Step 1: Extract basic road geometric information such as lane centerlines, road boundaries, and lane widths based on the three-dimensional real-scene point cloud data, and establish an xml (extensible makeup language) file according to the format requirements of the traffic simulation software Simulation of Urban Mobility (SUMO).

[0078] Step 2: Obtain the triangular grid models of different traffic individuals through an open-source network database, record the geometric dimension information of each model, and form a traffic individual model database together with the original grid model. The triangular grid model is generally defined by vertex coordinates and the vertex index matrix that constitutes the triangle, as Figure 2As shown, other information of the triangular mesh model such as material and color is generally stored together with the index.

[0079] Step 3: Import the xml file into the traffic micro-simulation software, establish a road network model consistent with the coordinate space of the real-scene point cloud data, simulate different traffic scenarios by configuring traffic parameters, link the 3D model library of traffic individuals based on the movement trajectories of traffic individuals, and use the 3D real scene as the static road environment to create a virtual-real fusion traffic scenario. The generation process of the virtual-real fusion traffic scenario is shown in Figure 3 .

[0080] 3. Sensor Model of Roadside Sensing Devices

[0081] Since the traffic scenario in the present invention is created on the basis of fusing 3D real-scene point cloud data and traffic simulation, the sensor model in the existing traffic simulation software cannot be called. Therefore, the present invention proposes a sensor simulation model for the virtual-real fusion traffic scenario.

[0082] The static road 3D environment is characterized by real-scene point cloud data, and traffic individuals are characterized by triangular mesh models. Considering the heterogeneity of the data, the present invention constructs sensor models for the static road environment and dynamic traffic individuals respectively. Let C s be the coordinates of the center of a single roadside sensing sensor, and α, β, and γ be the pitch angle, roll angle, and azimuth angle of the roadside sensing device respectively, where the azimuth angle is based on the due north direction. Let P e be the 3D real-scene point cloud data of the road environment, let P a be the vertex coordinate data constituting the 3D model of the traffic individual, let VI a be the index N a ×3 matrix of the triangular mesh, where N a is the number of vertices of the triangular mesh model.

[0083] The present invention proposes a sensor data acquisition model based on spherical coordinate projection, and the specific steps are as follows:

[0084] Step 1: Coordinate space transformation. Map both the static road environment and the traffic individual model in the virtual-real fusion scenario to the spherical coordinate space through coordinate transformation, and the formulas are shown in Equations (1) to (3). When performing the coordinate transformation of the traffic individual model, the structural relationship of the mesh model is not changed.

[0085]

[0086]

[0087] Where:

[0088] M 3×3 (α,β,γ)·(P e-C s ) is to convert the global coordinates to the local coordinates of the sensor, see Figure 4

[0089] C2S(.) is a function that converts Cartesian coordinates to spherical coordinates

[0090] are the spherical coordinates of the static road three-dimensional environment after conversion

[0091] are the spherical coordinates of the traffic individual model after conversion

[0092] φ and r are the coordinate components of the horizontal angle, pitch angle and distance in the spherical coordinates respectively.

[0093] Step 2: Preliminary data filtering. Set and [φ min , φ max are the horizontal angle range and vertical angle range of the sensing field of view of the roadside sensing device respectively. Taking φ, r as the coordinate components, a new Cartesian coordinate system is established. Project the road environment data and traffic individual model data after coordinate conversion onto the plane, and only retain the data within the sensing field of view of the roadside sensing device. In this process, for the traffic individual triangular mesh model, only when all three vertices forming the triangular mesh are outside the sensing field of view will the corresponding vertex data be eliminated, as Figure 5 shown.

[0094] Step 3: Sensing process modeling. On the plane, rasterize the projected road static environment data , and the pixel sizes along the direction and along the φ direction are and ε φ respectively, and obtain the depth perception map I e of the road static environment.

[0095] On the plane, the traffic individual is still a triangular mesh model, and the vertices of the triangle are relatively discrete, and it is impossible to directly obtain the depth map of the traffic individual using the rasterization method. The present invention proposes a new ray-triangle intersection solution method to realize the acquisition of traffic individual data by the sensor.

[0096] For the simulation of the three-dimensional sensing process of the sensor for traffic individuals, the industry mainly uses ray-tracing technology at present. Taking the sensor center C sStarting from , a large number of virtual rays are constructed, and the acquisition mechanism of traffic individual perception data is simulated by calculating the intersection points of the rays and the traffic individual model. Since the number of virtual rays to be constructed is huge, currently, a graphical processing unit (GPU) that relies on highly parallelized computing is needed to accelerate the ray-tracing process. During the calculation process, although the number of virtual rays is huge, the number of triangular units of the traffic individual model is limited. Therefore, most virtual rays will not effectively collect data, resulting in a waste of computing resources. The present invention proposes a ray-triangle search method based on an "encoding - reorganization - decoding" mechanism to reduce the construction of invalid virtual rays, and then concentrate the computing power resources on the rays that are more likely to intersect with the traffic individual triangular mesh.

[0097] Through the coordinate transformation method in the present invention, in The mapping relationship of virtual rays in the rectangular coordinate system and the local coordinate system of the sensor is as Figure 6 shown. On this basis, in A point on the plane is equivalent to a virtual ray in the local coordinate system of the sensor. On the plane, rasterization processing consistent with is performed on the projected traffic individual triangular mesh vertex data , and the pixel size also remains the same. After rasterization, each pixel point corresponds to a virtual ray.

[0098] Taking the triangular unit k in the traffic individual grid model as an example, an envelope box of triangle k is established based on the triangular vertices after rasterization. As Figure 7 shown, any pixel or grid point in the envelope box is encoded and concatenated. At the same time, in order to retain the information of triangle index k, a one-dimensional array with the same length as the concatenated encoding is established. Let (m, n) be the matrix position of a certain pixel in the envelope box, and it is linearly encoded using Equation (4). The advantage of this encoding method is If (m 1 , n 1 ) ≠ (m 2 , n 2 ), then there is

[0099]

[0100] where:

[0101]

[0102] where:

[0103] mod(.) and rem(.) are the modulo and remainder calculation functions respectively

[0104] M is the number of rows of the rasterized matrix

[0105] is the linear encoding value

[0106] Through the above process, the grid points within the two-dimensional bounding box are transformed into a 1×m k ×2 matrix, where m k is the number of grid points within the bounding box. To convert all triangles more efficiently, first pre-establish an n f ×m k_max ×2 empty matrix on the GPU, where n f and m k_max are the number of triangular elements and the m value corresponding to the largest triangle respectively k value. Then as Figure 8 shown, all triangles are converted into matrices through parallel computing on the GPU. Then, the layers with depth 1 and depth 2 in the n f ×m k_max ×2 matrix are concatenated respectively to obtain a (n f ·m k_max )×1 linear encoding matrix and a triangle index matrix

[0107] Reorder and organize the linear encoding. The triangle index matrix also adjusts the positions of the corresponding elements along with the reorganization of the linear encoding matrix. The same pixel point may correspond to multiple virtual rays. Through the sorting operation, multiple triangle indices corresponding to the same linear encoding value can be obtained. Through decoding calculation, is converted into the position (m, n) of the pixel point, and there is a one-to-one mapping relationship between (m, n) and the virtual ray, so that pairs of virtual rays (ray) and triangular elements (triangle) that may intersect can be obtained, thereby significantly reducing the number of invalid virtual rays and improving the calculation efficiency

[0108] Finally, a spatial geometry analysis method is used to solve the intersection points of the paired virtual rays and triangular elements. For the case where the same ray corresponds to multiple triangles, only the nearest data points are retained according to the perception principle. On the same computing device (memory 16GB, CPU Core TM i7-12700H, GPU RTX 3060), the method in the present invention is compared with the traditional GPU calculation method, and the running time is shown in Figure 10 . When the combination number N c of triangle-virtual ray is greater than 10 9 , the method in the present invention has significant advantages

[0109] The depth perception map I of the traffic individual triangular grid model is obtained through the above calculation process a .

[0110] Step 4: Perception data fusion. Set Ω s to represent the configuration parameter set {C s , α, β, γ} of a single roadside perception device. Execute Step 3 to obtain I s and I e respectively under the condition of Ω a . As Figure 11 shown, considering the occlusion effect of sensor perception, that is, the roadside perception device cannot obtain data in the occluded area, fuse I e and I a to obtain the depth perception map of traffic individuals under static and dynamic occlusion conditions. The formula is as shown in Equation (6).

[0111]

[0112] Wherein:

[0113] I a (m, n) and I e (m, n) are the pixel points of the m-th row and the n-th column of I a and I e respectively

[0114] I' a is the depth perception map of traffic individuals after fusion

[0115] 4. Cooperative perception efficiency evaluation

[0116] Use the inverse calculation process of Equation (1) to re-project the fused depth perception map I' a of traffic individuals back into the global coordinate system to obtain the perception data acquisition result of the roadside sensor for traffic individuals. The more data points, the better the perception effect on traffic individuals. For the modeling of the cooperative perception process of multiple roadside perceptions, apply the sensor models of roadside perception devices respectively, and combine the traffic individual data points in the global coordinate space to obtain the data acquisition result under cooperative perception conditions

[0117] The present invention uses the intersection over union parameter to quantitatively evaluate the perception efficiency. Use traffic simulation software to adjust traffic flow parameters and simulation duration. At any simulation time step, execute the roadside cooperative perception process to obtain the data acquisition result under cooperative perception. Based on the traffic individual data obtained by cooperative perception, fit a cuboid bounding box, and compare it with the original bounding box (obtained from the size of the traffic individual three-dimensional grid model) to calculate the intersection over union parameter. Take the average intersection over union parameter of all traffic individuals under multiple simulation steps as the evaluation index of the roadside cooperative perception efficiency

[0118] 5. Optimization of the Layout of Cooperative Sensing Devices

[0119] The process of optimizing the layout of roadside cooperative sensing devices in the present invention is as Figure 12 shown. A traffic scenario is created through a virtual-real fusion simulation method, and the configuration parameter set Ω s of multiple roadside sensing devices is used as a decision variable. The sensing efficiency index is obtained through continuous virtual-real fusion simulation and the application of a sensor simulation model. With the improvement of sensing efficiency as the optimization goal, the layout parameters of roadside sensing devices are continuously iterated through a Bayesian optimizer. Optimization constraints can be added during this process, such as the target sensing efficiency index or the upper limit of the number of roadside sensing units. The optimized layout result of roadside cooperative sensing devices can be obtained after the optimization process ends.

[0120] References

[0121] [1] Dosovitskiy A, Ros G, Codevilla F, et al. CARLA: An open urban driving simulator[C] / / Conference on robot learning. PMLR, 2017: 1-16.

[0122] [2] Lopez P A, Behrisch M, Bieker-Walz L, et al. Microscopic traffic simulation using sumo[C] / / 2018 21st international conference on intelligent transportation systems(ITSC). IEEE, 2018: 2575-2582.

[0123] [3] Vijay R, Cherian J, Riah R, et al. Optimal placement of roadside infrastructure sensors towards safer autonomous vehicle deployments[C] / / 2021 IEEE International Intelligent Transportation Systems Conference(ITSC). IEEE, 2021: 2589-2595.

[0124] [4]Du Y, Wang F, Zhao C, et al. Quantifying the performance and optimizing the placement of roadside sensors for cooperative vehicle-infrastructure systems[J]. IET Intelligent Transport Systems, 2022, 16(7): 908-925. https: / / ietresearch.onlinelibrary.wiley.com / doi / 10.1049 / itr2.12185

[0125] [5]Ma Y, Zheng Y, Wang S, et al. Point cloud-based optimization of roadside LiDAR placement at constructed highways[J]. Automation in Construction, 2022, 144: 104629.

[0126] [6]Ma Y, Zheng Y, Wang S, et al. A Virtual Method for Optimizing Deployment of Roadside Monitoring Lidars at As-Built Intersections[J]. IEEE Transactions on Intelligent Transportation Systems, 2023.

[0127] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and the inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. Optimization layout method for roadside perception devices integrating real - world point cloud and virtual simulation, characterized in that, it includes the following steps: (1) Create a file that meets the requirements of traffic simulation software according to the basic road geometric information in the 3D real - world point cloud data; establish a traffic individual model database, and obtain the original bounding box according to the size of each traffic individual triangular network model in the traffic individual model database; (2) Create a virtual - real fusion traffic scene Import the basic road geometric information file into the traffic simulation software, establish a road network model consistent with the coordinate space of the real - world point cloud data, link the traffic individual model database based on the traffic individual movement trajectory, and use the 3D real - world scene as the static road environment to create a virtual - real fusion traffic scene; (3) Establish a roadside perception device sensing model based on spherical coordinate projection Take the center of a single roadside perception sensor as the origin to establish a local coordinate system of the sensor. Through coordinate transformation, map both the static road environment and the traffic individual triangular network model in the virtual - real fusion scene in the global coordinate system to the local coordinate system of the sensor; use the sensor to perform 3D perception on the static road environment and the traffic individual in the local coordinate system of the sensor respectively, obtain the depth perception maps of the static road environment and the traffic individual triangular mesh model respectively, and fuse the two depth perception maps; (4) Cooperative perception efficiency evaluation Reproject the fused depth perception map back to the global coordinate system to obtain the perception data acquisition result of the roadside sensor for the traffic individual triangular mesh model; for the modeling of the cooperative perception process of multiple roadside perception sensors, apply the roadside perception device sensor model to each roadside perception sensor respectively, and combine the traffic individual data points in the global coordinate space to obtain the data acquisition result under cooperative perception conditions; Obtain the bounding box under roadside perception conditions according to the perception data acquisition result, and calculate the intersection - over - union parameter by comparing it with the original bounding box; use the average intersection - over - union parameter of all traffic individuals as the evaluation index of roadside cooperative perception efficiency; (5) Optimization layout of cooperative perception devices Take the configuration parameter sets of multiple roadside perception devices as decision variables, and obtain the perception efficiency index through continuous virtual - real fusion simulation and applying the sensor simulation model; take improving the perception efficiency as the optimization goal, and continuously iterate the layout parameters of the roadside perception devices through a Bayesian optimizer.

2. The optimization layout method for roadside perception devices integrating real - world point cloud and virtual simulation according to claim 1, characterized in that, the basic road geometric information includes lane centerlines, road boundary lines, and lane widths.

3. The optimization layout method for roadside perception devices integrating real - world point cloud and virtual simulation according to claim 1, characterized in that, The establishment of the traffic individual model database is specifically as follows: Obtain the triangular mesh models of different traffic individuals through an open - source network database, record the geometric size information of each model, and form a traffic individual model database together with the original mesh model; the triangular mesh model is defined by vertex coordinates and the vertex index matrix of the triangles that make it up.

4. The optimization layout method for roadside perception devices integrating real - world point cloud and virtual simulation according to claim 1, characterized in that, Both the static road environment and the traffic individual model in the virtual-real fusion scenario are mapped to the spherical coordinate space as follows: Let C s be the coordinates of the center of a single roadside perception sensor, and α, β, and γ be the pitch angle, roll angle, and azimuth angle of the roadside perception device, respectively, where the azimuth angle is based on the true north direction. Let P e be the 3D real-world point cloud data of the road environment, and let P a be the vertex coordinate data that make up the 3D model of the traffic individual. Let VI a be the index N a ×3 matrix, where N a is the number of vertices of the triangular mesh model; Both the static road environment and the traffic individual model in the virtual-real fusion scenario are mapped to the spherical coordinate space through coordinate transformation. The formulas are shown in Equations (1) to (3): where: M 3×3 (α, β, γ) · (P e - C s ) converts the global coordinates to the sensor local coordinates; C2S(.) is a function that converts Cartesian coordinates to spherical coordinates; are the spherical coordinates of the static road three-dimensional environment after conversion; are the spherical coordinates of the traffic individual model after conversion; φ and r are the coordinate components of the horizontal angle, elevation angle, and distance in spherical coordinates, respectively.

5. The method for optimizing the layout of roadside perception devices that fuses real-scene point clouds and virtual simulations according to claim 4, wherein, Three-dimensional perception of the static road environment and traffic individuals is performed using sensors as follows: (5.1) Setting With [φ min , φ max being the horizontal angle range and the vertical angle range of the sensing field of view of the roadside sensing device respectively; establish a new Cartesian coordinate system with φ, r as coordinate components; both the road environment data and the traffic individual model data after coordinate transformation are projected onto the plane, and only the data within the sensing field of view of the roadside sensing device is retained (5.2) On a plane, rasterize the projected road static environment data along the direction and the pixel sizes along the φ direction are respectively and ε φ , and obtain the road static environment depth perception map I e ; (5.3) On a plane, the ray-triangle intersection solution method is used to collect traffic individual data by the sensor, and the depth perception map I of the traffic individual triangular mesh model is obtained a .

6. The method for optimizing the layout of roadside perception devices that fuses real-scene point clouds and virtual simulations according to claim 5, wherein, The three-dimensional perception process of traffic individuals using the ray-triangle intersection solution method is as follows: (6.1) On the vertex data of the projected traffic individual triangular grid on the plane is subjected to rasterization processing consistent with the road static environment data and the pixel size is also kept consistent; after rasterization, each pixel point corresponds to a virtual ray; (6.2) Establish an envelope box for the triangular vertices after rasterizing individual traffic elements. Perform encoding processing on any pixel or grid point in the envelope box and concatenate them. At the same time, to retain the information of the triangle index k, establish a one-dimensional array with the same length as the concatenated encoding. Convert the grid points within the two-dimensional envelope box into a 1×m k ×2 matrix, where m k is the number of grid points within the envelope box; Let (m, n) be the matrix position of a certain pixel within the bounding box, and linear encoding is performed on it using Equation (4); where: Where: mod(.) and rem(.) are the modulo and remainder calculation functions respectively; M is the number of rows of the rasterized matrix; is the linear coding value; (6.3) Pre-establish an empty matrix of n f ×m k_max ×2 on the GPU, where n f and m k_max are the number of triangular elements and the m corresponding to the largest triangle respectively k value. Convert all triangles into matrices through parallel computing; then concatenate the layers with depth 1 and depth 2 in the n f ×m k_max ×2 matrix respectively to obtain a linear encoding matrix of (n f ·m k_max )×1 and a triangular index matrix; (6.4) Reorder and organize the linear encoding, and obtain multiple triangle indices corresponding to the same linear encoding value through the sorting operation; through decoding calculation, convert into the position (m, n) of the pixel point, and there is a one-to-one mapping relationship between (m, n) and the virtual ray, then the virtual ray and triangle unit pairs that may intersect are obtained; (6.5) Use the spatial geometric analysis method to solve the intersection points of paired virtual rays and triangular elements. For the case where the same ray corresponds to multiple triangles, only the nearest data points are retained according to the perception principle; obtain the depth perception map I of the traffic individual triangular grid model a 。 7. The method for optimizing the layout of roadside perception devices that fuses real-scene point clouds and virtual simulations according to claim 6, wherein, During the fusion process of the two depth perception maps, the occlusion effect of sensor perception needs to be considered, that is, the roadside perception device cannot obtain data in the occluded area: Set Ω s represents the set of configuration parameters {C s , α, β, γ} of a single roadside perception device; perform step 3 to obtain the road static environment depth perception map I s under the condition of Ω e and the depth perception map I of the traffic individual triangular grid model a ; for the road static environment depth perception map I e and the depth perception map I of the traffic individual triangular grid model a are fused to obtain the depth perception map of traffic individuals under static and dynamic occlusion conditions, and the formula is as shown in Equation (6): Where: I a (m, n) and I e (m, n) are respectively the a and I e pixel points at the m-th row and the n-th column; I' a is the depth perception map of individual traffic after fusion.

Images