Three-dimensional roadside sensor deployment method for collaborative sensing in internet of vehicles

Abstract

Disclosed in the present invention is a three-dimensional roadside sensor deployment method for collaborative sensing in the Internet of Vehicles. The method comprises: modeling an RSN model of a three-dimensional road environment; on the basis of the RSN model, constructing sensing detection functions for single sensors, and when there are a plurality of sensors providing coverage, constructing a global sensor detection function for each deployable area on the basis of the sensing detection functions for the single sensors; taking the maximization of the proportion of deployable areas that meets or exceeds a minimum detection probability as a first objective optimization problem function, which maximization is based on the global sensor detection function, taking the minimization of the total number of sensors in each deployable area as a second objective optimization problem function, and constructing a deployment optimization reward function consisting of the first objective optimization problem function and the second objective optimization problem function; and constructing a decision sequence, and on the basis of a decision transformer, solving the decision sequence to obtain a deployment policy, wherein a cumulative reward value is obtained by performing reward reallocation on the deployment optimization reward function by means of a two-layer optimization method. The present invention implements sensor deployment in a three-dimensional road environment.

Description

A method for deploying 3D roadside sensors for collaborative perception in vehicle-to-everything (V2X) networks Technical Field

[0001] This invention belongs to the field of vehicle networking, and specifically relates to a method for deploying three-dimensional roadside sensors for collaborative perception in vehicle networking. Background Technology

[0002] In the connected vehicle environment, collaborative perception technology is crucial for the safety and efficiency of autonomous vehicles. Connected vehicles achieve collaborative perception by deploying multi-source sensors (such as cameras, LiDAR, and radar) on both sides of the road and sharing this perception data with autonomous vehicles in real time, significantly improving the vehicles' perception capabilities in complex road environments. Autonomous and intelligent connected vehicles need to achieve efficient and safe driving in various complex scenarios, which places higher demands on sensor deployment.

[0003] Existing technologies primarily focus on 2D sensor deployment optimization, static 3D scene modeling, and local optimization algorithms. Traditional 2D sensor deployment optimization methods are typically based on simple geometric models, optimizing sensor positions and angles to cover the target area and maximizing sensing coverage by calculating the optimal sensor position and viewing angle. However, these methods fail to consider the complex 3D road environment and cannot solve the blind spot problem in practical applications. Some research attempts to extend sensor deployment to 3D scenes, but these often employ idealized 3D models, ignoring the actual complexity of roads and real-world constraints in sensor deployment. While these methods perform well in experimental environments, their effectiveness is often limited in real-world road environments, struggling to cope with the complexity and diversity of actual roads. Traditional 2D and idealized 3D scene sensor deployment methods typically employ static optimization algorithms, such as particle swarm optimization and genetic algorithms. These methods perform well in initial deployment but lack dynamic adaptability to environmental changes. In practical applications, changes in traffic conditions and environmental conditions affect sensor performance, and static optimization algorithms struggle to adjust deployment strategies in real time, leading to decreased sensing coverage and accuracy. For example, patent CN113743479B proposes an edge-cloud vehicle-road cooperative fusion perception architecture that optimizes information flow through multi-agent distributed coordination theory. While this method enhances the system's environmental adaptability, it still faces challenges in handling real-time data transmission and processing in highly dynamic traffic scenarios. Furthermore, patent CN117294738B proposes an optimized deployment and perception method for autonomous driving sensors, enhancing the system's perception capabilities by optimizing the spatial coverage and positional weights of sensors. However, this method still has limitations when dealing with complex and dynamically changing road environments.

[0004] In summary, existing collaborative sensing technologies face numerous challenges in sensor deployment, especially in complex and ever-changing urban road environments. Traditional two-dimensional or idealized three-dimensional scene modeling methods struggle to cope with the complexities of real-world roads, resulting in limitations in coverage, sensing accuracy, and real-time performance of existing sensor networks. Furthermore, traditional two-dimensional and idealized three-dimensional scene sensor deployment methods typically employ static optimization algorithms, lacking dynamic adaptability to environmental changes and failing to provide reliable sensing assistance under various unexpected road conditions and weather variations. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) networks. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] This invention provides a method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) networks, the method comprising:

[0007] Modeling includes RSNs models of the 3D road environment, comprising a set of deployable sensors, a set of deployable sensor locations, a set of deployable areas, and a set of restricted areas;

[0008] Based on the RSNs model of the three-dimensional road environment, a perception detection function for a single sensor in each deployable area is constructed. When multiple sensors cover each deployable area, a global sensor detection function for each deployable area is constructed based on the perception detection function of the single sensor.

[0009] The first objective optimization problem function is to maximize the proportion of deployable areas that meet or exceed the minimum detection probability according to the global sensor detection function, and the second objective optimization problem function is to minimize the total number of sensors in each deployable area. A deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function is constructed through a balance coefficient.

[0010] A decision sequence consisting of a state space, an action space, and a cumulative reward value is constructed, and a deployment strategy is obtained by solving the decision sequence based on a decision converter. The state space includes a set of deployable sensors, a set of deployable sensor locations, a set of deployable regions, a set of restricted regions, and a set of minimum detection probabilities. The action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors. The cumulative reward value is obtained by adaptively redistributing the deployment optimization reward function through a two-layer optimization method.

[0011] In one embodiment of the present invention, constructing a sensing and detection function for a single sensor includes:

[0012] Four sub-sensing detection functions are constructed to determine the detection probability of a single sensor, which are affected by distance, horizontal detection viewpoint, vertical detection viewpoint, and obstacles, respectively.

[0013] The perception and detection function of a single sensor is constructed by combining four sub-perception and detection functions.

[0014] In one embodiment of the present invention, the sub-sensing detection function affected by distance is expressed as follows:

[0015] in, This represents a sub-sensory detection function affected by distance. Indicates sensor s i,j The reference location information, Indicates sensor s i,j The actual location information, α represents The slope adjustment parameter, β, represents the maximum effective distance of the sensor;

[0016] The sub-sensory detection function affected by the horizontal detection viewpoint is expressed as follows:

[0017] in, This represents the sub-sensory detection function affected by the horizontal detection viewpoint. Indicates sensor s i,j The baseline horizontal angle, Indicates sensor s i,j The actual horizontal angle, ι represents The slope adjustment parameter, η, represents the maximum detection angle of the sensor;

[0018] The sub-perceptual detection function affected by the vertical detection viewpoint is expressed as follows:

[0019] in, This represents the sub-sensory detection function affected by the vertical detection viewpoint. Indicates sensor s i,j The reference vertical angle, Indicates sensor s i,j The actual vertical angle, ι represents The slope adjustment parameter, express The value at 50% of maximum coverage;

[0020] The sub-sensor detection function affected by obstacles is expressed by the following formula:

[0021] in, Let G represent the sub-sensing detection function affected by obstacles, and let G represent the obstacle adjustment parameter.

[0022] In one embodiment of the present invention, the sensing and detection function of a single sensor is expressed by the formula:

[0023] Where C(AiT,si,j) represents the sensing and detection function of a single sensor, i.e. Represents the set of deployable regions A T Deployable areas Sensors i,j The perception detection function, N T Indicates the number of deployable regions. This represents the i-th deployable region;

[0024] When multiple sensors are in coverage, a global sensor detection function is constructed for each deployable area, expressed as follows:

[0025] in, This represents the global sensor detection function, which can be used to deploy a set of regions. The sensing and detection functions of the sensor set si can be deployed in the middle. s i,j s i The j-th deployable sensor in the middle, Indicates deployable area The maximum number of sensors that can be deployed in the system.

[0026] In one embodiment of the present invention, the first objective optimization problem function is expressed as follows:

[0027] Where C1 represents the first objective optimization problem function, card{} represents finding the number of elements in the set, and A T Indicates a set of deployable regions. N T Indicates the number of deployable regions. Let P represent the i-th deployable region, and let P represent the set of minimum detection probabilities. Indicates deployable area The minimum detection probability, s i Indicates deployable area Sensor arrays can be deployed in the middle. Indicates deployable area The global sensor detection function;

[0028] The second objective optimization problem function is expressed as follows:

[0029] in, Indicates deployable area The number of sensors that can be deployed in the middle Indicates deployable area The set of locations where sensors can be deployed, Indicates deployable area The position information of the j-th sensor.

[0030] In one embodiment of the present invention, an adaptive reward redistribution of the deployment optimization reward function is achieved through a two-layer optimization method, including:

[0031] The initial cumulative reward value of the sensor deployment trajectory is calculated based on the deployment optimization reward function, and a proxy reward function is introduced to re-evaluate the reward of each time step in the sensor deployment trajectory to obtain the reshaping reward value. The reshaping reward function is constructed using the initial cumulative reward value and the reshaping reward value.

[0032] By using the reshaping reward function as the third objective optimization problem function and the validation loss function as the fourth objective optimization problem function, a two-level optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed by balancing the weight coefficients.

[0033] The cumulative reward value after redistribution is obtained by solving the bi-level optimization problem function using a bi-level optimization method. In the upper-level optimization process, the third objective optimization problem function is optimized, and in the lower-level optimization process, the fourth objective optimization problem function is optimized. Completing one upper-level and lower-level optimization achieves one single-step optimization.

[0034] In one embodiment of the present invention, the third objective optimization problem function is expressed as follows:

[0035] Where C3 represents the third objective optimization problem function, Let R represent the validation set, δ represent the sensor deployment trajectory, |δ| represent the length of the sensor deployment trajectory, and R ep (δ) represents the cumulative reward value of the sensor deployment trajectory. This represents the reshaping reward value at time step t;

[0036] The fourth objective optimization problem function is expressed as follows:

[0037] Where C4 represents the fourth objective function, Let ξ*(ζ) represent the validation loss function, ξ*(ζ) represent the policy model to be solved, and ζ represent the parameters of the reward model.

[0038] In one embodiment of the present invention, during the upper and lower layer optimization processes, an alternating optimization method is used to optimize the bi-layer optimization problem function; wherein...

[0039] During the optimization process, the policy model for the current single-step optimization is solved by training the loss function, which serves as an approximate solution for the policy model to be solved in the validation loss function. This approximate solution is then used as the policy model to be solved in the next single-step optimization to update the validation loss function. The validation loss function is then returned as the fourth objective optimization problem function until the stopping condition is met.

[0040] In one embodiment of the present invention, the current single-step optimized policy model is solved by training the loss function, which serves as an approximate solution for the policy model to be solved by verifying the loss function. The formula is expressed as follows:

[0041] Where ξ′(ζ) represents the current single-step optimization strategy model, ξ(ζ) represents the current strategy model, and α ξ This represents the learning rate for the current single-step optimization. This indicates the gradient calculation operation. This represents the training loss function.

[0042] The beneficial effects of this invention are:

[0043] This invention proposes a 3D roadside sensor deployment method for collaborative perception in vehicle-to-everything (V2X) networks. It constructs a Road Sensor Network (RSN) model that accurately reflects complex 3D road environments and designs a sensor deployment method based on a decision converter. This method utilizes the Transformer model and multi-head attention mechanism within the decision converter to dynamically adapt to various road conditions and environmental changes, effectively solving the problem of deploying sensors in complex and constantly changing 3D road environments. Furthermore, to improve the performance of the decision converter, this invention introduces a two-layer optimization technique. This technique optimizes the cumulative reward in the decision sequence of the decision converter, allowing for real-time adjustments to the deployment scheme to adapt to complex road and environmental changes. In summary, this invention improves the coverage and perception accuracy of 3D road environments while reducing deployment costs, enhancing economic efficiency and practical application effectiveness, thereby significantly improving the safety and efficiency of autonomous and intelligent connected vehicles in complex environments.

[0044] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0045] Figure 1 is a flowchart illustrating a method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) networks, according to an embodiment of the present invention.

[0046] Figure 2 is a schematic diagram of the RSNs model of the three-dimensional road environment provided in an embodiment of the present invention;

[0047] Figure 3 illustrates how the coverage rate varies with the number of sensors and lanes according to an embodiment of the present invention.

[0048] Figure 4 is a schematic diagram showing how the sensing accuracy varies with the number of deployed sensors according to an embodiment of the present invention. Detailed Implementation

[0049] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0050] Please refer to Figure 1. This embodiment of the invention provides a method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication, specifically including the following steps:

[0051] S10. Modeling an RSNs model of a 3D road environment, including a set of deployable sensors, a set of deployable sensor locations, a set of deployable areas, and a set of restricted areas.

[0052] The RSNs model of the three-dimensional road environment modeled in this embodiment of the invention is shown in Figure 2, which may include a set of deployable sensors, a set of deployable sensor locations, a set of deployable areas, and a set of restricted areas.

[0053] Assume the RSNs model consists of a set of N deployable sensors, denoted as S. N T Indicates the number of deployable regions, s i This represents the set of deployable sensors for the i-th deployable region. s i,j s i The j-th deployable sensor in the middle, This represents the maximum number of deployable sensors in the i-th deployable region; N represents the maximum number of sensors that can be deployed. T A is a set of deployable regions consisting of _ ... T , This represents the i-th deployable region, where each deployable region is a rectangular area. Defined as the coordinates of the angle opposite its diagonal. Indicates deployable area The coordinates of the top left corner Indicates deployable area The coordinates in the lower right corner define the boundaries and size of the deployable area, which helps to accurately assess coverage requirements. To ensure the reliability of the autonomous driving system, it is necessary to ensure that the perception probability within the deployable area is not lower than the set threshold.

[0054] In real-world road environments, not all areas above and along roads are suitable locations for sensor deployment; for example, roadside green belts and guardrails may be suitable. Therefore, when considering sensor deployment, areas along roads where sensor installation is explicitly prohibited need to be taken into account. Assume that the RSNs model in this embodiment of the invention includes: N R A is a set of restricted regions consisting of several restricted regions. R , This represents the k-th restricted region, where each restricted region is a rectangular region. Indicates restricted area The coordinates of the top left corner Indicates restricted area The coordinates in the lower right corner precisely define the location and size of the restricted area, ensuring that no type of sensor is installed within this restricted area. Establishing this restricted area is crucial for preventing interference with critical operations or for areas where sensors are sensitive.

[0055] The sensors required for collaborative sensing are strategically positioned on either side or above the road to ensure that normal traffic flow is not affected. Assume that the RSNs model in this embodiment of the invention includes: N T The set of deployable sensor locations, consisting of _p_deployable sensor locations, is denoted as p_deployable sensor locations. D , Indicates deployable area The set of locations where sensors can be deployed, Indicates deployable area The position information of the j-th sensor, such as its three-dimensional coordinates, can be denoted as... Considering the impact of sensor deployment height, here The function represents the position (x) D ,y D The horizontal height function at point z i,j This indicates the altitude of the deployed sensors above the ground. Embodiments of the present invention enhance the integration and analysis of data collection by determining the exact location of each sensor within the deployable area.

[0056] S20. Based on the RSNs model of the three-dimensional road environment, construct the perception and detection function of a single sensor, and when multiple sensors are in coverage, construct the global sensor detection function based on the perception and detection function of the single sensor.

[0057] Actual sensor behavior is not solely determined by the distance between the sensor and the target and the area covered by the sensor; the perception model is primarily influenced by factors including perception distance, horizontal viewing angle, vertical viewing angle, and obstacles. Therefore, embodiments of this invention construct a perception detection function for a single sensor, including:

[0058] Four sub-sensory detection functions are constructed to illustrate how the detection probability of a single sensor is affected by distance, horizontal detection viewpoint, vertical detection viewpoint, and obstacles. These four sub-sensory detection functions are then combined to construct the overall perception detection function for a single sensor. Specifically:

[0059] In autonomous driving systems, sensor efficiency is typically evaluated by its perception detection probability. Notably, the sensor's perception detection capability tends to decrease as the perception detection distance increases. A key factor influencing this pattern is the Euclidean distance used to quantify the separation between the sensor and its target, ultimately leading to a distance-dependent sub-perception detection function, expressed as:

[0060] in, This represents a sub-sensory detection function affected by distance. Indicates sensor s i,j The reference location information, Indicates sensor s i,j The actual location information, α represents The slope adjustment parameter plays an important role in the slope of the adjustment function, affecting the degree to which the sensing and detection probability decreases rapidly with increasing distance. β represents the maximum effective distance of the sensor.

[0061] In the field of sensing and detection using complex sensors, both the sensing and detection distance and the horizontal deviation of the sensor's central axis are crucial. As the horizontal detection viewpoint deviates from the central axis, the sensing and detection probability typically decreases. The final sub-sensing and detection function influenced by the horizontal detection viewpoint is expressed as follows:

[0062] in, This represents the sub-sensory detection function affected by the horizontal detection viewpoint. Indicates sensor s i,j The baseline horizontal angle, Indicates sensor s i,j The actual horizontal angle, ι represents The slope adjustment parameter reflects the rate at which the sensing and detection probability decreases as the horizontal field of view widens. η represents the maximum detection angle of the sensor; beyond this angle, the sensing and detection probability drops sharply.

[0063] Similar to horizontal perspective perception, vertical detection perspective also affects the detection probability. This probability is related to the sensor's central axis, meaning that objects are more easily detected when close to the vertical central axis. The final sub-sensory detection function affected by the vertical detection perspective is expressed as:

[0064] in, This represents the sub-sensory detection function affected by the vertical detection viewpoint. Indicates sensor s i,j The reference vertical angle, Indicates sensor s i,j The actual vertical angle, ι represents The slope adjustment parameter reflects the rate at which the probability of perception detection decreases as the vertical viewing angle widens. express The value at 50% of the maximum coverage.

[0065] Sensors in autonomous vehicle systems excel at environmental perception, but may encounter challenges that hinder their perception capabilities. Obstacles, whether physical barriers or adverse weather conditions, can significantly affect line-of-sight or perception detection quality, leading to a decrease in the probability of perception detection within a predetermined detection area. Ultimately, this embodiment of the invention uses a low-complexity Bresenham algorithm to construct the impact of obstacles on the perception detection probability. The constructed sub-perception detection function affected by obstacles is expressed as follows:

[0066] in, Let G represent the sub-sensory detection function affected by obstacles, and let G be the obstacle adjustment parameter, a variable determined by the type of obstacle. For example, G might be 0.9 for a transparent plastic bag and 0.1 for a red plastic bag. The sub-sensory detection function affected by obstacles illustrates the change in perception capability caused by obstacles. The impact of the presence of obstacles on the perception detection probability is quantified by the sub-sensory detection function affected by obstacles. In contrast, when there are no obstacles, the probability is 1, indicating unimpeded perception.

[0067] Ultimately, the sensing and detection capability of a single sensor in this embodiment of the invention is constructed from four sub-sensing and detection functions affected by distance, horizontal detection angle, vertical detection angle, and obstacles. The constructed sensing and detection function of a single sensor is expressed by the following formula:

[0068] in, The sensing and detection function of a single sensor, i.e. Represents the set of deployable regions A T Deployable areas Sensors i,j The perception detection function.

[0069] In autonomous driving systems, a single sensor has inherent limitations in its coverage area. When sensors work together, the probability of perception and detection within the target area increases significantly. While a single sensor may only cover a portion of the target area, the combined use of multiple sensors ensures full coverage, thereby improving the likelihood of accurate perception and detection throughout the entire target area. The global sensor detection function, constructed when the target area is covered by multiple sensors, is expressed as:

[0070] in, This represents the global sensor detection function, i.e., the function over the set of deployable areas. A set of sensors can be deployed in the middle. i The perception detection function, s i,j s i The j-th deployable sensor in the middle, Indicates deployable area The maximum number of sensors that can be deployed in the system.

[0071] S30. The first objective optimization problem function is to maximize the proportion of deployable areas that meet or exceed the minimum detection probability according to the global sensor detection function. The second objective optimization problem function is to minimize the total number of sensors in each deployable area. A deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function is constructed through a balance coefficient.

[0072] However, achieving such broad sensing coverage presents challenges, particularly due to budget constraints. Therefore, constraints are introduced, emphasizing the balance between maximizing sensing detection capability and cost-effectiveness. The primary objective is to achieve the highest detection coverage through multi-sensor collaboration while efficiently utilizing available resources in an economical manner. Thus, the objective of this invention is to develop a sensor deployment strategy for 3D RSNs that maximizes sensing assistance while minimizing the number of sensors deployed, considering both deployable and restricted areas—that is, covering the deployable area with a single sensor network while ensuring that the sensing probability within that deployable area is not less than the minimum detection probability for each deployable area. The threshold set in the method is used to avoid placing sensors in restricted areas. Finally, in this embodiment of the invention, the proportion of deployable areas that maximize or exceed the minimum detection probability according to the global sensor detection function is used as the first objective optimization problem function, expressed by the formula:

[0073] Where C1 represents the first objective optimization problem function, card{} represents finding the number of elements in the set, and A T Indicates a set of deployable regions. NT Indicates the number of deployable regions. Let P represent the i-th deployable region, and let P represent the set of minimum detection probabilities. Indicates deployable area The minimum detection probability, Indicates deployable area The global sensor detection function.

[0074] Simultaneously, minimizing the total number of sensors in each deployable area is taken as the second objective optimization problem function, expressed by the formula:

[0075] in, Indicates deployable area The number of sensors that can be deployed in the middle Indicates deployable area The set of locations where sensors can be deployed, Indicates deployable area The position information of the j-th sensor.

[0076] A deployment optimization reward function, composed of the first objective optimization problem function and the second objective optimization problem function, is constructed using a balance coefficient. The formula is expressed as follows:

[0077] Where λ1 and λ2 represent balance coefficients, the specific values ​​of which are set according to the actual situation. It can be seen that the deployment optimization reward function includes constraints ensuring that sensors are not placed in restricted areas, and that the total number of sensors at any location does not exceed [a certain limit]. This refers to the maximum number of sensors allowed at that location. The deployment optimization reward function allows for the optimization of RSNs in a three-dimensional road environment, balancing the need for full coverage with practical constraints such as limited areas and budget limitations.

[0078] S40. Construct a decision sequence consisting of a state space, an action space, and a cumulative reward value. Solve the decision sequence using a decision converter to obtain the deployment strategy. The state space includes a set of deployable sensors, a set of deployable sensor locations, a deployable region, a restricted region, and the minimum detection probability of the deployable region. The action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors. The cumulative reward value is obtained by adaptively redistributing the deployment optimization reward function using a two-layer optimization method.

[0079] To address the reward function optimization problem in 3D RSN deployment, this invention proposes a method based on a Decision Transformer (DT). This method utilizes reinforcement learning techniques, particularly the Transformer model, to optimize sensor deployment under complex constraints, flexibly addressing challenges such as insufficient roadside sensor coverage. Furthermore, the multi-head attention mechanism in DT further enhances this strategy, enabling it to simultaneously consider changes in external conditions and traffic patterns, making it a highly effective solution for deploying sensors in complex and constantly changing 3D road environments.

[0080] DT (Deployment-Driven Learning), a sequence-modeling-based reinforcement learning (RL) approach, offers a novel way to handle complex optimization problems such as sensor deployment. DT optimizes deployment strategies through its sequence modeling capabilities. By analyzing historical sequences, the model learns to predict the outcomes of various actions, thereby improving its future decision-making strategies. This optimization is a continuous process, evolving as the model gains more insights in each subsequent deployment scenario. The training and decision-making processes are aligned with our primary goals of maximizing coverage and minimizing cost. The model's performance is continuously evaluated against these goals, ensuring that the developed deployment strategies are both practical and effective. This alignment is crucial to ensuring that the model's decisions are not only theoretically sound but also feasible in real-world applications. After deployment, the DT-recommended strategy is evaluated based on its effectiveness in achieving the set goals. This evaluation phase is essential for identifying areas for improvement in model performance. Feedback from this phase is fed into further training cycles, enabling the model to learn from real-world deployments and continuously improve its decision-making accuracy.

[0081] Unlike traditional RL methods, DT redefines the problem as a sequence modeling task, enabling more nuanced decision-making based on historical data. Therefore, the foundation of this invention is the construction of decision sequences, which is crucial for DT's understanding and learning. These decision sequences are represented by a state space s. t Action space a t and cumulative reward R t The combination of , where the state space s t Includes a set of deployable sensors S and a set of deployable areas A T Restricted region set A R The set of possible sensor deployment locations p D And the minimum detection probability set P of the deployable area, denoted as: s t ={S,A T A R ,p DThis rich state ensures that the DT model can acquire all relevant information in order to make informed deployment decisions.

[0082] By defining an action space to encompass all possible sensor deployment decisions at each time step, this action space includes deploying a new sensor. deploy Reconfigure existing sensors (areconfigure), remove sensor a remove , denoted as a t ={a deploy ,areconfigure,a remove The specific deployment area depends on the state space and the target region. This flexible action space allows the model to explore a wide range of deployment strategies to adapt to different conditions and needs.

[0083] The reward function is a key component that directly affects the learning focus of the DT model. It reflects the effectiveness of sensor coverage and the associated deployment costs. Thus, in this embodiment of the invention, the dual objectives of the first objective optimization problem function and the second objective optimization problem function are embedded into the decision-making process of the DT model, and the reward function is finally determined as formula (9).

[0084] However, the low performance of DT (Delayed Reward) persists in environments with delayed reward signals. The problem lies in the accumulation of undesirable biases in the autoregressive learning of the global signal, a consequence of delayed rewards. This bias manifests as an imbalance in the attention distribution and propagates through the layers of the neural network, leading to degraded model performance and weakening the effectiveness of the decision model. In particular, in scenarios involving significant delayed rewards, the semantic association between the reward signal and its corresponding time step is severely disrupted. Considering that the reward is only realized in the last time step, all previous target rewards in the trajectory are essentially the same. This setup results in most reward signals being redundant and potentially misleading, as they hinder the ability of classic self-attention models to effectively identify and learn diverse dependencies and unbiased attention distributions. To address these challenges, this invention proposes a two-layer optimization method to achieve adaptive reward redistribution. Specifically, the two-layer optimization method adaptively redistributes the deployed optimized reward function, including:

[0085] The initial cumulative reward value of the sensor deployment trajectory is calculated based on the deployment optimization reward function. A surrogate reward function is introduced to re-evaluate the reward at each time step in the sensor deployment trajectory to obtain a reshaped reward value. The reshaped reward function is constructed using the initial cumulative reward value and the reshaped reward value to achieve accurate redistribution of the original reward. The reshaped reward function is used as the third objective optimization problem function, and the validation loss function is used as the fourth objective optimization problem function. A two-level optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed by balancing the weight coefficients. The redistributed cumulative reward value is obtained by solving the two-level optimization problem function using the two-level optimization method. In the upper-level optimization process, the third objective optimization problem function is optimized, and in the lower-level optimization process, the fourth objective optimization problem function is optimized. One upper-level and lower-level optimization is completed to achieve one single-step optimization.

[0086] Next, we will discuss how to guide sensor deployment in 3D RSNs by reshaping the reward function, and explore a two-layer optimization method, emphasizing the improvement of policy model optimization efficiency through alternating iterative optimization of the reward function. Specifically:

[0087] In DT-based 3D road environment sensor deployment, reward redistribution in sensor deployment trajectories is crucial. Consider a set of offline sensor deployment trajectories: δ={δ1,δ2,…,δ n The goal is to effectively redistribute the cumulative impact of trajectories at each time step. This involves reshaping the reward function to help learn an optimized sensor deployment strategy. This embodiment of the invention introduces a surrogate reward function f. ψ The function recalibrates the reward at each time step of the sensor deployment trajectory, reshaping the reward at each time step (t). The formula is expressed as follows:

[0088] Among them, (s) t ,d t ∈δ), and (f ψ :s t ×d t →R t ) is the reward function parameterized as ψ, which is obtained through the state space s t and sensor deployment trajectory d t Implement and optimize the reward function R t The mapping relationship and detailed implementation process are described in existing technologies and will not be repeated here. The proxy reward function f... ψ Assign a revised reward value to each state-deployment decision pair, and In other words, this embodiment of the invention decomposes the reward into the contribution of each step, where the contribution of each step is determined by the reward value R at the current time. tand the reward value R at the next time step t-1 The difference between them is given, which represents the cumulative reward value under the current sensor deployment trajectory, where δ is the reward of the sensor deployment trajectory, and |δ| is the length of the sensor deployment trajectory. This is achieved by using the proxy reward function f... ψ Adjusted to the dynamics of specific road environments and sensor functions, the DT model is significantly enhanced in learning efficient deployment strategies under complex and delayed reward conditions.

[0089] In DT-based sensor deployment, a two-layer optimization framework comprises two interrelated tasks: optimizing the policy model for sensor placement and optimizing the reward function guiding this optimization. This approach is particularly suitable for complex scenarios in 3D road environments, where the temporal distribution and allocation of rewards are crucial for effective sensor deployment strategies. To determine the ideal reward function for DT-based sensor deployment, it is constrained to decompose the total reward of the deployment trajectory into individual rewards for each step by training a regression model. However, this approach is primarily unidirectional, relying on regression of the reward function to train the agent, which can lead to overfitting and a lack of adaptability in the presence of significantly delayed rewards. To address these limitations, embodiments of the present invention introduce a comprehensive bidirectional two-layer optimization framework. This framework is more suitable for scenarios with long reward delays and offers several advantages. Specifically:

[0090] First, it enables the trained reward function not only to guide agent training but also to allow the agent to iteratively optimize the reward function, forming a dynamic loop of bidirectional feedback and joint optimization. Second, it addresses the overfitting problem by introducing a high-level optimization process into the optimization of the reward function. The core concept is to use a validation set to evaluate the impact of reward redistribution on the policy model's performance, thereby guiding the optimization of the reward function. Therefore, all sensor deployment trajectory data δ are divided into two sets: a training set for policy model optimization. and a validation set for reward function optimization The focus of lower-level optimization is to enhance the policy model π. ξ It deploys trajectories from a given set of sensors. The optimal deployment strategy is learned from the data. This process involves training a policy model to adapt to the reshaped rewards, ensuring it accurately reflects the deployment goals and constraints of the RSNs. The policy model π ξ Trained to make decisions consistent with reshaping the reward function, the goal is to maximize the effectiveness of sensor placement while considering environmental factors and deployment constraints. The higher-level task focuses on optimizing the deployment and reward function R. t The aim is to discover methods that can significantly improve the performance of policy models on independent validation sets. Configure the performance-optimized reward function to verify the effectiveness of the reward refactoring. Deploy and optimize the reward function R. tThe strategy is adjusted to maximize the decision-making performance of the policy model, ensuring that the optimized policy model performs best in different RSN scenarios. This embodiment of the invention uses the reshaping reward function as the third objective optimization problem function and the validation loss function as the fourth objective optimization problem function. A two-layer optimization problem function consisting of the third and fourth objective optimization problems is constructed through balancing coefficients. The third objective optimization problem function is expressed as:

[0091] Where C3 represents the third objective optimization problem function, Let R represent the validation set, δ represent the sensor deployment trajectory, |δ| represent the length of the sensor deployment trajectory, and R ep (δ) represents the cumulative reward value under the sensor deployment trajectory. This represents the reshaping reward value at time step t;

[0092] The fourth objective optimization problem function is expressed as follows:

[0093] Where C4 represents the fourth objective function, Let ξ represent the validation loss function. * (ζ) represents the policy model to be solved, and ζ represents the parameters of the reward model.

[0094] The constructed two-level optimization problem function is to minimize the validation loss function. The condition is that the total reward remains consistent before and after the redistribution. In other words, to transform this constrained bilevel optimization problem into an unconstrained one, a quadratic penalty function is introduced. The final objective function is expressed as:

[0095] Wherein, λ represents the balance weight coefficient between the third objective optimization function and the fourth objective optimization function, and the specific value is set according to the actual situation.

[0096] In the DT-based 3D road environment sensor deployment framework, the two-layer optimization process is crucial for model efficiency. However, accurately calculating the upper-layer deployment optimization reward function R... t The gradient may be affected by its relationship with the lower-level policy optimization π. ξ The tight coupling between the layers makes this process challenging. This complexity typically requires multiple iterations to determine the optimal policy π for the lower layers. ξThis can lead to a potentially time-consuming process. To address this issue, an alternating optimization method is used in both the upper and lower optimization processes to optimize the bi-level optimization problem function. During optimization, the policy model for the current single-step optimization is solved by training the loss function, serving as an approximate solution for the policy model to be solved in the validation loss function. This approximate solution is then used as the policy model to be solved in the next single-step optimization to update the validation loss function. The process continues until the stopping condition is met. This alternating iterative optimization technique effectively separates the parameters ξ and ζ, thereby accelerating the optimization process. The formula for solving the policy model for the current single-step optimization by training the loss function, serving as an approximate solution for the policy model to be solved in the validation loss function, is expressed as:

[0097] Where, ξ * (ζ) represents the policy model to be solved for validating the loss function, ξ′(ζ) represents the policy model for the current single-step optimization, ξ(ζ) represents the current policy model, and α ξ This represents the learning rate for the current single-step optimization. This indicates the gradient calculation operation. This represents the training loss function.

[0098] This invention adopts the principle of meta-learning and adjusts ξ through single-step training. The approximation scheme of formula (14) ensures efficient computation and accelerates the overall optimization process of sensor deployment in the DT framework, which is crucial for handling complex deployment scenarios with delayed rewards.

[0099] To verify the effectiveness of the three-dimensional roadside sensor deployment method for vehicle-to-everything (V2X) collaborative perception provided in this embodiment of the invention, the superiority of this invention in terms of coverage and detection accuracy is demonstrated in multiple real-world driving scenarios through comparison with traditional methods.

[0100] Numerical results clearly demonstrate that the method proposed in this invention significantly improves upon existing technologies in terms of sensing, communication, and computational collaboration efficiency, as well as resource allocation efficiency. Specifically, when evaluating the effectiveness of the sensor deployment strategy, the performance of two key indicators (coverage area and sensing accuracy) on key influencing factors—deployment cost (number of deployed sensors) and road conditions (number of lanes)—was considered. Referring to Figure 3, compared to traditional methods such as Q-Learning, MEP-PSO (a method combining the Minimum Exposure Path Problem (MEP) and Particle Swarm Optimization (PSO), and Cat Swarm (cat swarm optimization), the left side of Figure 3 shows that the horizontal axis represents the number of sensors, and the vertical axis represents coverage. It can be seen that as the number of sensors increases from 10 to 50, the coverage of the method proposed in this invention increases from 83.8% to 97.8%, enabling more efficient coverage of the target area. The increase in the number of traffic lanes has a certain impact on coverage. As shown in the right side of Figure 3, the horizontal axis represents the number of lanes and the vertical axis represents the coverage. It can be seen that the coverage of the method proposed in this invention decreased from 98.2% for two lanes to 90.3% for six lanes. However, this decrease is more moderate than that of traditional methods, which shows that the DT method can maintain high adaptability and coverage efficiency under complex road conditions.

[0101] This invention further analyzes the perception accuracy under different conditions. Referring to Figure 4, the horizontal axis represents the number of deployed sensors, and the vertical axis represents the perception accuracy. Compared to traditional methods such as Q-Learning, MEP-PSO, and Cat Swarm, the perception accuracy shows a significant upward trend with the increase in the number of deployed sensors, improving from an initial 85.8% to 97.4%. The perception accuracy increases rapidly with the increase in the number of deployed sensors, then tends to stabilize, indicating a saturation effect when the number of deployed sensors is high. Compared to traditional methods, the method proposed in this invention performs excellently on all deployed sensors. The performance enhancement is mainly attributed to the DT method employed, which integrates the Transformer architecture with RL, enabling the Agent to be closely integrated with the vehicle's sensing requirements, and outperforming traditional RL methods and static optimization methods.

[0102] In summary, the proposed method for deploying 3D roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication constructs a Road Sensor Network (RSN) model that accurately reflects complex 3D road environments. Based on this RSN model, a sensor deployment method based on a decision converter is designed. This method utilizes the Transformer model and multi-head attention mechanism within the decision converter to dynamically adapt to various road conditions and environmental changes, effectively solving the problem of deploying sensors in complex and constantly changing 3D road environments. Furthermore, to improve the performance of the decision converter, this invention introduces a two-layer optimization technique. This technique optimizes the cumulative reward in the decision sequence of the decision converter, allowing for real-time adjustments to the deployment scheme to adapt to complex road and environmental changes. Overall, this invention improves the coverage and perception accuracy of 3D road environments while reducing deployment costs, enhancing economic efficiency and practical application effectiveness, thereby significantly improving the safety and efficiency of autonomous and intelligent connected vehicles in complex environments.

[0103] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0104] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the specification and accompanying drawings, will understand and implement other variations of the disclosed embodiments in carrying out the claimed invention. In the specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. While certain measures are described in different embodiments, this does not mean that these measures cannot be combined to produce good results.

[0105] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) networks, characterized in that, The method includes: Modeling includes RSNs models of the 3D road environment, comprising a set of deployable sensors, a set of deployable sensor locations, a set of deployable areas, and a set of restricted areas; Based on the RSNs model of the three-dimensional road environment, a perception detection function for a single sensor in each deployable area is constructed. When multiple sensors cover each deployable area, a global sensor detection function for each deployable area is constructed based on the perception detection function of the single sensor. The first objective optimization problem function is to maximize the proportion of deployable areas that meet or exceed the minimum detection probability according to the global sensor detection function, and the second objective optimization problem function is to minimize the total number of sensors in each deployable area. A deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function is constructed through a balance coefficient. A decision sequence consisting of a state space, an action space, and a cumulative reward value is constructed, and a deployment strategy is obtained by solving the decision sequence based on a decision converter. The state space includes a set of deployable sensors, a set of deployable sensor locations, a set of deployable regions, a set of restricted regions, and a set of minimum detection probabilities. The action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors. The cumulative reward value is obtained by adaptively redistributing the deployment optimization reward function using a two-layer optimization method.

2. The method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication according to claim 1, characterized in that, Construct the sensing and detection function for a single sensor, including: Four sub-sensing detection functions are constructed to determine the detection probability of a single sensor, which are affected by distance, horizontal detection viewpoint, vertical detection viewpoint, and obstacles, respectively. The perception and detection function of a single sensor is constructed by combining four sub-perception and detection functions.

3. The method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication according to claim 2, characterized in that, The sub-sensor detection function affected by distance is expressed by the following formula: in, This represents a sub-sensory detection function affected by distance. Indicates sensor s i,j The reference location information, Indicates sensor s i,j The actual location information, α represents The slope adjustment parameter, β, represents the maximum effective distance of the sensor; The sub-sensory detection function affected by the horizontal detection viewpoint is expressed as follows: in, This represents the sub-sensory detection function affected by the horizontal detection viewpoint. Indicates sensor s i,j The baseline horizontal angle, Indicates sensor s i,j The actual horizontal angle, ι represents The slope adjustment parameter, η, represents the maximum detection angle of the sensor; The sub-perceptual detection function affected by the vertical detection viewpoint is expressed as follows: in, This represents the sub-sensory detection function affected by the vertical detection viewpoint. Indicates sensor s i,j The reference vertical angle, Indicates sensor s i,j The actual vertical angle, ι represents The slope adjustment parameter, express The value at 50% of maximum coverage; The sub-sensor detection function affected by obstacles is expressed by the following formula: in, Let G represent the sub-sensing detection function affected by obstacles, and let G represent the obstacle adjustment parameter.

4. The method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication according to claim 3, characterized in that, The constructed sensing and detection function for a single sensor is expressed by the following formula: in, The sensing and detection function of a single sensor, i.e. Represents the set of deployable regions A T Deployable areas Sensors i,j The perception detection function, N T Indicates the number of deployable regions. This represents the i-th deployable region; When multiple sensors are in coverage, a global sensor detection function is constructed for each deployable area, expressed as follows: in, This represents the global sensor detection function, which can be used to deploy a set of regions. A set of sensors can be deployed in the middle. i The perception detection function, s i,j s i The j-th deployable sensor in the middle, Indicates deployable area The maximum number of sensors that can be deployed in the system.

5. The method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication according to claim 1, characterized in that, The first objective optimization problem function is expressed as follows: Where C1 represents the first objective optimization problem function, card{} represents finding the number of elements in the set, and A T Indicates the set of deployable regions. N T Indicates the number of deployable regions. Let P represent the i-th deployable region, and let P represent the set of minimum detection probabilities. Indicates deployable area The minimum detection probability, s i Indicates deployable area Sensor arrays can be deployed in the middle. Indicates deployable area The global sensor detection function; The second objective optimization problem function is expressed as follows: in, Indicates deployable area The number of sensors that can be deployed in the middle Indicates deployable area The set of locations where sensors can be deployed, Indicates deployable area The position information of the j-th sensor.

6. The method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication according to claim 1, characterized in that, An adaptive reward redistribution of the deployment optimization reward function is achieved through a two-layer optimization method, including: The initial cumulative reward value of the sensor deployment trajectory is calculated based on the deployment optimization reward function, and a proxy reward function is introduced to re-evaluate the reward of each time step in the sensor deployment trajectory to obtain the reshaping reward value. The reshaping reward function is constructed using the initial cumulative reward value and the reshaping reward value. By using the reshaping reward function as the third objective optimization problem function and the validation loss function as the fourth objective optimization problem function, a two-level optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed by balancing the weight coefficients. The cumulative reward value after redistribution is obtained by solving the bi-level optimization problem function using a bi-level optimization method. In the upper-level optimization process, the third objective optimization problem function is optimized, and in the lower-level optimization process, the fourth objective optimization problem function is optimized. Completing one upper-level and lower-level optimization achieves one single-step optimization.

7. The method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication according to claim 6, characterized in that, The third objective optimization problem function is expressed as follows: Where C3 represents the third objective optimization problem function, Let R represent the validation set, δ represent the sensor deployment trajectory, |δ| represent the length of the sensor deployment trajectory, and R ep (δ) represents the cumulative reward value of the sensor deployment trajectory. This represents the reshaping reward value at time step t; The fourth objective optimization problem function is expressed as follows: Where C4 represents the fourth objective function, Let ξ represent the validation loss function. * (ζ) represents the policy model to be solved, and ζ represents the parameters of the reward model.

8. The method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication according to claim 6, characterized in that, In the upper and lower optimization processes, an alternating optimization method is used to optimize the bi-level optimization problem function; among them, During the optimization process, the policy model for the current single-step optimization is solved by training the loss function, which serves as an approximate solution for the policy model to be solved in the validation loss function. This approximate solution is then used as the policy model to be solved in the next single-step optimization to update the validation loss function. The validation loss function is then returned as the fourth objective optimization problem function until the stopping condition is met.

9. The method for deploying three-dimensional roadside sensors for collaborative perception in vehicle-to-everything (V2X) communication according to claim 8, characterized in that, The current single-step optimized policy model is solved by training the loss function, which serves as an approximate solution for the policy model to be solved, validating the loss function. The formula is as follows: Where ξ′(ζ) represents the current single-step optimization strategy model, ξ(ζ) represents the current strategy model, and α ξ This represents the learning rate for the current single-step optimization. This indicates the gradient calculation operation. This represents the training loss function.

Images