A three-dimensional roadside sensor deployment method for collaborative perception in connected vehicles

Abstract

The present invention discloses a three-dimensional roadside sensor deployment method for collaborative perception in the Internet of Vehicles (IoV). The method comprises: modeling a three-dimensional road environment using an RSNs model; constructing a perception detection function for a single sensor based on the RSNs model; and constructing a global sensor detection function for each deployable area based on the perception detection function of the single sensor when multiple sensors are present. The method then optimizes the first objective, maximizing the proportion of deployable areas that meet or exceed a minimum detection probability according to the global sensor detection function, and minimizing the total number of sensors in each deployable area as a second objective. The method then constructs a deployment optimization reward function composed of the first and second objective optimization problem functions. The method then constructs a decision sequence and solves the decision sequence using a decision transformer to obtain a deployment strategy. The method further implements sensor deployment in a three-dimensional road environment.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle networking, and specifically relates to a three-dimensional roadside sensor deployment method for vehicle networking collaborative perception. Background Art

[0002] In the connected vehicle (IoV) environment, collaborative perception technology is crucial to the safety and efficiency of autonomous vehicles. IoV achieves collaborative perception by deploying multiple sensors (such as cameras, lidar, and radar) on both sides of the road. These sensors share this data with autonomous vehicles in real time, significantly improving their perception capabilities in complex road environments. Autonomous and intelligently connected vehicles must operate efficiently and safely in a variety of complex scenarios, placing higher demands on sensor deployment.

[0003] Existing technical solutions primarily focus on 2D sensor deployment optimization, static 3D scene modeling, and local optimization algorithms. Traditional 2D sensor deployment optimization methods are typically based on simple geometric models, optimizing sensor positions and angles to cover the target area and maximizing sensor coverage by calculating optimal sensor positions and viewing angles. However, these methods fail to account for the complex 3D road environment and are unable to address blind spots in practical applications. Some studies have attempted to expand sensor deployment to 3D scenes, but these methods typically employ idealized 3D models, ignoring the actual complexity of roads and the constraints inherent in sensor deployment. While these methods perform well in experimental settings, their effectiveness is often limited in real-world road environments, making them incapable of tackling the complexities and diversity of real-world roads. Traditional sensor deployment methods for 2D and idealized 3D scenes typically employ static optimization algorithms, such as particle swarm optimization and genetic algorithms. These methods perform well during initial deployment but lack the ability to dynamically adapt to environmental changes. In real-world applications, changing traffic and environmental conditions can impact sensor performance, making static optimization algorithms difficult to adjust deployment strategies in real time, resulting in decreased perception coverage and accuracy. For example, patent CN113743479B proposes an end-edge-cloud vehicle-road collaborative fusion perception architecture that optimizes information flow through multi-agent distributed coordination theory. Although this approach enhances the system's environmental adaptability, it still faces challenges in handling real-time data transmission and processing in highly dynamic traffic scenarios. Furthermore, patent CN117294738B proposes a method for optimizing the deployment and perception of autonomous driving sensors, which enhances the system's perception capabilities by optimizing the spatial coverage and location weights of sensors. However, this approach still has limitations when dealing with complex and dynamically changing road environments.

[0004] In summary, existing collaborative sensing technologies face numerous challenges in sensor deployment, particularly in complex and volatile urban road environments. Traditional two-dimensional or idealized three-dimensional scene modeling methods struggle to cope with the complexity of real roads, leading to limitations in coverage, perception accuracy, and real-time performance of existing sensor networks. Furthermore, traditional two-dimensional and idealized three-dimensional scene sensor deployment methods typically employ static optimization algorithms, lacking the ability to dynamically adapt to environmental changes and making it difficult to provide reliable perception assistance under various unpredictable road conditions and weather changes. Summary of the Invention

[0005] To address the aforementioned issues in the prior art, the present invention provides a method for deploying three-dimensional roadside sensors for collaborative perception in connected vehicles. The technical issues addressed by the present invention are achieved through the following technical solutions:

[0006] An embodiment of the present invention provides a method for deploying three-dimensional roadside sensors for collaborative perception in an Internet of Vehicles (IoV). The method includes:

[0007] Modeling the RSNs model of a three-dimensional road environment including a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, and a set of restricted areas;

[0008] Based on the RSNs model of the three-dimensional road environment, constructing a perception detection function of a single sensor in each deployable area, and when multiple sensors cover each deployable area, constructing a global sensor detection function for each deployable area based on the perception detection function of the single sensor;

[0009] Maximizing the proportion of deployable areas that meet or exceed the minimum detection probability according to the global sensor detection function is used as a first objective optimization problem function, minimizing the total number of sensors in each deployable area is used as a second objective optimization problem function, and constructing a deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function through a balancing coefficient;

[0010] A decision sequence consisting of a state space, an action space, and a cumulative reward value is constructed, and the deployment strategy is obtained by solving the decision sequence based on a decision transformer; wherein the state space includes a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, a set of restricted areas, and a set of minimum detection probabilities; the action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors; the cumulative reward value is obtained by adaptively redistributing the rewards of the deployment optimization reward function using a two-layer optimization method.

[0011] In one embodiment of the present invention, constructing a perception detection function of a single sensor includes:

[0012] Four sub-perception detection functions are constructed, in which the detection probability of a single sensor is affected by distance, horizontal detection angle of view, vertical detection angle of view, and obstacles.

[0013] The perception detection function of a single sensor is constructed by the four sub-perception detection functions.

[0014] In one embodiment of the present invention, the sub-perception detection function affected by distance is expressed as follows:

[0015]

[0016] in, represents the sub-perception detection function affected by distance, Indicates sensor s i,j The reference position information, Indicates sensor s i,j The actual position information, α represents The slope adjustment parameter, β represents the maximum effective distance of the sensor;

[0017] The sub-perception detection function affected by the horizontal detection viewing angle is expressed as follows:

[0018]

[0019] in, represents the sub-perception detection function affected by the horizontal detection viewing angle, Indicates sensor s i,j The reference horizontal angle, Indicates sensor s i,j The actual horizontal angle, ι represents The slope adjustment parameter, η represents the maximum detection angle of the sensor;

[0020] The sub-perception detection function affected by the vertical detection viewing angle is expressed as:

[0021]

[0022] in, represents the sub-perception detection function affected by the vertical detection viewing angle, Indicates sensor s i,j The base vertical angle, Indicates sensor s i,j The actual vertical angle, ι represents The slope adjustment parameter, express Value at 50% of maximum coverage;

[0023] The sub-perception detection function affected by obstacles is expressed as follows:

[0024]

[0025] in, represents the sub-perception detection function affected by obstacles, and G represents the obstacle adjustment parameter.

[0026] In one embodiment of the present invention, the perception detection function of a single sensor is constructed, and the formula is expressed as follows:

[0027]

[0028] in, Represents the perception detection function of a single sensor, that is Represents the deployable area set A T Deployable area Sensors i,j The perceptual detection function, N T Indicates the number of deployable regions, represents the i-th deployable area;

[0029] When multiple sensors are covered, a global sensor detection function is constructed for each deployable area, which is expressed as:

[0030]

[0031] in, Represents the global sensor detection function, which can be deployed as a set of regions Deployable sensor sets i The perceptual detection function, s i,j Indicates s i The jth deployable sensor in Indicates the deployable area The maximum number of deployable sensors in .

[0032] In one embodiment of the present invention, the first objective optimization problem function is expressed as follows:

[0033]

[0034] Among them, C1 represents the first objective optimization problem function, card{} represents the number of elements in the set, A T Represents a set of deployable regions. N T Indicates the number of deployable regions, represents the i-th deployable area, P represents the minimum detection probability set, Indicates the deployable area The minimum detection probability, s i Indicates the deployable area A collection of deployable sensors, Indicates the deployable area Global sensor detection function;

[0035] The second objective optimization problem function is expressed as follows:

[0036]

[0037] in, Indicates the deployable area The number of deployable sensors in Indicates the deployable area The set of sensor deployable locations, Indicates the deployable area The position information of the jth sensor in .

[0038] In one embodiment of the present invention, adaptive reward redistribution of the deployment optimization reward function is achieved through a two-layer optimization method, including:

[0039] An initial cumulative reward value of the sensor deployment trajectory is calculated based on the deployment optimization reward function, and a proxy reward function is introduced to re-evaluate the reward of each time step in the sensor deployment trajectory to obtain a reshaping reward value, and a reshaping reward function is constructed using the initial cumulative reward value and the reshaping reward value;

[0040] The reshaped reward function is used as the third objective optimization problem function, the verification loss function is used as the fourth objective optimization problem function, and a two-layer optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed by balancing the weight coefficients;

[0041] The double-layer optimization problem function is solved by a double-layer optimization method to obtain the reallocated cumulative reward value; wherein, the third objective optimization problem function is optimized in the upper-layer optimization process, and the fourth objective optimization problem function is optimized in the lower-layer optimization process, completing the upper-layer and lower-layer optimization once to achieve a single-step optimization.

[0042] In one embodiment of the present invention, the third objective optimization problem function is expressed as follows:

[0043]

[0044] Among them, C3 represents the third objective optimization problem function, represents the validation set, δ represents the sensor deployment trajectory, |δ| represents the length of the sensor deployment trajectory, R ep(δ) represents the cumulative reward value of the sensor deployment trajectory, represents the reshape reward value at time step t;

[0045] The fourth objective optimization problem function is expressed as follows:

[0046]

[0047] Among them, C4 represents the fourth objective optimization problem function, represents the validation loss function, ξ * (ζ) represents the policy model to be solved, and ζ represents the parameters of the reward model.

[0048] In one embodiment of the present invention, during the optimization process of the upper layer and the lower layer, an alternating optimization method is used to optimize the double-layer optimization problem function; wherein,

[0049] During the optimization process, the strategy model of the current single-step optimization is solved by training the loss function as an approximate solution of the strategy model to be solved for the verification loss function, and the approximate solution is used as the strategy model to be solved for the verification loss function in the next single-step optimization to update the verification loss function. The verification loss function is returned as the fourth objective optimization problem function until the stopping condition is met.

[0050] In one embodiment of the present invention, the current single-step optimized strategy model is solved by training the loss function as an approximate solution of the strategy model to be solved for verifying the loss function. The formula is expressed as:

[0051]

[0052] Among them, ξ′(ζ) represents the current single-step optimization strategy model, ξ(ζ) represents the current strategy model, α ξ Represents the learning rate of the current single-step optimization, represents the gradient operation, Represents the training loss function.

[0053] Beneficial effects of the present invention:

[0054] The present invention proposes a three-dimensional roadside sensor deployment method for collaborative perception in the Internet of Vehicles (IoV) system. This method constructs a Road Sensor Network (RSN) model that can realistically reflect complex three-dimensional road environments. Based on this RSN model, a sensor deployment method based on a decision transformer is designed. This method utilizes the Transformer model and multi-head attention mechanism within the decision transformer to dynamically adapt to various road conditions and environmental changes, effectively solving the problem of difficult sensor deployment in complex and ever-changing three-dimensional road environments. Furthermore, to improve the performance of the decision transformer, the present invention also introduces a two-layer optimization technique. By optimizing the cumulative rewards in the decision transformer's decision sequence, the deployment plan can be adjusted in real time to adapt to complex road and environmental changes. Overall, the present invention improves the coverage and perception accuracy of three-dimensional road environments while reducing deployment costs, enhancing economic efficiency and practical application effectiveness, thereby significantly improving the safety and efficiency of autonomous driving and intelligent connected vehicles in complex environments.

[0055] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of a method for deploying three-dimensional roadside sensors for collaborative perception in Internet of Vehicles, provided by an embodiment of the present invention;

[0057] Figure 2 Schematic diagram of the RSNs model of a three-dimensional road environment provided by an embodiment of the present invention;

[0058] Figure 3 This is the intention of the embodiment of the present invention to show how coverage varies with the number of sensors and the number of lanes.

[0059] Figure 4 This is a schematic diagram of how perception accuracy changes with the number of deployed sensors, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0061] See Figure 1 The embodiment of the present invention provides a three-dimensional roadside sensor deployment method for collaborative perception of an Internet of Vehicles, which specifically includes the following steps:

[0062] S10. Modeling an RSN model of a three-dimensional road environment including a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, and a set of restricted areas.

[0063] The RSNs model of the three-dimensional road environment modeled by the embodiment of the present invention is as follows: Figure 2 As shown, it can include a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, and a set of restricted areas.

[0064] Assume that the RSNs model includes: a deployable sensor set consisting of N deployable sensors, denoted as S, N T Indicates the number of deployable regions, s i represents the set of deployable sensors in the ith deployable area, s i,j Indicates s i The jth deployable sensor in Indicates the maximum number of deployable sensors in the i-th deployable area; N T A deployable area set consisting of deployable areas, denoted as A T , represents the i-th deployable area, each deployable area is a rectangular area, is defined as the coordinates of its diagonally opposite corner, Indicates the deployable area The coordinates of the upper left corner, Indicates the deployable area The coordinates in the lower right corner define the boundaries and dimensions of the deployable area, which helps to accurately assess coverage requirements. To ensure the reliability of the autonomous driving system, it is necessary to ensure that the perception probability within the deployable area is not lower than the set threshold.

[0065] In actual road environments, not all locations above and on both sides of the road are suitable for sensor deployment, such as roadside green belts, guardrails, etc. Therefore, when considering sensor deployment, it is necessary to consider areas along the road where sensor installation is explicitly prohibited. R The restricted area set consisting of restricted areas is recorded as AR. represents the kth restricted area, each restricted area is a rectangular area, Indicates a restricted area The coordinates of the upper left corner, Indicates a restricted area The coordinates in the lower right corner precisely define the location and size of the restricted area to ensure that no sensor of any type is installed within it. Establishing this restricted area is crucial to prevent interference with important operations or areas sensitive to sensors.

[0066] The sensors required for cooperative sensing are strategically placed on both sides of the road or above the road to ensure that normal traffic flow is not affected. T The set of sensor deployable locations consisting of sensor deployable locations is denoted as p D , Indicates the deployable area The set of sensor deployable locations, Indicates the deployable area The position information of the jth sensor in , such as the three-dimensional coordinates can be recorded as Considering the impact of sensor deployment height, here Function represents the position (x D ,y D ) horizontal height function, z i,j Indicates the height of the deployed sensor from the ground. Embodiments of the present invention enhance the integration and analysis of data collection by determining the exact location of each sensor within the deployable area.

[0067] S20. Based on the RSNs model of the three-dimensional road environment, a perception detection function of a single sensor is constructed, and when multiple sensors are covered, a global sensor detection function is constructed based on the perception detection function of the single sensor.

[0068] Actual sensor behavior is not solely determined by the distance between the sensor and the target and the area covered by the sensor. The perception model is primarily influenced by the perception distance, horizontal viewing angle, vertical viewing angle, and obstacles. Therefore, the present invention constructs a single sensor perception detection function, including:

[0069] Four sub-perception detection functions are constructed to determine the impact of distance, horizontal detection angle, vertical detection angle, and obstacles on the detection probability of a single sensor. The four sub-perception detection functions are then used together to construct the perception detection function of a single sensor. Specifically:

[0070] In autonomous driving systems, sensor efficiency is often evaluated by its perceptual detection probability. It's worth noting that as the perceptual detection distance increases, the sensor's perceptual detection capability tends to weaken. The key factor influencing this pattern is the Euclidean distance, which quantifies the separation between the sensor and its target. The resulting sub-perceptual detection function, influenced by distance, is expressed as:

[0071]

[0072] in, represents the sub-perception detection function affected by distance, Indicates sensor s i,j The reference position information, Indicates sensor s i,j The actual position information, α represents The slope adjustment parameter plays an important role in adjusting the slope of the function, affecting the degree to which the detection probability decreases rapidly with increasing distance. β represents the maximum effective distance of the sensor.

[0073] In the field of perception and detection of complex sensors, both the perception detection distance and the horizontal deviation of the sensor's central axis are important. As the horizontal detection angle of view deviates from the central axis, the perception detection probability generally decreases. The final constructed sub-perception detection function affected by the horizontal detection angle is expressed as follows:

[0074]

[0075] in, represents the sub-perception detection function affected by the horizontal detection viewing angle, Indicates sensor s i,j The reference horizontal angle, Indicates sensor s i,j The actual horizontal angle, ι represents The slope adjustment parameter reflects the rate at which the detection probability decreases as the horizontal viewing angle becomes wider. η represents the maximum detection angle of the sensor, beyond which the detection probability drops sharply.

[0076] Similar to horizontal perspective perception, vertical detection perspective also affects the perception detection probability. The perception detection probability is related to the central axis of the sensor, which means that if the object is close to the vertical central axis, the sensor is more likely to detect it. The final constructed sub-perception detection function affected by the vertical detection perspective is expressed as follows:

[0077]

[0078] in, represents the sub-perception detection function affected by the vertical detection viewing angle, Indicates sensor s i,j The base vertical angle, Indicates sensor s i,j The actual vertical angle, ι represents The slope adjustment parameter reflects the rate at which the probability of perceptual detection decreases as the vertical viewing angle becomes wider. express Value at 50% of maximum coverage.

[0079] Sensors in autonomous vehicle systems excel at environmental perception, but may encounter challenges that hinder their ability to perceive. Obstacles, whether physical barriers or adverse weather conditions, can significantly affect line of sight or the quality of perceptual detection, resulting in a reduced probability of perceptual detection within the intended detection area. Ultimately, the present invention uses a low-complexity Bresenham algorithm to model the impact of obstacles on perceptual detection probability. The resulting sub-perceptual detection function affected by obstacles is expressed as follows:

[0080]

[0081] in, represents the sub-perception detection function affected by an obstacle, and G represents the obstacle adjustment parameter, which is a variable determined by the obstacle type. For example, when the obstacle is a transparent plastic bag, G might be 0.9 and 0.1, respectively. The sub-perception detection function affected by the obstacle describes the change in perception capability due to the obstacle. The sub-perception detection function affected by the obstacle quantifies the impact of the perception detection probability in the presence of an obstacle. In contrast, when the obstacle is absent, the probability is 1, indicating unimpeded perception.

[0082] Ultimately, the perception and detection capability of a single sensor in this embodiment of the present invention is constructed using four sub-perception and detection functions affected by distance, horizontal detection angle of view, vertical detection angle of view, and obstacles. The constructed perception and detection function of a single sensor is expressed as follows:

[0083]

[0084] in, Represents the perception detection function of a single sensor, that is Represents the deployable area set A T Deployable area Sensors i,j Perception detection function.

[0085] In autonomous driving systems, individual sensors have inherent limitations in their coverage range. When sensors work together, the probability of sensing detection within the target area (i.e., the deployment area) increases significantly. While a single sensor may only cover a portion of the target area, the combination of multiple sensors ensures full coverage, thereby increasing the likelihood of accurate sensing detection within the entire target area. When the target area is covered by multiple sensors, the global sensor detection function is constructed, which is expressed as follows:

[0086]

[0087] in, Represents the global sensor detection function, that is, the set of deployable areas Deployable sensor sets i The perceptual detection function, s i,j Indicates s i The jth deployable sensor in Indicates the deployable area The maximum number of deployable sensors in .

[0088] S30. Maximizing the proportion of deployable areas that meet or exceed the minimum detection probability according to the global sensor detection function is used as the first objective optimization problem function, minimizing the total number of sensors in each deployable area is used as the second objective optimization problem function, and constructing a deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function through the balance coefficient.

[0089] However, achieving such extensive perception coverage is challenging, especially due to budget constraints. Therefore, constraints are introduced that emphasize the balance between maximizing perception detection capabilities and cost-effectiveness. The main goal is to achieve the highest detection coverage through multi-sensor collaboration and effectively utilize available resources in an economical manner. Therefore, the goal of this invention is to develop a sensor deployment strategy for 3D RSNs that maximizes perception assistance while minimizing the number of sensors deployed, taking into account both deployable areas and restricted areas, i.e., covering the deployable area with one sensor network while ensuring that the perception probability within the deployable area is not less than the minimum detection probability of each deployable area. The threshold set in , and avoid placing sensors in restricted areas. Finally, the embodiment of the present invention maximizes the proportion of deployable areas that meet or exceed the minimum detection probability based on the global sensor detection function as the first objective optimization problem function, which is expressed as follows:

[0090]

[0091] Among them, C1 represents the first objective optimization problem function, card{} represents the number of elements in the set, A T Represents a set of deployable regions. N T Indicates the number of deployable regions, represents the i-th deployable area, P represents the minimum detection probability set, Indicates the deployable area The minimum detection probability, Indicates the deployable area Global sensor detection function.

[0092] At the same time, minimizing the total number of sensors in each deployable area is used as the second objective optimization problem function, which is expressed as follows:

[0093]

[0094] in, Indicates the deployable area The number of deployable sensors in Indicates the deployable area The set of sensor deployable locations, Indicates the deployable area The position information of the jth sensor in .

[0095] The deployment optimization reward function consisting of the first objective optimization problem function and the second objective optimization problem function is constructed through the balance coefficient. The formula is expressed as:

[0096] R t =λ1×C1-λ2×C2

[0097]

[0098] Among them, λ1 and λ2 represent the balance coefficients, and their specific values ​​are set according to the actual situation. It can be seen that the deployment optimization reward function includes the constraints that ensure that sensors are not placed in restricted areas, and the total number of sensors at any location does not exceed That is, the maximum number of sensors allowed at that location, i.e., the deployment optimization reward function allows optimizing RSNs in a 3D road environment, balancing the need for comprehensive coverage with practical constraints such as restricted areas and budgetary limitations.

[0099] S40. Construct a decision sequence consisting of a state space, an action space, and a cumulative reward value, and obtain a deployment strategy by solving the decision sequence based on a decision converter; wherein, the state space includes a set of deployable sensors, a set of sensor deployable locations, a deployable area, a restricted area, and a minimum detection probability of a deployable area; the action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors; the cumulative reward value is obtained by adaptively redistributing the rewards of the deployment optimization reward function using a two-layer optimization method.

[0100] To address the deployment optimization reward function problem in 3D RSN deployment, embodiments of the present invention propose a Decision Transformer (DT)-based approach. This approach leverages reinforcement learning techniques, specifically the Transformer model, to optimize sensor deployment under complex constraints, flexibly addressing challenges such as insufficient roadside sensor coverage. Furthermore, the multi-head attention mechanism within the DT further enhances this strategy, enabling it to simultaneously account for changes in external conditions and traffic patterns, making it a highly effective solution for deploying sensors in complex and ever-changing 3D road environments.

[0101] DT, a sequence modeling-based reinforcement learning (RL) approach, offers a novel approach to complex optimization problems such as sensor deployment. DT leverages its sequence modeling capabilities to optimize deployment policies. By analyzing historical sequences, the model learns to predict the outcomes of various actions, thereby improving its future decision-making policies. This optimization is a continuous process that evolves as the model gains more insights with each subsequent deployment scenario. The training and decision-making processes are aligned with our primary goals of maximizing coverage and minimizing cost. The model's performance is continuously evaluated against these goals, ensuring that the deployment policies it develops are both practical and effective. This alignment is critical to ensuring that the model's decisions are not only theoretically sound but also feasible in real-world applications. After deployment, the policies recommended by DT are evaluated based on how well they achieve the set goals. This evaluation phase is crucial for identifying areas for model performance improvement. Feedback from this phase feeds into further training cycles, enabling the model to learn from real-world deployments and continuously improve its decision-making accuracy.

[0102] Unlike traditional RL methods, DT redefines the problem as a sequence modeling task, which can make more detailed decisions based on historical data. Therefore, the basis of the embodiment of the present invention is to build a decision sequence, which is the key to DT understanding and learning. The decision sequence is represented by the state space s t , action space a t and cumulative reward R t The state space s t Including deployable sensor set S, deployable area set A T , restricted area set A R , sensor deployment location set p D , and the minimum detection probability set P of the deployable area, denoted as: s t ={S,A T ,A R ,p D ,P},This rich state ensures that the DT model can capture all relevant information in order to make informed deployment decisions.

[0103] By defining an action space to cover all possible sensor deployment decisions in each time step, the action space includes deploying a new sensor a deploy , reconfigure existing sensors a reconfigure , remove sensor a remove , denoted as a t ={a deploy ,a reconfigure ,a remove}, depending on the state space and the target area, i.e., the deployment area. This flexible action space allows the model to explore a wide range of deployment strategies to adapt to different conditions and needs.

[0104] The reward function is a key component that directly affects the learning focus of the DT model. It reflects the effectiveness of sensor coverage and the related deployment costs. Therefore, the embodiment of the present invention embeds the dual objectives of the first objective optimization problem function and the second objective optimization problem function into the decision-making process of the DT model, and finally determines the reward function as formula (9).

[0105] However, the low performance problem of DT still exists in the delayed reward signal environment. The problem is that undesirable biases are accumulated in the autoregressive learning of the global signal, which is the result of delayed rewards. This bias manifests itself as an imbalance in the distribution of attention and propagates through the layers of the neural network, resulting in degraded model performance and weakening the effectiveness of the decision model. In particular, in scenarios involving significantly delayed rewards, the semantic association between the reward signal and its corresponding time step is severely damaged. Considering that the reward is only realized in the last time step, all previous target rewards in the trajectory are actually the same. This setting causes most reward signals to be redundant and may be misleading because they hinder the ability of classic self-attention models to effectively identify and learn diverse dependencies and unbiased attention distribution. To address these challenges, an embodiment of the present invention designs a two-layer optimization method to achieve adaptive reward redistribution, specifically, the two-layer optimization method is used to achieve adaptive reward redistribution of the deployment optimization reward function, including:

[0106] Based on the deployment optimization reward function, the initial cumulative reward value of the sensor deployment trajectory is calculated, and the proxy reward function is introduced to re-evaluate the reward of each time step in the sensor deployment trajectory to obtain the reshaped reward value. The initial cumulative reward value and the reshaped reward value are used to construct the reshaped reward function to achieve accurate redistribution of the original reward; the reshaped reward function is used as the third objective optimization problem function, and the verification loss function is used as the fourth objective optimization problem function. By balancing the weight coefficients, a two-layer optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed; the two-layer optimization problem function is solved by the two-layer optimization method to obtain the redistributed cumulative reward value; among them, the third objective optimization problem function is optimized in the upper optimization process, and the fourth objective optimization problem function is optimized in the lower optimization process. The upper and lower layer optimizations are completed once to achieve a single-step optimization.

[0107] Next, we will discuss how to guide sensor deployment in 3D RSNs by reshaping the reward function and explore a two-layer optimization method, emphasizing the improvement of the optimization efficiency of the policy model by alternately iteratively optimizing the reward function. Specifically:

[0108] In DT-based 3D road environment sensor deployment, reward redistribution in sensor deployment trajectory is key. Consider a set of offline sensor deployment trajectories: δ = {δ1,δ2,…,δ n}, the goal is to effectively redistribute the cumulative impact of the trajectory at each time step. This involves reshaping the reward function to help learn an optimized sensor deployment strategy. The embodiment of the present invention introduces a proxy reward function f ψ , which recalibrates the reward at each time step of the sensor deployment trajectory, and the reshaped reward at time step (t) The formula is as follows:

[0109]

[0110] Among them, (s t ,d t ∈δ), and (f ψ :s t ×d t →R t ) is the reward function parameterized as ψ, that is, through the state space s t and sensor deployment trajectory d t Implementing and deploying the optimized reward function R t The mapping relationship of , the detailed implementation process can be found in the existing technology, which will not be repeated here. ψ Assign a revised reward value to each state-deployment decision pair, and That is, the embodiment of the present invention decomposes the reward into the contribution of each step, where the contribution of each step is composed of the reward value R at the current time. t and the reward value R at the next time step t-1 The difference between them is given, that is, it represents the cumulative reward value under the current sensor deployment trajectory, where δ is the rewarded sensor deployment trajectory and |δ| is the length of the sensor deployment trajectory. By transforming the agent reward function f ψ Dynamically tuned to specific road environments and sensor capabilities, DT models are significantly enhanced in learning efficient deployment policies under complex and delayed reward conditions.

[0111] In DT-based sensor deployment, a two-level optimization framework is adopted, which includes two interrelated tasks: optimizing the policy model for sensor placement and optimizing the reward function that guides this optimization. This method is particularly suitable for complex scenarios in three-dimensional road environments, where the temporal distribution and allocation of rewards are crucial to effective sensor deployment strategies. In order to determine the ideal reward function for DT-based sensor deployment, the total reward of the deployment trajectory is decomposed into separate rewards for each step by training a regression model. However, this approach is mainly one-way and relies on the regression of the reward function to train the agent, which may lead to overfitting and lack adaptability in the presence of significantly delayed rewards. To address these limitations, embodiments of the present invention introduce a comprehensive two-way two-level optimization framework. This framework is more suitable for scenarios with longer reward delays and provides multiple advantages. Specifically:

[0112] First, it enables the trained reward function to not only guide the training of the agent, but also enables the agent to iteratively optimize the reward function, forming a dynamic cycle of two-way feedback and joint optimization. Second, it solves the overfitting problem by introducing high-level optimization processes into the optimization of the reward function. The core concept is to use the validation set to evaluate the impact of reward redistribution on the performance of the policy model, thereby guiding the optimization of the reward function. Therefore, all sensor deployment trajectory data δ are divided into two sets: the training set for policy model optimization and the validation set for policy model optimization. and validation set for reward function optimization The focus of lower-level optimization is to enhance the policy model π ξ , which deploys trajectories from a given set of sensors This process involves training the policy model to adapt to the reshaped rewards, ensuring that it accurately reflects the deployment goals and constraints of RSNs. Policy model π ξ is trained to make decisions consistent with the reshaped reward function, aiming to maximize the effectiveness of sensor placement while taking into account environmental factors and deployment constraints. The upper-level task focuses on optimizing the deployment optimization reward function R t , whose purpose is to find strategies that can significantly improve the performance of the policy model on an independent validation set. The reward function configuration of the performance is used to verify the effectiveness of the reshape reward. Deploy the optimized reward function R t It is adjusted to maximize the decision-making performance of the policy model and ensure that the optimized policy model performs best in different RSNs scenarios. In this embodiment of the present invention, the reshape reward function is used as the third objective optimization problem function, and the verification loss function is used as the fourth objective optimization problem function. A two-layer optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed through the balance coefficient. The third objective optimization problem function is expressed as follows:

[0113]

[0114] Among them, C3 represents the third objective optimization problem function, represents the validation set, δ represents the sensor deployment trajectory, |δ| represents the length of the sensor deployment trajectory, R ep (δ) represents the cumulative reward value under the sensor deployment trajectory, represents the reshape reward value at time step t;

[0115] The fourth objective optimization problem function is expressed as follows:

[0116]

[0117] Among them, C4 represents the fourth objective optimization problem function, represents the validation loss function, ξ * (ζ) represents the policy model to be solved, and ζ represents the parameters of the reward model.

[0118] The constructed two-layer optimization problem function is to minimize the verification loss function The condition is that the total reward remains consistent before and after redistribution. In order to transform this constrained two-level optimization problem into an unconstrained problem, a quadratic penalty function is introduced. The final objective optimization problem function is expressed as follows:

[0119]

[0120] Wherein, λ represents the balancing weight coefficient for balancing the third objective optimization problem function and the fourth objective optimization problem function, and the specific value is set according to the actual situation.

[0121] In the DT-based 3D road environment sensor deployment framework, the two-layer optimization process is crucial for model efficiency. However, accurately calculating the upper layer deployment optimization reward function R t The gradient of the underlying policy may be due to its ξ This complexity usually requires multiple iterations to determine the optimal policy π for the lower layer. ξ, which may lead to a time-consuming process. In order to solve this problem, the alternating optimization method is used to optimize the two-layer optimization problem function in the upper and lower layer optimization processes. During the optimization process, the policy model of the current single-step optimization is solved by training the loss function, which is used as an approximate solution of the policy model to be solved for the verification loss function, and the approximate solution is used as the policy model to be solved for the verification loss function in the next single-step optimization to update the verification loss function, and return to use the verification loss function as the fourth objective optimization problem function until the stopping condition is met. This alternating iterative optimization technique effectively separates the parameters ξ and ζ, thereby speeding up the optimization process. The policy model of the current single-step optimization is solved by training the loss function, which is used as an approximate solution of the policy model to be solved for the verification loss function. The formula is expressed as:

[0122]

[0123] Among them, ξ * (ζ) represents the strategy model to be solved for the verification loss function, ξ′(ζ) represents the strategy model for the current single-step optimization, ξ(ζ) represents the current strategy model, α ξ Represents the learning rate of the current single-step optimization, represents the gradient operation, represents the training loss function.

[0124] In this embodiment of the present invention, the principles of meta-learning are used to adjust ξ through single-step training. The approximation of formula (14) ensures efficient computation and accelerates the overall optimization process of sensor deployment in the DT framework, which is crucial for handling complex deployment scenarios with delayed rewards.

[0125] In order to verify the effectiveness of the three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles provided by an embodiment of the present invention, the superiority of the present invention in terms of coverage and detection accuracy in multiple actual driving scenarios was demonstrated by comparison with traditional methods.

[0126] Numerical results clearly demonstrate that the proposed method significantly improves the efficiency of perception, communication, and computational collaboration, as well as resource allocation, compared to existing technologies. Specifically, when evaluating the effectiveness of sensor deployment strategies, two key metrics (coverage area and perception accuracy) were considered in relation to key influencing factors: deployment cost (number of deployed sensors) and road conditions (number of lanes). Figure 3 Compared with traditional methods such as Q-Learning (Q learning method), MEP-PSO (a method that combines the minimum exposure path MEP problem and particle swarm optimization PSO), and Cat Swarm (cat swarm optimization method), Figure 3In the left figure, the horizontal axis represents the number of sensors and the vertical axis represents the coverage rate. It can be seen that as the number of sensors increases from 10 to 50, the coverage rate of the proposed method increases from 83.8% to 97.8%, which can cover the target area more efficiently. The increase in the number of traffic lanes has a certain impact on the coverage rate. Figure 3 In the figure on the right, the horizontal axis represents the number of lanes and the vertical axis represents the coverage rate. It can be seen that the coverage rate of the method proposed in this invention drops from 98.2% for two lanes to 90.3% for six lanes, but this drop is more moderate than that of the traditional method, which shows that the DT method can maintain high adaptability and coverage efficiency under complex road conditions.

[0127] The present invention further analyzes the perception accuracy under different conditions. Figure 4 , Figure 4 The horizontal axis represents the number of deployed sensors, and the vertical axis represents perception accuracy. Compared to traditional methods such as Q-Learning, MEP-PSO, and CatSwarm, perception accuracy shows a clear upward trend with increasing number of deployed sensors, increasing from an initial 85.8% to 97.4%. Perception accuracy accelerates with increasing number of deployed sensors and then stabilizes, indicating a saturation effect at high numbers of deployed sensors. Compared to traditional methods, the proposed method performs well across all deployed sensors. This performance improvement is primarily attributed to the DT approach employed, which integrates the Transformer architecture with RL. This allows the agent to be closely aligned with the vehicle's sensing requirements and outperforms traditional RL methods and static optimization methods.

[0128] In summary, the three-dimensional roadside sensor deployment method for collaborative perception in the Internet of Vehicles proposed in the embodiments of the present invention constructs a road sensor network (RSN) model that can truly reflect complex three-dimensional road environments. Based on this RSN model, a sensor deployment method based on a decision transformer is designed. The Transformer model and multi-head attention mechanism in the decision transformer are used to dynamically adapt to various road conditions and environmental changes, thereby effectively solving the problem of difficult sensor deployment in complex and ever-changing three-dimensional road environments. At the same time, to improve the performance of the decision transformer, the embodiments of the present invention also introduce a two-layer optimization technique. By optimizing the cumulative reward in the decision sequence of the decision transformer, the deployment plan can be adjusted in real time to adapt to complex road and environmental changes. In summary, the embodiments of the present invention improve the coverage and perception accuracy of the three-dimensional road environment, while reducing deployment costs, enhancing economic efficiency and practical application effects, thereby significantly improving the safety and efficiency of autonomous driving and intelligent connected vehicles in complex environments.

[0129] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0130] Although the present invention is described herein in conjunction with various embodiments, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the specification and accompanying drawings in the process of implementing the claimed invention. In the specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components or steps. The fact that certain measures are described in different embodiments does not mean that these measures cannot be combined to produce good results.

[0131] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A three-dimensional roadside sensor deployment method for collaborative perception of vehicle networks, characterized in that: The method comprises: Modeling the RSNs model of a three-dimensional road environment including a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, and a set of restricted areas; Based on the RSNs model of the three-dimensional road environment, constructing a perception detection function of a single sensor in each deployable area, and when multiple sensors cover each deployable area, constructing a global sensor detection function for each deployable area based on the perception detection function of the single sensor; Maximizing the proportion of deployable areas that meet or exceed the minimum detection probability according to the global sensor detection function is used as a first objective optimization problem function, minimizing the total number of sensors in each deployable area is used as a second objective optimization problem function, and constructing a deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function through a balancing coefficient; A decision sequence consisting of a state space, an action space, and a cumulative reward value is constructed, and the deployment strategy is obtained by solving the decision sequence based on a decision transformer; wherein the state space includes a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, a set of restricted areas, and a set of minimum detection probabilities; the action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors; the cumulative reward value is obtained by adaptively redistributing the rewards of the deployment optimization reward function using a two-layer optimization method.

2. The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles according to claim 1 is characterized in that: Construct the perception detection function of a single sensor, including: Four sub-perception detection functions are constructed, in which the detection probability of a single sensor is affected by distance, horizontal detection angle of view, vertical detection angle of view, and obstacles. The perception detection function of a single sensor is constructed by the four sub-perception detection functions.

3. The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles according to claim 2 is characterized in that: The sub-perception detection function affected by distance is expressed as follows: in, represents the sub-perception detection function affected by distance, represents the reference position information of sensor si,j, represents the actual position information of sensor si,j, α represents The slope adjustment parameter, β represents the maximum effective distance of the sensor; The sub-perception detection function affected by the horizontal detection viewing angle is expressed as follows: in, represents the sub-perception detection function affected by the horizontal detection viewing angle, represents the reference horizontal angle of sensor si,j, represents the actual horizontal angle of sensor si,j, ι represents The slope adjustment parameter, η represents the maximum detection angle of the sensor; The sub-perception detection function affected by the vertical detection viewing angle is expressed as: in, represents the sub-perception detection function affected by the vertical detection viewing angle, represents the reference vertical angle of sensor si,j, Indicates the actual vertical angle of sensor si,j, l indicates The slope adjustment parameter, express Value at 50% of maximum coverage; The sub-perception detection function affected by obstacles is expressed as follows: in, represents the sub-perception detection function affected by obstacles, and G represents the obstacle adjustment parameter.

4. The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles according to claim 3 is characterized in that: The perception detection function of a single sensor is constructed, and the formula is expressed as: in, Represents the perception detection function of a single sensor, that is Indicates the deployable area in the deployable area set AT The perception detection function of sensor si,j, NT represents the number of deployable regions. represents the i-th deployable area; When multiple sensors are covered, a global sensor detection function is constructed for each deployable area, which is expressed as: in, Represents the global sensor detection function, which can be deployed as a set of regions The perception detection function of the deployable sensor set si is si,j represents the jth deployable sensor in si, Indicates the deployable area The maximum number of deployable sensors in .

5. The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles according to claim 1 is characterized in that: The first objective optimization problem function is expressed as follows: Where C1 represents the first objective optimization problem function, card{} represents the number of elements in the set, and AT represents the set of deployable areas. NT represents the number of deployable regions. represents the i-th deployable area, P represents the minimum detection probability set, Indicates the deployable area The minimum detection probability, si represents the deployable area A collection of deployable sensors, Indicates the deployable area Global sensor detection function; The second objective optimization problem function is expressed as follows: in, Indicates the deployable area The number of deployable sensors in Indicates the deployable area The set of sensor deployable locations, Indicates the deployable area The position information of the jth sensor in .

6. The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles according to claim 1 is characterized in that: Adaptive reward redistribution of the deployment optimization reward function is achieved through a two-layer optimization approach, including: An initial cumulative reward value of the sensor deployment trajectory is calculated based on the deployment optimization reward function, and a proxy reward function is introduced to re-evaluate the reward of each time step in the sensor deployment trajectory to obtain a reshaping reward value, and a reshaping reward function is constructed using the initial cumulative reward value and the reshaping reward value; The reshaped reward function is used as the third objective optimization problem function, the verification loss function is used as the fourth objective optimization problem function, and a two-layer optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed by balancing the weight coefficients; The double-layer optimization problem function is solved by a double-layer optimization method to obtain the reallocated cumulative reward value; wherein, the third objective optimization problem function is optimized in the upper-layer optimization process, and the fourth objective optimization problem function is optimized in the lower-layer optimization process, completing the upper-layer and lower-layer optimization once to achieve a single-step optimization.

7. The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles according to claim 6 is characterized in that: The third objective optimization problem function is expressed as follows: Among them, C3 represents the third objective optimization problem function, represents the validation set, δ represents the sensor deployment trajectory, |δ represents the length of the sensor deployment trajectory, Rep(δ) represents the cumulative reward value of the sensor deployment trajectory, represents the reshape reward value at time step t; The fourth objective optimization problem function is expressed as follows: Among them, C4 represents the fourth objective optimization problem function, Represents the verification loss function, ξ*(ζ) represents the policy model to be solved, and ζ represents the parameters of the reward model.

8. The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles according to claim 6 is characterized in that: In the optimization process of the upper layer and the lower layer, the alternating optimization method is used to optimize the double-layer optimization problem function; During the optimization process, the strategy model of the current single-step optimization is solved by training the loss function as an approximate solution of the strategy model to be solved for the verification loss function, and the approximate solution is used as the strategy model to be solved for the verification loss function in the next single-step optimization to update the verification loss function. The verification loss function is returned as the fourth objective optimization problem function until the stopping condition is met.

9. The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles according to claim 8, characterized in that: The current single-step optimization strategy model is solved by training the loss function, which is used as an approximate solution of the strategy model to be solved to verify the loss function. The formula is expressed as: Among them, ξ′(ζ) represents the policy model of the current single-step optimization, ξ(ζ) represents the current policy model, αξ represents the learning rate of the current single-step optimization, represents the gradient operation, represents the training loss function.

Images