Three-dimensional roadside sensor deployment method for collaborative awareness of Internet of Vehicles

Abstract

The invention discloses a three-dimensional roadside sensor deployment method for cooperative perception of the Internet of Vehicles. The method comprises the following steps: modeling an RSNs model of a three-dimensional road environment; constructing a sensing detection function of a single sensor based on an RSNs model, and constructing a global sensor detection function of each deployable area according to the sensing detection function of the single sensor when coverage of multiple sensors exists; taking the proportion of the deployable areas which meet or exceed the minimum detection probability according to the global sensor detection function maximization as a first target optimization problem, taking the total number of sensors which are minimized in each deployable area as a second target optimization problem, and constructing a deployment optimization reward function consisting of the first target optimization problem function and the second target optimization problem function; constructing a decision sequence, and solving the decision sequence based on a decision converter to obtain a deployment strategy; wherein the accumulated reward value is obtained by reward redistribution of the deployment optimization reward function through a double-layer optimization method. According to the invention, the sensor deployment of the three-dimensional road environment is realized.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle networking, and specifically relates to a three-dimensional roadside sensor deployment method for vehicle networking collaborative perception. Background Art

[0002] In the Internet of Vehicles environment, collaborative perception technology is crucial to the safety and efficiency of autonomous vehicles. The Internet of Vehicles achieves collaborative perception by deploying multi-source sensors (such as cameras, lidar, radar, etc.) on both sides of the road, and shares their perception data with autonomous vehicles in real time, significantly improving the vehicle's perception ability in complex road environments. Autonomous driving and intelligent connected vehicles need to achieve efficient and safe driving in various complex scenarios, which puts higher requirements on the deployment of sensors.

[0003] Existing technical solutions mainly focus on two-dimensional sensor deployment optimization, static three-dimensional scene modeling, and local optimization algorithms. Traditional two-dimensional sensor deployment optimization methods are usually based on simple geometric models, which cover the target area by optimizing the position and angle of the sensor and maximize the perception coverage by calculating the optimal position and viewing angle of the sensor. However, these methods fail to consider the complex three-dimensional road environment and cannot solve the blind spot problem in practical applications. Some studies have tried to expand sensor deployment to three-dimensional scenes, but they usually adopt idealized three-dimensional models, ignoring the actual complexity of the road and the actual constraints in sensor deployment. These methods perform well in experimental environments, but in real road environments, their effects are often limited and difficult to cope with the complexity and diversity of actual roads. Traditional two-dimensional and ideal three-dimensional scene sensor deployment methods usually use static optimization algorithms, such as particle swarm optimization and genetic algorithms. These methods perform well in initial deployment, but lack the ability to dynamically adapt to environmental changes. In practical applications, changes in traffic conditions and environmental conditions can affect the performance of sensors. Static optimization algorithms are difficult to adjust deployment strategies in real time, resulting in reduced perception coverage and accuracy. For example, patent CN113743479B proposes an end-edge-cloud vehicle-road collaborative fusion perception architecture, which optimizes information flow through the distributed coordination theory of multiple agents. Although this method enhances the environmental adaptability of the system, it still faces challenges in handling real-time data transmission and processing in highly dynamic traffic scenarios. In addition, patent CN117294738B proposes an optimized deployment and perception method for autonomous driving sensors, which enhances the system's perception capabilities by optimizing the spatial coverage and location weights of sensors. However, this method still has limitations when dealing with complex and dynamically changing road environments.

[0004] In summary, it can be seen that existing collaborative perception technologies face many challenges in sensor deployment, especially in complex and changeable urban road environments. Traditional two-dimensional or idealized three-dimensional scene modeling methods are difficult to cope with the complexity of actual roads, which leads to limitations in coverage, perception accuracy, and real-time performance of existing sensor networks. In addition, traditional two-dimensional and idealized three-dimensional scene sensor deployment methods usually use static optimization algorithms, which lack the ability to dynamically adapt to environmental changes and are difficult to provide reliable perception assistance under various sudden road conditions and weather changes. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a three-dimensional roadside sensor deployment method for vehicle network collaborative perception. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] An embodiment of the present invention provides a method for deploying three-dimensional roadside sensors for collaborative perception of an Internet of Vehicles, the method comprising:

[0007] Modeling a RSNs model of a three-dimensional road environment including a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, and a set of restricted areas;

[0008] Based on the RSNs model of the three-dimensional road environment, construct a perception detection function of a single sensor in each deployable area, and when there are multiple sensors covering each deployable area, construct a global sensor detection function of each deployable area according to the perception detection function of the single sensor;

[0009] The proportion of deployable areas that maximizes or exceeds the minimum detection probability according to the global sensor detection function is used as the first objective optimization problem function, the total number of sensors in each deployable area is minimized as the second objective optimization problem function, and a deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function is constructed through a balancing coefficient;

[0010] A decision sequence consisting of a state space, an action space, and a cumulative reward value is constructed, and a deployment strategy is obtained by solving the decision sequence based on a decision converter; wherein the state space includes a set of deployable sensors, a set of sensor deployable positions, a set of deployable areas, a set of restricted areas, and a set of minimum detection probabilities; the action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors; and the cumulative reward value is obtained by adaptively redistributing the deployment optimization reward function using a two-layer optimization method.

[0011] In one embodiment of the present invention, constructing a perception detection function of a single sensor includes:

[0012] Four sub-perception detection functions are constructed respectively, in which the detection probability of a single sensor is affected by distance, horizontal detection angle of view, vertical detection angle of view and obstacles;

[0013] The perception detection function of a single sensor is constructed by the four sub-perception detection functions.

[0014] In one embodiment of the present invention, the sub-perception detection function affected by distance is expressed as follows:

[0015]

[0016] in, represents the sub-perception detection function affected by distance, Indicates sensor i,j The reference position information, Indicates sensor i,j The actual position information, α represents The slope adjustment parameter, β represents the maximum effective distance of the sensor;

[0017] The sub-perception detection function affected by the horizontal detection viewing angle is expressed as:

[0018]

[0019] in, represents the sub-perceptual detection function affected by the horizontal detection viewing angle, Indicates sensor i,j The reference horizontal angle, Indicates sensor i,j The actual horizontal angle, ι represents The slope adjustment parameter, η represents the maximum detection angle of the sensor;

[0020] The sub-perception detection function affected by the vertical detection viewing angle is expressed as:

[0021]

[0022] in, represents the sub-perceptual detection function affected by the vertical detection viewing angle, Indicates sensor i,j The base vertical angle, Indicates sensor i,j The actual vertical angle, ι represents The slope adjustment parameter is express Value at 50% of maximum coverage;

[0023] The sub-perception detection function affected by obstacles is expressed as:

[0024]

[0025] in, represents the sub-perception detection function affected by obstacles, and G represents the obstacle adjustment parameter.

[0026] In one embodiment of the present invention, the perception detection function of a single sensor is constructed, and the formula is expressed as:

[0027]

[0028] in, Represents the perception detection function of a single sensor, that is Indicates the deployable area set A T Deployable area Sensors i,j The perceptual detection function, N T Indicates the number of deployable regions. represents the i-th deployable area;

[0029] When multiple sensors are covered, a global sensor detection function is constructed for each deployable area, and the formula is expressed as:

[0030]

[0031] in, Represents the global sensor detection function, that is, the set of deployment areas Deployable sensor collections i The perceptual detection function, s i,j Indicates i The jth deployable sensor in Indicates the deployable area The maximum number of deployable sensors in .

[0032] In one embodiment of the present invention, the first objective optimization problem function is expressed as:

[0033]

[0034] Among them, C1 represents the first objective optimization problem function, card{} represents the number of elements in the set, and A T Represents a set of deployable regions. N T Indicates the number of deployable regions. represents the i-th deployable area, P represents the minimum detection probability set, Indicates the deployable area The minimum detection probability, s i Indicates the deployable area A collection of deployable sensors, Indicates the deployable area The global sensor detection function;

[0035] The second objective optimization problem function is expressed as:

[0036]

[0037] in, Indicates the deployable area The number of deployable sensors in Indicates the deployable area The set of locations where sensors can be deployed, Indicates the deployable area The position information of the jth sensor in .

[0038] In one embodiment of the present invention, adaptive reward redistribution of the deployment optimization reward function is achieved through a two-layer optimization method, including:

[0039] An initial cumulative reward value of the sensor deployment trajectory is calculated based on the deployment optimization reward function, and a proxy reward function is introduced to re-evaluate the reward of each time step in the sensor deployment trajectory to obtain a reshaping reward value, and a reshaping reward function is constructed using the initial cumulative reward value and the reshaping reward value;

[0040] The reshape reward function is used as the third objective optimization problem function, the verification loss function is used as the fourth objective optimization problem function, and a two-layer optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed by balancing the weight coefficients;

[0041] The double-layer optimization problem function is solved by a double-layer optimization method to obtain the reallocated cumulative reward value; wherein, the third objective optimization problem function is optimized in the upper-layer optimization process, and the fourth objective optimization problem function is optimized in the lower-layer optimization process, completing the upper-layer and lower-layer optimizations once to achieve a single-step optimization.

[0042] In one embodiment of the present invention, the third objective optimization problem function is expressed as:

[0043]

[0044] Among them, C3 represents the third objective optimization problem function, represents the validation set, δ represents the sensor deployment trajectory, |δ| represents the length of the sensor deployment trajectory, R ep(δ) represents the cumulative reward value of the sensor deployment trajectory, represents the reshape reward value at time step t;

[0045] The fourth objective optimization problem function is expressed as follows:

[0046]

[0047] Among them, C4 represents the fourth objective optimization problem function, represents the validation loss function, ξ * (ζ) represents the strategy model to be solved, and ζ represents the parameters of the reward model.

[0048] In one embodiment of the present invention, during the optimization process of the upper layer and the lower layer, an alternating optimization method is used to optimize the double-layer optimization problem function; wherein,

[0049] During the optimization process, the strategy model of the current single-step optimization is solved by training the loss function as an approximate solution to the strategy model to be solved for the verification loss function, and the approximate solution is used as the strategy model to be solved for the verification loss function in the next single-step optimization to update the verification loss function, and return to use the verification loss function as the fourth objective optimization problem function until the stopping condition is met.

[0050] In one embodiment of the present invention, the current single-step optimized strategy model is solved by training the loss function as an approximate solution of the strategy model to be solved for verifying the loss function. The formula is expressed as:

[0051]

[0052] Among them, ξ′(ζ) represents the current single-step optimization strategy model, ξ(ζ) represents the current strategy model, α ξ Represents the learning rate of the current single-step optimization, represents the gradient operation, Represents the training loss function.

[0053] Beneficial effects of the present invention:

[0054] The three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles proposed in the present invention constructs a road sensor network RSNs model that can truly reflect the complex three-dimensional road environment, and designs a sensor deployment method based on the decision converter based on the RSNs model, which uses the Transformer model and multi-head attention mechanism in the decision converter to dynamically adapt to various road conditions and environmental changes, thereby well solving the problem of the difficulty of deploying sensors in a complex and constantly changing three-dimensional road environment; at the same time, in order to improve the performance of the decision converter, the embodiment of the present invention also introduces a two-layer optimization technology, which optimizes the cumulative rewards in the decision sequence of the decision converter through the two-layer optimization technology, and can adjust the deployment plan in real time to adapt to complex road and environmental changes. In general, the present invention improves the coverage and perception accuracy of the three-dimensional road environment, while reducing the deployment cost, enhancing the economy and practical application effect, thereby significantly improving the safety and efficiency of autonomous driving and intelligent networked vehicles in complex environments.

[0055] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of a method for deploying three-dimensional roadside sensors for collaborative perception of Internet of Vehicles provided by an embodiment of the present invention;

[0057] Figure 2 is a schematic diagram of an RSNs model of a three-dimensional road environment provided by an embodiment of the present invention;

[0058] Figure 3 This is the intention of the embodiment of the present invention to show how the coverage varies with the number of sensors and the number of lanes;

[0059] Figure 4 It is a schematic diagram of the change of the perception accuracy with the number of deployed sensors provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0061] See also Figure 1 The embodiment of the present invention provides a method for deploying three-dimensional roadside sensors for collaborative perception of Internet of Vehicles, which specifically includes the following steps:

[0062] S10. Modeling an RSNs model of a three-dimensional road environment including a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, and a set of restricted areas.

[0063] The RSNs model of the three-dimensional road environment modeled by the embodiment of the present invention is as follows: Figure 2 As shown, it may include a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, and a set of restricted areas.

[0064] Assume that the RSNs model includes: a deployable sensor set consisting of N deployable sensors, denoted as S, N T Indicates the number of deployable regions, s i represents the set of deployable sensors in the ith deployable area, s i,j Indicates i The jth deployable sensor in represents the maximum number of deployable sensors in the ith deployable area; N T A deployable area set consisting of deployable areas T , represents the i-th deployable area, each deployable area is a rectangular area, is defined as the coordinates of the angle opposite its diagonal, Indicates the deployable area The coordinates of the upper left corner, Indicates the deployable area The coordinates in the lower right corner define the boundaries and dimensions of the deployable area, which helps to accurately assess the coverage requirements. In order to ensure the reliability of the autonomous driving system, it is necessary to ensure that the perception probability within the deployable area is not lower than the set threshold.

[0065] In the actual road environment, not all places above and on both sides of the road are places where sensors can be deployed, such as roadside green belts, guardrails, etc. Therefore, when considering sensor deployment, it is necessary to consider areas beside the road where the installation of sensors is explicitly prohibited. R The restricted area set consisting of restricted areas is denoted as AR. represents the kth restricted area, each restricted area is a rectangular area, Indicates restricted area The coordinates of the upper left corner, Indicates restricted area The coordinates in the lower right corner ensure that no sensor of any type is installed within this restricted area by accurately defining the location and size of the restricted area. Establishing this restricted area is critical to preventing interference with important operations or sensitive areas for sensors.

[0066] The sensors required for cooperative sensing are strategically arranged on both sides of the road or above the road to ensure that normal traffic flow is not affected. Assume that the RSNs model of the embodiment of the present invention includes: T The sensor deployable location set consists of sensor deployable locations, denoted as p D , Indicates the deployable area The set of locations where sensors can be deployed, Indicates the deployable area The position information of the jth sensor in, for example, the three-dimensional coordinates can be recorded as Considering the impact of sensor deployment height, here The function represents the position (x D ,y D ) horizontal height function, z i,j Indicates the height of the deployed sensor from the ground. Embodiments of the present invention enhance the integration and analysis of data collection by determining the exact location of each sensor within the deployable area.

[0067] S20. Based on the RSNs model of the three-dimensional road environment, a perception detection function of a single sensor is constructed, and when multiple sensors are covered, a global sensor detection function is constructed according to the perception detection function of a single sensor.

[0068] The actual sensor behavior is not only determined by the distance between the sensor and the target and the area covered by the sensor, but the perception model is mainly affected by the perception distance, horizontal viewing angle, vertical viewing angle and obstacles. Therefore, the embodiment of the present invention constructs a perception detection function of a single sensor, including:

[0069] Four sub-perception detection functions are constructed respectively, in which the detection probability of a single sensor is affected by distance, horizontal detection angle of view, vertical detection angle of view and obstacles; and the perception detection function of a single sensor is constructed by the four sub-perception detection functions. Specifically:

[0070] In autonomous driving systems, sensor efficiency is usually evaluated by its perception detection probability. It is worth noting that as the perception detection distance increases, the perception detection ability of the sensor tends to weaken. The key factor affecting this pattern is the Euclidean distance used to quantify the separation between the sensor and its target. The final constructed sub-perception detection function affected by the distance is expressed as:

[0071]

[0072] in, represents the sub-perception detection function affected by distance, Indicates sensor i,j The reference position information, Indicates sensor i,j The actual position information, α represents The slope adjustment parameter plays an important role in adjusting the slope of the function, affecting the degree to which the perceived detection probability decreases rapidly with increasing distance. β represents the maximum effective distance of the sensor.

[0073] In the field of perception detection of complex sensors, both the perception detection distance and the horizontal deviation of the sensor's central axis are important. As the horizontal detection angle of view deviates from the central axis, the perception detection probability generally decreases. The final constructed sub-perception detection function affected by the horizontal detection angle of view is expressed as:

[0074]

[0075] in, represents the sub-perceptual detection function affected by the horizontal detection viewing angle, Indicates sensor i,j The reference horizontal angle, Indicates sensor i,j The actual horizontal angle, ι represents The slope adjustment parameter reflects the rate at which the perceived detection probability decreases as the horizontal viewing angle becomes wider. η represents the maximum detection angle of the sensor, beyond which the perceived detection probability drops sharply.

[0076] Similar to horizontal perspective perception, vertical detection perspective also affects the perception detection probability, which is related to the central axis of the sensor, which means that if the object is close to the vertical central axis, the sensor is more likely to detect it. The final constructed sub-perception detection function affected by the vertical detection perspective is expressed as:

[0077]

[0078] in, represents the sub-perceptual detection function affected by the vertical detection viewing angle, Indicates sensor i,j The base vertical angle, Indicates sensor i,j The actual vertical angle, ι represents The slope adjustment parameter reflects the rate at which the perceived detection probability decreases as the vertical viewing angle widens. express Value at 50% of maximum coverage.

[0079] Sensors in autonomous vehicle systems are good at environmental perception, but may encounter challenges that hinder their perception capabilities. Obstacles, whether physical barriers or adverse weather conditions, can significantly affect the line of sight or the quality of perception detection, resulting in a reduced probability of perception detection within the predetermined detection area. Finally, the embodiment of the present invention uses a low-complexity Bresenham algorithm to construct the impact of obstacles on the perception detection probability, and constructs a sub-perception detection function affected by obstacles, which is expressed as follows:

[0080]

[0081] in, represents the sub-perception detection function affected by obstacles, G represents the obstacle adjustment parameter, which is a variable determined by the obstacle type, for example, when the obstacle is a transparent plastic bag and a red plastic bag, G may be 0.9 and 0.1 respectively. The sub-perception detection function affected by obstacles illustrates the change in perception ability due to obstacles. In the presence of obstacles, the impact of the perception detection probability is quantified by the sub-perception detection function affected by obstacles. In contrast, when the obstacle is not present, the probability is 1, indicating that perception is unimpeded.

[0082] Finally, the perception detection capability of a single sensor in the embodiment of the present invention is constructed by four sub-perception detection functions affected by distance, horizontal detection angle of view, vertical detection angle of view and obstacles. The constructed perception detection function of a single sensor is expressed as follows:

[0083]

[0084] in, Represents the perception detection function of a single sensor, that is Indicates the deployable area set A T Deployable area Sensors i,j The perceptual detection function.

[0085] In an autonomous driving system, a single sensor has inherent limitations in its coverage. When sensors work together, the probability of perception detection within the target area, i.e., the deployment area, increases significantly. Although a single sensor may only cover a portion of the target area, the combination of multiple sensors ensures full coverage, thereby increasing the likelihood of accurate perception detection within the entire target area. When the target area is covered by multiple sensors, the global sensor detection function is constructed, which is expressed as:

[0086]

[0087] in, Represents the global sensor detection function, that is, the set of deployable areas Deployable sensor collections i The perceptual detection function, s i,j Indicates i The jth deployable sensor in Indicates the deployable area The maximum number of deployable sensors in .

[0088] S30. Maximizing the proportion of deployable areas that meet or exceed the minimum detection probability according to the global sensor detection function is used as the first objective optimization problem function, minimizing the total number of sensors in each deployable area is used as the second objective optimization problem function, and constructing a deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function through a balancing coefficient.

[0089] However, achieving such extensive perception coverage faces challenges, especially due to budget constraints. Therefore, constraints are introduced, emphasizing the balance between maximizing perception detection capabilities and cost-effectiveness. The main goal is to achieve the highest detection coverage through multi-sensor collaboration and effectively utilize available resources in an economical manner. Therefore, the goal of the present invention is to develop a sensor deployment strategy for 3D RSNs that maximizes perception assistance while minimizing the number of sensors deployed, while considering deployable areas and restricted areas, that is, covering the deployable area with one sensor network while ensuring that the perception probability within the deployable area is not less than the minimum detection probability of each deployable area Finally, the embodiment of the present invention maximizes the proportion of deployable areas that meet or exceed the minimum detection probability according to the global sensor detection function as the first objective optimization problem function, and the formula is expressed as:

[0090]

[0091] Among them, C1 represents the first objective optimization problem function, card{} represents the number of elements in the set, and A T Represents a set of deployable regions. N T Indicates the number of deployable regions. represents the i-th deployable area, P represents the minimum detection probability set, Indicates the deployable area The minimum detection probability is Indicates the deployable area Global sensor detection function.

[0092] At the same time, minimizing the total number of sensors in each deployable area is taken as the second objective optimization problem function, and the formula is expressed as:

[0093]

[0094] in, Indicates the deployable area The number of deployable sensors in Indicates the deployable area The set of locations where sensors can be deployed, Indicates the deployable area The position information of the jth sensor in .

[0095] The deployment optimization reward function consisting of the first objective optimization problem function and the second objective optimization problem function is constructed through the balance coefficient. The formula is expressed as:

[0096] R t =λ1×C1-λ2×C2

[0097]

[0098] Among them, λ1 and λ2 represent the balance coefficients, and their specific values ​​are set according to the actual situation. It can be seen that the deployment optimization reward function includes constraints to ensure that sensors are not placed in restricted areas, and the total number of sensors at any location does not exceed i.e., the maximum number of sensors allowed at that location, i.e., the deployment optimization reward function allows the optimization of RSNs in a 3D road environment, balancing the need for comprehensive coverage with practical constraints such as confined areas and budgetary restrictions.

[0099] S40, construct a decision sequence consisting of a state space, an action space, and a cumulative reward value, and obtain a deployment strategy by solving the decision sequence based on a decision converter; wherein the state space includes a set of deployable sensors, a set of sensor deployable locations, a deployable area, a restricted area, and a minimum detection probability of a deployable area; the action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors; the cumulative reward value is obtained by adaptively redistributing the deployment optimization reward function using a two-layer optimization method.

[0100] In order to solve the deployment optimization reward function problem in the deployment of three-dimensional RSNs, an embodiment of the present invention proposes a method based on a decision transformer (DT). This method uses reinforcement learning technology, especially the Transformer model, to optimize the deployment of sensors under complex constraints, and can flexibly respond to challenges such as insufficient coverage of roadside sensors. In addition, the multi-head attention mechanism in DT further enhances the strategy, enabling it to simultaneously consider changes in external conditions and traffic patterns, becoming a highly effective solution for deploying sensors in complex and changing three-dimensional road environments.

[0101] As a sequence modeling RL (Reinforcement Learning) method, DT provides a novel way to handle complex optimization problems such as sensor deployment. DT optimizes deployment strategies through its sequence modeling capabilities. By analyzing historical sequences, the model learns to predict the results of various actions and thus improves its future decision-making strategies. This optimization is a continuous process that evolves as the model gains more insights in each subsequent deployment scenario. The training and decision-making process is aligned with our main goals of maximizing coverage and minimizing cost. The performance of the model is continuously evaluated against these goals, ensuring that the deployment strategies it develops are both practical and effective. This alignment is critical to ensuring that the model's decisions are not only theoretically reasonable, but also feasible in practical applications. After deployment, the strategies recommended by DT are evaluated based on how well they achieve the set goals. This evaluation phase is critical to identifying areas where the model's performance can be improved. Feedback from this phase feeds into further training cycles, allowing the model to learn from actual deployments and continuously improve its decision accuracy.

[0102] Different from traditional RL methods, DT redefines the problem as a sequence modeling task, which can make more detailed decisions based on historical data. Therefore, the basis of the embodiment of the present invention is to build a decision sequence, which is the key to DT understanding and learning. The decision sequence is composed of the state space s t , action space a t and cumulative reward R t The state space s t Including deployable sensor set S, deployable area set A T , restricted area set A R , the set of sensor deployable locations p D , and the minimum detection probability set P of the deployable area, denoted as: s t ={S,A T ,A R ,p D ,P},This rich state ensures that the DT model can capture all relevant information in order to make informed deployment decisions.

[0103] By defining an action space to cover all possible sensor deployment decisions at each time step, the action space includes deploying a new sensor a deploy , reconfigure existing sensors a reconfigure , remove sensor a remove , denoted as a t ={a deploy ,a reconfigure ,a remove}, depending on the state space and the target area, i.e., the deployment area. This flexible action space allows the model to explore a wide range of deployment strategies to adapt to different conditions and requirements.

[0104] The reward function is a key component that directly affects the learning focus of the DT model. It reflects the effectiveness of sensor coverage and the related deployment costs. Therefore, the embodiment of the present invention embeds the dual objectives of the first objective optimization problem function and the second objective optimization problem function into the decision-making process of the DT model, and finally determines the reward function as formula (9).

[0105] However, the low performance of DT still exists in the delayed reward signal environment. The problem is that undesirable biases are accumulated in the autoregressive learning of the global signal, which is the result of delayed rewards. This bias manifests itself as an imbalance in the distribution of attention and propagates through the layers of the neural network, resulting in degraded model performance and weakening the effectiveness of the decision model. In particular, in scenarios involving significantly delayed rewards, the semantic association between the reward signal and its corresponding time step is severely damaged. Considering that the reward is only realized in the last time step, all previous target rewards in the trajectory are actually the same. This setting causes most reward signals to be redundant and may be misleading because they hinder the ability of the classic self-attention model to effectively identify and learn diverse dependencies and unbiased attention distribution. In order to address these challenges, an embodiment of the present invention designs a two-layer optimization method to achieve adaptive reward redistribution, specifically, the adaptive reward redistribution of the deployment optimization reward function is achieved through a two-layer optimization method, including:

[0106] The initial cumulative reward value of the sensor deployment trajectory is calculated based on the deployment optimization reward function, and the proxy reward function is introduced to re-evaluate the reward of each time step in the sensor deployment trajectory to obtain a reshaped reward value. The initial cumulative reward value and the reshaped reward value are used to construct a reshaped reward function to achieve accurate redistribution of the original reward; the reshaped reward function is used as the third objective optimization problem function, and the verification loss function is used as the fourth objective optimization problem function. A two-layer optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed by balancing the weight coefficients; the two-layer optimization problem function is solved by a two-layer optimization method to obtain the redistributed cumulative reward value; wherein, the third objective optimization problem function is optimized in the upper optimization process, and the fourth objective optimization problem function is optimized in the lower optimization process, completing an upper and lower optimization to achieve a single-step optimization.

[0107] Next, we will discuss how to guide sensor deployment in 3D RSNs by reshaping the reward function and explore a two-layer optimization method that emphasizes improving the optimization efficiency of the policy model by alternately iteratively optimizing the reward function. Specifically:

[0108] In DT-based 3D road environment sensor deployment, reward redistribution in sensor deployment trajectories is critical. Consider a set of offline sensor deployment trajectories: δ = {δ1, δ2, …, δ n}, the goal is to effectively redistribute the cumulative impact of the trajectory at each time step. This involves reshaping the reward function to help learn an optimized sensor deployment strategy. The embodiment of the present invention introduces a proxy reward function f ψ , which recalibrates the reward at each time step of the sensor deployment trajectory. The reshaped reward at time step (t) is The formula is as follows:

[0109]

[0110] Among them, (s t ,d t ∈δ), and (f ψ :s t ×d t →R t ) is the reward function parameterized as ψ, that is, through the state space s t and sensor deployment trajectory d t Implementing and deploying the optimized reward function R t The mapping relationship of , the detailed implementation process can be found in the prior art, and will not be repeated here. ψ Assign a revised reward value to each state-deployment decision pair, and That is, the embodiment of the present invention decomposes the reward into the contribution of each step, where the contribution of each step is composed of the reward value R at the current time t and the reward value R at the next time step t-1 The difference between them is given, that is, it represents the cumulative reward value under the current sensor deployment trajectory, where δ is the rewarded sensor deployment trajectory and |δ| is the length of the sensor deployment trajectory. By transforming the proxy reward function f ψ Tuned to the specific road environment and sensor capabilities, the DT model is significantly enhanced in learning efficient deployment policies under complex and delayed reward conditions.

[0111] In DT-based sensor deployment, a two-level optimization framework is adopted, which includes two interrelated tasks: optimizing the policy model for sensor placement and optimizing the reward function that guides this optimization. This method is particularly suitable for complex scenarios in three-dimensional road environments, where the temporal distribution and allocation of rewards are crucial to effective sensor deployment strategies. In order to determine the ideal reward function for DT-based sensor deployment, the total reward of the deployment trajectory is decomposed into separate rewards for each step by training a regression model. However, this approach is mainly one-way and relies on the regression of the reward function to train the agent, which may lead to overfitting and lack adaptability in the presence of significantly delayed rewards. To address these limitations, embodiments of the present invention introduce a comprehensive bidirectional two-level optimization framework. This framework is more suitable for scenarios with longer reward delays and provides multiple advantages. Specifically:

[0112] First, it enables the trained reward function to not only guide the training of the agent, but also enables the agent to iteratively optimize the reward function, forming a dynamic cycle of two-way feedback and joint optimization. Second, it solves the overfitting problem by introducing high-level optimization processes into the optimization of the reward function. The core concept is to use the validation set to evaluate the impact of reward redistribution on the performance of the policy model, thereby guiding the optimization of the reward function. Therefore, all sensor deployment trajectory data δ are divided into two sets: the training set for policy model optimization and the validation set for policy model optimization. and validation set for reward function optimization The focus of lower-level optimization is to enhance the policy model π ξ , which deploys trajectories from a given set of sensors This process involves training the policy model to adapt to the reshaped rewards, ensuring that it accurately reflects the deployment goals and constraints of RSNs. Policy model π ξ is trained to make decisions consistent with a reshaped reward function that aims to maximize the effectiveness of sensor placement while taking into account environmental factors and deployment constraints. The upper-level task focuses on optimizing the deployment optimization reward function R t , whose purpose is to find strategies that can significantly improve the performance of the policy model on an independent validation set. The reward function configuration of the performance is used to verify the effectiveness of the reshaping reward. Deployment optimization reward function R t is adjusted to maximize the decision-making performance of the policy model and ensure that the optimized policy model performs best in different RSNs scenarios. In this embodiment of the present invention, the reshape reward function is used as the third objective optimization problem function, and the verification loss function is used as the fourth objective optimization problem function. A double-layer optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed through a balance coefficient, wherein the third objective optimization problem function is expressed as follows:

[0113]

[0114] Among them, C3 represents the third objective optimization problem function, represents the validation set, δ represents the sensor deployment trajectory, |δ| represents the length of the sensor deployment trajectory, R ep (δ) represents the cumulative reward value under the sensor deployment trajectory, represents the reshape reward value at time step t;

[0115] The fourth objective optimization problem function is expressed as:

[0116]

[0117] Among them, C4 represents the fourth objective optimization problem function, represents the validation loss function, ξ * (ζ) represents the strategy model to be solved, and ζ represents the parameters of the reward model.

[0118] The constructed two-layer optimization problem function is to minimize the verification loss function The condition is that the total reward remains consistent before and after redistribution, that is, in order to transform this constrained two-level optimization problem into an unconstrained problem, a quadratic penalty function is introduced, and the final target optimization problem function is expressed as:

[0119]

[0120] Wherein, λ represents the balancing weight coefficient of balancing the third objective optimization problem function and the fourth objective optimization problem function, and the specific value is set according to the actual situation.

[0121] In the DT-based 3D road environment sensor deployment framework, the two-layer optimization process is crucial to model efficiency. However, accurately calculating the upper layer deployment optimization reward function R t The gradient of may be due to its interaction with the underlying policy optimization π ξ This complexity usually requires multiple iterations to determine the optimal strategy π for the underlying layer. ξ, resulting in a potentially time-consuming process. In order to solve this problem, the alternating optimization method is used to optimize the double-layer optimization problem function in the upper and lower optimization processes. During the optimization process, the policy model of the current single-step optimization is solved by training the loss function as an approximate solution of the policy model to be solved for the verification loss function, and the approximate solution is used as the policy model to be solved for the verification loss function in the next single-step optimization to update the verification loss function, and return to use the verification loss function as the fourth objective optimization problem function until the stopping condition is met. This alternating iterative optimization technique effectively separates the parameters ξ and ζ, thereby speeding up the optimization process. The policy model of the current single-step optimization is solved by training the loss function as an approximate solution of the policy model to be solved for the verification loss function. The formula is expressed as:

[0122]

[0123] Among them, ξ * (ζ) represents the strategy model to be solved for the verification loss function, ξ′(ζ) represents the strategy model for the current single-step optimization, ξ(ζ) represents the current strategy model, and α ξ Represents the learning rate of the current single-step optimization, represents the gradient operation, Represents the training loss function.

[0124] The embodiment of the present invention draws on the principle of meta-learning to adjust ξ through single-step training. The approximation scheme of formula (14) ensures efficient calculation and accelerates the overall optimization process of sensor deployment in the DT framework, which is crucial for dealing with complex deployment scenarios with delayed rewards.

[0125] In order to verify the effectiveness of the three-dimensional roadside sensor deployment method for collaborative perception of vehicle networks provided by an embodiment of the present invention, the superiority of the present invention in terms of coverage and detection accuracy in multiple actual driving scenarios is demonstrated by comparison with traditional methods.

[0126] The numerical results clearly show that the proposed method has significant improvements over the existing technologies in terms of improving the efficiency of perception, communication and computing collaboration, as well as resource allocation efficiency. Specifically, when evaluating the effectiveness of the sensor deployment strategy, the performance of two key indicators (coverage area and perception accuracy) in key influencing factors, namely, deployment cost (number of deployed sensors) and road conditions (number of lanes), is considered. Figure 3 Compared with traditional methods such as Q-Learning (Q learning method), MEP-PSO (a method that combines the minimum exposure path MEP problem and particle swarm optimization PSO), and Cat Swarm (cat swarm optimization method), Figure 3In the left figure, the horizontal axis represents the number of sensors and the vertical axis represents the coverage rate. It can be seen that as the number of sensors increases from 10 to 50, the coverage rate of the method proposed in the present invention increases from 83.8% to 97.8%, which can cover the target area more efficiently. The increase in the number of traffic lanes has a certain impact on the coverage rate. Figure 3 In the figure on the right, the horizontal axis represents the number of lanes and the vertical axis represents the coverage rate. It can be seen that the coverage rate of the method proposed in the present invention drops from 98.2% for two lanes to 90.3% for six lanes, but this drop is more moderate than that of the traditional method, which shows that the DT method can maintain high adaptability and coverage efficiency under complex road conditions.

[0127] The present invention further analyzes the perception accuracy under different conditions. Figure 4 , Figure 4 The horizontal axis represents the number of deployed sensors, and the vertical axis represents the perception accuracy. Compared with traditional methods such as Q-Learning, MEP-PSO, and CatSwarm, the perception accuracy shows an obvious upward trend as the number of deployed sensors increases, and the perception accuracy increases from the initial 85.8% to 97.4%. The perception accuracy will accelerate with the increase in the number of deployed sensors, and then tend to stabilize, indicating that a saturation effect will occur when the number of deployed sensors is high. Compared with traditional methods, the method proposed in this invention performs well on all deployed sensors. The performance enhancement is mainly attributed to the DT method adopted, including the integration of the Transformer architecture and RL, which enables the Agent to be closely integrated with the vehicle's sensing requirements and is superior to traditional RL methods and static optimization methods.

[0128] In summary, the three-dimensional roadside sensor deployment method for collaborative perception of the Internet of Vehicles proposed in the embodiment of the present invention constructs a road sensor network RSNs model that can truly reflect the complex three-dimensional road environment, and designs a sensor deployment method based on the decision converter based on the RSNs model, using the Transformer model and multi-head attention mechanism in the decision converter to dynamically adapt to various road conditions and environmental changes, thereby well solving the problem of the difficulty of deploying sensors in a complex and constantly changing three-dimensional road environment; at the same time, in order to improve the performance of the decision converter, the embodiment of the present invention also introduces a two-layer optimization technology, which optimizes the cumulative reward in the decision sequence of the decision converter through the two-layer optimization technology, and can adjust the deployment plan in real time to adapt to complex road and environmental changes. In general, the embodiment of the present invention improves the coverage and perception accuracy of the three-dimensional road environment, while reducing the deployment cost, enhancing the economy and practical application effect, thereby significantly improving the safety and efficiency of autonomous driving and intelligent networked vehicles in complex environments.

[0129] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0130] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the specification and its drawings. In the specification, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude multiple situations. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0131] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A three-dimensional roadside sensor deployment method for collaborative perception of vehicle networks, characterized in that: The method comprises: Modeling a RSNs model of a three-dimensional road environment including a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, and a set of restricted areas; Based on the RSNs model of the three-dimensional road environment, construct a perception detection function of a single sensor in each deployable area, and when there are multiple sensors covering each deployable area, construct a global sensor detection function of each deployable area according to the perception detection function of the single sensor; The proportion of deployable areas that maximizes or exceeds the minimum detection probability according to the global sensor detection function is used as the first objective optimization problem function, the total number of sensors in each deployable area is minimized as the second objective optimization problem function, and a deployment optimization reward function composed of the first objective optimization problem function and the second objective optimization problem function is constructed through a balancing coefficient; A decision sequence consisting of a state space, an action space, and a cumulative reward value is constructed, and a deployment strategy is obtained by solving the decision sequence based on a decision converter; wherein the state space includes a set of deployable sensors, a set of sensor deployable locations, a set of deployable areas, a set of restricted areas, and a set of minimum detection probabilities; the action space includes deploying new sensors, reconfiguring existing sensors, and removing existing sensors; and the cumulative reward value is obtained by adaptively redistributing the deployment optimization reward function using a two-layer optimization method.

2. The three-dimensional roadside sensor deployment method for collaborative perception of vehicle networks according to claim 1 is characterized in that: Construct the perception detection function of a single sensor, including: Four sub-perception detection functions are constructed respectively, in which the detection probability of a single sensor is affected by distance, horizontal detection angle of view, vertical detection angle of view and obstacles; The perception detection function of a single sensor is constructed by the four sub-perception detection functions.

3. The three-dimensional roadside sensor deployment method for collaborative perception of vehicle networks according to claim 2 is characterized in that: The sub-perception detection function affected by distance is expressed as: in, represents the sub-perception detection function affected by distance, represents the reference position information of sensor si,j, represents the actual position information of sensor si,j, α represents The slope adjustment parameter, β represents the maximum effective distance of the sensor; The sub-perception detection function affected by the horizontal detection viewing angle is expressed as: in, represents the sub-perceptual detection function affected by the horizontal detection viewing angle, represents the reference horizontal angle of sensor si,j, represents the actual horizontal angle of sensor si,j, ι represents The slope adjustment parameter, η represents the maximum detection angle of the sensor; The sub-perception detection function affected by the vertical detection viewing angle is expressed as: in, represents the sub-perceptual detection function affected by the vertical detection viewing angle, represents the reference vertical angle of sensor si,j, represents the actual vertical angle of sensor si,j, and l represents The slope adjustment parameter is express Value at 50% of maximum coverage; The sub-perception detection function affected by obstacles is expressed as follows: in, represents the sub-perception detection function affected by obstacles, and G represents the obstacle adjustment parameter.

4. The three-dimensional roadside sensor deployment method for collaborative perception of Internet of Vehicles according to claim 3 is characterized in that: The perception detection function of a single sensor is constructed, and the formula is expressed as: in, Represents the perception detection function of a single sensor, that is Indicates the deployable area in the deployable area set AT The perception detection function of sensor si,j, NT represents the number of deployable regions. represents the i-th deployable area; When multiple sensors are covered, a global sensor detection function is constructed for each deployable area, and the formula is expressed as: in, Represents the global sensor detection function, that is, the set of deployment areas The perception detection function of the deployable sensor set si, si,j represents the jth deployable sensor in si, Indicates the deployable area The maximum number of deployable sensors in .

5. The three-dimensional roadside sensor deployment method for collaborative perception of Internet of Vehicles according to claim 1 is characterized in that: The first objective optimization problem function is expressed as: Where C1 represents the first objective optimization problem function, card{} represents the number of elements in the set, AT represents the set of deployable areas, NT represents the number of deployable regions. represents the i-th deployable area, P represents the minimum detection probability set, Indicates the deployable area The minimum detection probability, si represents the deployable area A collection of deployable sensors, Indicates the deployable area The global sensor detection function; The second objective optimization problem function is expressed as: in, Indicates the deployable area The number of deployable sensors in Indicates the deployable area The set of locations where sensors can be deployed, Indicates the deployable area The position information of the jth sensor in .

6. The three-dimensional roadside sensor deployment method for collaborative perception of Internet of Vehicles according to claim 1 is characterized in that: Adaptive reward redistribution of the deployment optimization reward function is achieved through a two-layer optimization method, including: An initial cumulative reward value of the sensor deployment trajectory is calculated based on the deployment optimization reward function, and a proxy reward function is introduced to re-evaluate the reward of each time step in the sensor deployment trajectory to obtain a reshaping reward value, and a reshaping reward function is constructed using the initial cumulative reward value and the reshaping reward value; The reshape reward function is used as the third objective optimization problem function, the verification loss function is used as the fourth objective optimization problem function, and a two-layer optimization problem function consisting of the third objective optimization problem function and the fourth objective optimization problem function is constructed by balancing the weight coefficients; The double-layer optimization problem function is solved by a double-layer optimization method to obtain the reallocated cumulative reward value; wherein, the third objective optimization problem function is optimized in the upper-layer optimization process, and the fourth objective optimization problem function is optimized in the lower-layer optimization process, completing the upper-layer and lower-layer optimizations once to achieve a single-step optimization.

7. The method for deploying three-dimensional roadside sensors for collaborative perception of Internet of Vehicles according to claim 6, characterized in that: The third objective optimization problem function is expressed as follows: Among them, C3 represents the third objective optimization problem function, represents the validation set, δ represents the sensor deployment trajectory, |δ represents the length of the sensor deployment trajectory, Rep(δ) represents the cumulative reward value of the sensor deployment trajectory, represents the reshape reward value at time step t; The fourth objective optimization problem function is expressed as follows: Among them, C4 represents the fourth objective optimization problem function, represents the verification loss function, ξ*(ζ) represents the strategy model to be solved, and ζ represents the parameters of the reward model.

8. The method for deploying three-dimensional roadside sensors for collaborative perception of Internet of Vehicles according to claim 6, characterized in that: In the optimization process of the upper layer and the lower layer, the alternating optimization method is used to optimize the double-layer optimization problem function; among them, During the optimization process, the strategy model of the current single-step optimization is solved by training the loss function as an approximate solution to the strategy model to be solved for the verification loss function, and the approximate solution is used as the strategy model to be solved for the verification loss function in the next single-step optimization to update the verification loss function, and return to use the verification loss function as the fourth objective optimization problem function until the stopping condition is met.

9. The method for deploying three-dimensional roadside sensors for collaborative perception of Internet of Vehicles according to claim 8, characterized in that: The current single-step optimization strategy model is solved by training the loss function, which is used as an approximate solution of the strategy model to be solved to verify the loss function. The formula is expressed as: Among them, ξ′(ζ) represents the policy model of the current single-step optimization, ξ(ζ) represents the current policy model, αξ represents the learning rate of the current single-step optimization, represents the gradient operation, Represents the training loss function.

Images