AI flexible control method for Internet of Vehicles based on auxiliary service

Abstract

The invention discloses an Internet of Vehicles AI flexible control method based on auxiliary service, and the method comprises the following steps: S1, collecting original environment data through multi-mode sensors disposed at a vehicle, a roadside unit and a cloud end, the original environment data comprises vehicle operation original data, road environment original data, user behavior original data, visible light images, infrared thermal imaging data and millimeter wave radar point cloud data; s2, constructing a cross-modal feature alignment network, performing space-time alignment and feature fusion on the visible light image, the infrared thermal imaging data and the millimeter wave radar point cloud data, and generating an environment fusion feature map; and S3, generating a flexible control strategy model based on a deep reinforcement learning model, wherein the deep reinforcement learning model comprises a fuzzy membership calculation module, a strategy optimization module and a multi-target reward function design. According to the method, multi-modal fusion perception and dynamic strategy adaptation scenes are realized, and the efficiency is improved by resource collaboration.

Description

Technical Field

[0001] This invention relates to the field of vehicle control methods, and more specifically to a vehicle-to-everything (V2X) AI flexible control method based on auxiliary services. Background Technology

[0002] With the rapid development of vehicle-to-everything (V2X) technology, the collaborative interaction between vehicles, between vehicles and roadside units, and between vehicles and the cloud is becoming increasingly frequent. Various auxiliary services, such as adaptive cruise control, emergency braking assist, and real-time traffic navigation, are placing higher demands on the comprehensiveness of environmental perception, the adaptability of control strategies, and the rationality of resource allocation.

[0003] However, existing technologies have many limitations: on the one hand, traditional solutions often rely on single-type data or single control logic, making it difficult to cope with the fusion processing needs of multimodal data in the Internet of Vehicles (IoV), resulting in insufficient perception accuracy in complex traffic environments. Especially in adverse scenarios such as rain, fog, and low light, service reliability is easily affected by missing data or misjudgment. On the other hand, traditional control strategies often adopt fixed parameters or single-objective optimization, making it difficult to balance multi-dimensional needs such as safety, energy consumption, and service latency. Moreover, they are poorly adaptable to user driving preferences and dynamic scene changes, resulting in low matching degree between control parameters and actual scene requirements. In addition, the resource competition problem of multiple service requests is prominent in the IoV. Existing resource allocation methods lack dynamic consideration of inter-service dependencies and real-time needs, which can easily lead to communication delays, waste of computing resources, and other problems, restricting the collaborative efficiency of auxiliary services. Therefore, an AI flexible control method for IoV based on auxiliary services is proposed. Summary of the Invention

[0004] The present invention solves the problems of the prior art through the following technical solution, and the present invention includes the following steps: S1: Collect raw environmental data through multimodal sensors deployed in vehicles, roadside units, and the cloud. The raw environmental data includes raw vehicle operation data, raw road environment data, raw user behavior data, visible light images, infrared thermal imaging data, and millimeter-wave radar point cloud data. S2: Construct a cross-modal feature alignment network to perform spatiotemporal alignment and feature fusion on visible light images, infrared thermal imaging data and millimeter-wave radar point cloud data to generate an environment fusion feature map; S3: Generate a flexible control strategy model based on a deep reinforcement learning model, wherein the deep reinforcement learning model includes a fuzzy membership calculation module, a policy optimization module, and a multi-objective reward function; Among them, the fuzzy membership calculation module is used to quantify the fuzzy attributes of the input features; The strategy optimization module is used to perform iterative optimization of the strategy based on the fuzzy membership calculation results and multi-objective reward function, using the gradient descent method to minimize the control error and maximize the comprehensive reward, and finally output dynamic control parameters that meet the needs of the scenario. The inputs to the flexible control strategy model include: The compressed vector Xf extracted from the environmental fusion feature map; The vehicle state vector Xs is obtained by preprocessing the raw vehicle operation data. The user preference vector Xp is extracted from raw user behavior data; The output is dynamic control parameters for auxiliary services; S4: Deploy a conflict resolution module in the service collaboration management platform to dynamically allocate computing and communication resources based on a flexible control strategy model.

[0005] Furthermore, the original vehicle operation data in step S1 includes real-time sensor readings of vehicle speed, acceleration, steering angle, braking status and fault codes. The data sampling frequency is 10Hz. The braking status is represented by binary 0 to indicate no braking and 1 to indicate braking. The fault codes are represented by digital codes to indicate the fault type and level. The vehicle state vector Xs in step S3 is generated through the following steps: Preprocess the raw vehicle operation data: The original data was smoothed using a sliding window filtering method, with the window size set to 5 sampling periods to remove high-frequency noise; Outliers are identified using the 3σ criterion, which means that when a data value exceeds the mean of the data series by ±3 times the standard deviation, it is determined to be an outlier and removed. The min-ma normalization formula Xnorm=(X−Xmin) / (Xmax−Xmin) is used to map all data to the interval [0,1] to unify the data scale; Xnorm is the normalized vehicle operation data value, mapped to the [0,1] interval, used to unify the data scale; X represents the original vehicle operation data values ​​before preprocessing (such as real-time sensor readings of vehicle speed, acceleration, steering angle, etc.). Xmin is the minimum value in the original data sequence of this type (such as the minimum vehicle speed among all sampling points). Xmax is the maximum value in the original data sequence of this type (such as the maximum vehicle speed among all sampling points). Features are extracted from the preprocessed data, including time-domain features and frequency-domain features; Time-domain features are calculated based on the time series data of the original data: The dynamic rate of change of vehicle speed is calculated by the ratio of the difference in vehicle speed between two adjacent sampling periods to the sampling time interval, i.e., 0.1s, i.e., (v(t)−v(t−1)) / 0.1; v(t) is the vehicle speed value in the current sampling period (unit: km / h or m / s, consistent with the unit of the original data). v(t−1) is the vehicle speed value in the previous sampling period; The peak acceleration is the maximum acceleration value extracted within a sliding window of 10 sampling periods, i.e., 1 second. The steering angle fluctuation frequency is the number of times the steering angle change direction switches within 10 sampling periods, that is, the number of times it switches between positive and negative directions; The duration of the braking state is the product of the number of consecutive samples and the sampling period (0.1s) when the braking state is 1. Fault code frequency: Count the number of times a fault code has a non-zero value within 600 sampling periods, i.e., within 60 seconds; Frequency domain features are obtained by performing a Fourier transform on the preprocessed raw data: The dominant frequency of engine vibration is determined by performing a Fourier transform on the acceleration time series and selecting the frequency value with the highest power spectral density. The chassis resonance frequency is obtained by performing a Fourier transform on the steering angle time series and taking the frequency value corresponding to the second largest power spectral density. The real-time operating status of vehicles is classified according to preset rules, and the vehicle status classification results are obtained: The preset rules include: a vehicle speed ≥120km / h is considered high risk; a braking duration of ≥5s within 1 minute is considered frequent braking; and a fault code of Class A (codes 100-199) is considered a dangerous condition. A state classifier based on C4.5 decision tree is used. The preprocessed raw data is taken as input and combined with the above-mentioned preset rules to output the vehicle state classification results, including safe state, risk state, and dangerous state. Feature selection is used to filter features: the vehicle state classification results are used as the decision attribute set D, and features with γi>τ (τ is the correlation threshold, with a value of 0.7) are selected to form the dimension-compressed vehicle state vector Xs. Grey relational analysis was used to classify vehicle status as the decision attribute set D. Calculate the correlation γi between the extracted time-domain features and frequency-domain features and D; Features with γi>τ (τ is the correlation threshold, with a value of 0.7) are selected to form the dimensionally compressed vehicle state vector Xs.

[0006] Furthermore, the feature selection algorithm uses grey relational analysis to obtain Xs, and the specific process is as follows: The vehicle status classification results are used as the decision attribute set D; Calculate the correlation between time-domain or frequency-domain features and D. : ; Where n is the number of samples; Let be the decision attribute value of the k-th sample; Let i be the value of the i-th feature in the k-th sample; It is the minimum difference between two levels; The maximum difference between the two levels; ρ is the resolution coefficient; τ is the correlation threshold; choose The features constitute Xs.

[0007] Furthermore, step S2 also includes: S21: A two-stream convolutional neural network (CNN) encoder is used to process the two types of image data separately: Visible light image data I v Spatial features are extracted using a two-stream convolutional neural network model Φ to obtain visible light features F. v Specifically, ; Infrared thermal imaging data I t Infrared features were obtained by extracting thermal distribution features using the same two-stream convolutional neural network model Φ. Specifically: ; in, This is a convolutional neural network model used to extract features from visible light images and infrared thermal imaging. S22: P for millimeter-wave radar point cloud data r Voxelization is performed to generate millimeter-wave radar point cloud features F. r Specifically, this includes: dividing Pr into a uniform three-dimensional grid, i.e., a voxel grid; calculating the point cloud density and spatial distribution entropy of each voxel grid, and integrating them into the millimeter-wave radar point cloud feature Fr; S23: Through the cross-modal attention module , F r Perform spatiotemporal alignment and output alignment feature F. align The calculation formula is: ; σ is the activation function; Q is the query vector; K is the key vector; V is the value vector; d k K represents the dimension of the key vector K; T This is the transpose of the key vector; S24: Align feature F align Input a feature pyramid network, and generate the final environment fusion feature map Ff through multi-scale feature fusion. usion .

[0008] Furthermore, the specific process of S22 is as follows: S221: Divide the data into a three-dimensional voxel lattice, and then divide the millimeter-wave radar point cloud data P. r The data is divided into a uniformly distributed three-dimensional grid, with each grid defined as a voxel grid, serving as the basic spatial unit for point cloud feature extraction. S222: Calculate the point cloud density feature ρ within each voxel lattice. For each voxel lattice, calculate the density feature of the point cloud within it using the following formula: ; Among them, among them, This represents the number of radar points contained within the voxel lattice. The spatial volume of the voxel lattice (determined by the 3D mesh size); S223: Extract the spatial distribution entropy Hv of the point cloud within the voxel grid. For each voxel grid, calculate its distribution entropy Hv based on the spatial distribution of the points, using the following formula: ; Where M is the total number of points in the voxel lattice, p i Let be the probability distribution of the i-th point within the solid lattice (i.e., the proportion of the i-th point to the total number of points). S224: Generating millimeter-wave radar point cloud features F r The density feature ρ and spatial distribution entropy of each voxel lattice are combined. The data is integrated to form millimeter-wave radar point cloud data P. r The corresponding millimeter-wave radar point cloud feature F r .

[0009] Furthermore, in step S3, the fuzzy membership calculation module is used to perform the following operations: S31: Calculate fuzzy membership values ​​using an adaptive Gaussian membership function. ; in, The input feature vector; σ is the center parameter corresponding to the i-th feature of the j-th fuzzy set; ij Let be the standard deviation parameter of the i-th feature corresponding to the j-th fuzzy set; S32: Dynamically update the membership function parameters, the specific process is as follows: , ; is the update amount of the standard deviation parameter ; η is the learning rate; J is the loss function of reinforcement learning; and is the partial derivative of the loss function with respect to the parameter , σ ij .

[0010] Furthermore, three-dimensional space coordinates are defined for each service request Ri , where maps the timeliness level: ; where β is the proportionality coefficient; T max is the maximum tolerable delay of the service request Ri; For any two service requests Ri and Rj, calculate the three-dimensional space conflict distance : ; where , and are the three-dimensional space coordinates of the service request R i , and are the physical space coordinates, is the timeliness component; , and are the three-dimensional space coordinates of the service request R j ; Compare the calculated conflict distance with the conflict radius ri of the service request R i (reflecting the influence range of the resources required by R i ); If < ri, it is determined that there is a resource conflict between R i and R j (the two compete for the same resource in terms of space or timeliness); If ≥ ri, it is determined that there is no resource conflict between the two.

[0011] Furthermore, it also includes: S5: Construct a service dependency graph G = (V, E), where the node V represents a service instance and the edge E represents the data flow between services; S6: Calculate the collaborative cost function C and optimize it dynamically: ; is the communication delay weight coefficient of the edge e; The communication delay corresponding to edge e; Calculate the load weight coefficient for node v; This represents the computational load corresponding to node v.

[0012] Furthermore, the process of obtaining the multi-objective reward function in step S3 is as follows: ; , and The weights of each objective are (summing up to 1); r safety For safety rewards (such as negative feedback for collision risk); r energy For energy consumption rewards (such as negative feedback on vehicle energy consumption); r latency For delayed rewards (such as negative feedback for service response delays); Among them, the weighting coefficient Dynamically adjust according to service scenarios The dynamic weight coefficient for the i-th objective is defined as follows: ; qi is the service quality evaluation value of the i-th target; τ is the temperature parameter; exp(∙) is the exponential function.

[0013] Furthermore, the specific strategy optimization process of the strategy optimization module in step S3 includes: SS1: Initialize policy parameters θ; SS2: Fuzzy membership degree μ based on the output of the fuzzy membership degree calculation module ij The initial control parameters π(θ) are generated by combining the input vectors (Xf, Xs, Xp) of the flexible control strategy model. SS3: Calculate the control error E and the overall reward r: The control error E = |π(θ) - π*|, where π* is the ideal control parameter; The overall reward r is calculated using a multi-objective reward function; SS4: Construct the policy loss function L=α・E-(1-α)・r, where α is the balance coefficient; SS5: Update policy parameters using gradient descent. θnew=θ-η・∂L / ∂θ, where η is the learning rate and ∂L / ∂θ is the gradient of the loss function with respect to the parameter θ; SS6: Repeat steps SS1-SS5 until |θnew-θ|<ε, where ε is the convergence threshold, and output the final dynamic control parameter π(θnew).

[0014] Compared with existing technologies, this invention has the following advantages: This AI-based flexible control method for connected vehicles based on assisted services collects vehicle operation data, road environment data, user behavior data, and various data types such as visible light images, infrared thermal imaging, and millimeter-wave radar point clouds through multimodal sensors deployed in vehicles, roadside units, and the cloud. It then uses a cross-modal feature alignment network for spatiotemporal alignment and fusion, overcoming the limitations of single data types. By combining voxelization processing to extract the density features and spatial distribution entropy of radar point clouds, and cross-modal attention modules to align multi-source features, it can more comprehensively and accurately characterize complex traffic environments and improve the robustness of environmental perception under adverse weather or lighting conditions.

[0015] A flexible control strategy based on a deep reinforcement learning model quantifies the fuzzy attributes of input features through a fuzzy membership calculation module, combines a multi-objective reward function, and achieves dynamic adjustment of the control strategy through gradient descent iterative optimization in the policy optimization module. This approach minimizes control error while maximizing overall reward, making the output dynamic control parameters more closely match real-time scenario requirements. It solves the problems of poor adaptability and difficulty in balancing multiple objectives inherent in traditional control strategies.

[0016] Through the conflict resolution module of the service collaboration management platform, service requests are defined based on three-dimensional spatial coordinates and conflict distances are calculated. Resource conflicts are accurately determined and computing and communication resources are dynamically allocated. At the same time, a service dependency graph is constructed and the collaboration cost function is optimized, which effectively reduces resource competition between services, improves resource utilization efficiency, reduces service response latency, and enhances the stability and efficiency of multi-service collaboration in the Internet of Vehicles.

[0017] In the process of generating vehicle state vectors, grey relational analysis is used for feature selection. Dimensional compression is achieved by calculating the correlation between features and decision attributes, which reduces redundant data while retaining key information and improves data processing efficiency. Cross-modal feature fusion and multi-source data preprocessing further ensure the effectiveness of input features, providing a high-quality data foundation for precise control.

[0018] The multi-objective reward function, through dynamic weight adjustment, can flexibly balance the priorities of safety, energy consumption, and latency according to different scenarios. For example, in emergency scenarios, safety is prioritized, while in regular commutes, energy consumption is reduced, achieving on-demand optimization and significantly improving the overall quality of vehicle-to-everything (V2X) assistance services and user experience. Attached Figure Description

[0019] Figure 1 This is the overall flowchart of the present invention. Detailed Implementation

[0020] The embodiments of the present invention will be described in detail below. These embodiments are based on the technical solutions of the present invention and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0021] like Figure 1 As shown, this embodiment provides a technical solution: a vehicle-to-everything (V2X) AI flexible control method based on assisted services, comprising the following steps: S1: Collect raw environmental data through multimodal sensors deployed in vehicles, roadside units, and the cloud. The raw environmental data includes raw vehicle operation data, raw road environment data, raw user behavior data, visible light images, infrared thermal imaging data, and millimeter-wave radar point cloud data. S2: Construct a cross-modal feature alignment network to perform spatiotemporal alignment and feature fusion on visible light images, infrared thermal imaging data and millimeter-wave radar point cloud data to generate an environment fusion feature map; S3: Generate a flexible control strategy model based on a deep reinforcement learning model, wherein the deep reinforcement learning model includes a fuzzy membership calculation module, a policy optimization module, and a multi-objective reward function; Among them, the fuzzy membership calculation module is used to quantify the fuzzy attributes of the input features; The strategy optimization module is used to perform iterative optimization of the strategy based on the fuzzy membership calculation results and multi-objective reward function, using the gradient descent method to minimize the control error and maximize the comprehensive reward, and finally output dynamic control parameters that meet the needs of the scenario. The inputs to the flexible control strategy model include: The compressed vector Xf extracted from the environmental fusion feature map; The vehicle state vector Xs is obtained by preprocessing the raw vehicle operation data. The user preference vector Xp is extracted from raw user behavior data; The output is dynamic control parameters for auxiliary services; S4: Deploy a conflict resolution module in the service collaboration management platform to dynamically allocate computing and communication resources based on a flexible control strategy model.

[0022] Furthermore, the original vehicle operation data in step S1 includes real-time sensor readings of vehicle speed, acceleration, steering angle, braking status and fault codes. The data sampling frequency is 10Hz. The braking status is represented by binary 0 to indicate no braking and 1 to indicate braking. The fault codes are represented by digital codes to indicate the fault type and level. The vehicle state vector Xs in step S3 is generated through the following steps: Preprocess the raw vehicle operation data: The original data was smoothed using a sliding window filtering method, with the window size set to 5 sampling periods to remove high-frequency noise; Outliers are identified using the 3σ criterion, which means that when a data value exceeds the mean of the data series by ±3 times the standard deviation, it is determined to be an outlier and removed. The min-max normalization formula Xnorm=(X−Xmin) / (Xmax−Xmin) is used to map all data to the interval [0,1] to unify the data scale; Xnorm is the normalized vehicle operation data value, mapped to the [0,1] interval, used to unify the data scale; X represents the original vehicle operation data values ​​before preprocessing (such as real-time sensor readings of vehicle speed, acceleration, steering angle, etc.). Xmin is the minimum value in the original data sequence of this type (such as the minimum vehicle speed among all sampling points). Xmax is the maximum value in the original data sequence of this type (such as the maximum vehicle speed among all sampling points). Features are extracted from the preprocessed data, including time-domain features and frequency-domain features; Time-domain features are calculated based on the time series data of the original data: The dynamic rate of change of vehicle speed is calculated by the ratio of the difference in vehicle speed between two adjacent sampling periods to the sampling time interval, i.e., 0.1s, i.e., (v(t)−v(t−1)) / 0.1; v(t) is the vehicle speed value in the current sampling period (unit: km / h or m / s, consistent with the unit of the original data). v(t−1) is the vehicle speed value in the previous sampling period; The peak acceleration is the maximum acceleration value extracted within a sliding window of 10 sampling periods, i.e., 1 second. The steering angle fluctuation frequency is the number of times the steering angle change direction switches within 10 sampling periods, that is, the number of times it switches between positive and negative directions; The duration of the braking state is the product of the number of consecutive samples and the sampling period (0.1s) when the braking state is 1. Fault code frequency: Count the number of times a fault code has a non-zero value within 600 sampling periods, i.e., within 60 seconds; Frequency domain features are obtained by performing a Fourier transform on the preprocessed raw data: The dominant frequency of engine vibration is determined by performing a Fourier transform on the acceleration time series and selecting the frequency value with the highest power spectral density. The chassis resonance frequency is obtained by performing a Fourier transform on the steering angle time series and taking the frequency value corresponding to the second largest power spectral density. The real-time operating status of vehicles is classified according to preset rules, and the vehicle status classification results are obtained: The preset rules include: a vehicle speed ≥120km / h is considered high risk; a braking duration of ≥5s within 1 minute is considered frequent braking; and a fault code of Class A (such as code 100-199) is considered a dangerous condition. A state classifier based on C4.5 decision tree is used. The preprocessed raw data is taken as input and combined with the above-mentioned preset rules to output the vehicle state classification results, including safe state, risk state, and dangerous state. Feature selection is used to filter features: the vehicle state classification results are used as the decision attribute set D, and features with γi>τ (τ is the correlation threshold, with a value of 0.7) are selected to form the dimension-compressed vehicle state vector Xs. Grey relational analysis was used to classify vehicle status as the decision attribute set D. Calculate the correlation γi between the extracted time-domain features and frequency-domain features and D; Features with γi>τ (τ is the correlation threshold, with a value of 0.7) are selected to form the dimension-compressed vehicle state vector Xs; The original vehicle operation data in step S1 includes real-time sensor readings such as vehicle speed, acceleration, steering angle, braking status, and fault codes. These data directly reflect the dynamic driving status and health status of the vehicle, providing comprehensive and crucial original inputs for the subsequent generation of the vehicle state vector Xs. This avoids deviations in state characterization due to missing data and ensures the reliability of the data foundation.

[0023] Cleaning and normalization can remove noise and outliers from the original data and unify the data scale, thus avoiding the impact of data interference on subsequent models. Extracting time-domain and frequency-domain features can capture the deep-seated patterns of vehicle operation from two dimensions: dynamic change trends and signal frequency characteristics, thus enriching the feature dimensions. The feature selection algorithm eliminates redundant features through dimensionality compression, reducing data dimensionality while retaining key information, thus reducing the computational burden on subsequent models, improving processing efficiency, and avoiding the risk of model overfitting caused by the curse of dimensionality, ensuring that Xs can accurately reflect the core state of the vehicle.

[0024] The feature selection algorithm uses grey relational analysis to obtain Xs. The specific process is as follows: The vehicle status classification results are used as the decision attribute set D; Calculate the correlation between time-domain or frequency-domain features and D. : ; Where n is the number of samples; Let be the decision attribute value of the k-th sample; Let i be the value of the i-th feature in the k-th sample; For two-level minimum difference, among all features and all samples | The minimum value of |; For the maximum difference between two levels, across all features and all samples | The maximum value of |; ρ is the resolution coefficient, usually set to 0.5, used to adjust the sensitivity of the correlation; τ is the correlation threshold, features with γi>τ are selected; choose The features constitute Xs; By calculating the correlation between time-domain and frequency-domain features and vehicle state classification results (decision attribute set D), only features with a correlation γi higher than the threshold τ are retained. This ensures that Xs focuses on core features that strongly explain vehicle states (safety, risk, danger), while weakly correlated or irrelevant features are eliminated to avoid redundant information interfering with model decisions. Dimensional compression reduces the number of features in the vehicle state vector, lowers the computational complexity of subsequent deep reinforcement learning models, and improves data processing efficiency. At the same time, it avoids the risk of model overfitting caused by the "curse of dimensionality." The selected features have a stronger correlation with the vehicle state classification results, enabling Xs to more accurately reflect the core operating state of the vehicle, providing high-quality input for the flexible control strategy model, and improving the reliability of control parameters.

[0025] Assuming the vehicle state classification results (decision attribute set D) are safe state, risk state, and dangerous state, the features to be screened include: Feature A: Braking frequency (time-domain characteristic, reflecting the frequency of braking); Feature B: Steering angle fluctuation frequency (time domain characteristic, reflecting steering stability); Feature C: Chassis resonance frequency (frequency domain characteristics, reflecting chassis stability); The correlation degree between each feature and D is calculated using grey relational analysis: γA (correlation degree between braking frequency and D) = 0.82; γB (correlation degree between steering angle fluctuation frequency and D) = 0.75; γC (correlation degree between chassis resonance frequency and D) = 0.68; The correlation threshold τ = 0.7 is set. Since γA > 0.7, γB > 0.7, and γC < 0.7, the final vehicle state vector Xs is composed of feature A and feature B, and feature C is removed.

[0026] This process ensures that Xs contains only features strongly correlated with vehicle state classification, which simplifies the vector dimension and guarantees an accurate characterization of the vehicle state, providing a reliable foundation for subsequent control strategy optimization.

[0027] Step S2 further includes: S21: A two-stream convolutional neural network (CNN) encoder is used to process the two types of image data separately: Visible light image data I v Spatial features are extracted using a two-stream convolutional neural network model Φ to obtain visible light features F. v Specifically, ; Infrared thermal imaging data I t Infrared features were obtained by extracting thermal distribution features using the same two-stream convolutional neural network model Φ. Specifically: ; in, This is a convolutional neural network model used to extract features from visible light images and infrared thermal imaging. S22: P for millimeter-wave radar point cloud data r Voxelization is performed to generate millimeter-wave radar point cloud features F. r Specifically, this includes: P r The data is divided into a uniform three-dimensional grid, i.e., a voxel grid; the point cloud density and spatial distribution entropy of each voxel grid are calculated and integrated into the millimeter-wave radar point cloud feature F. r ; S23: Through the cross-modal attention module , F r Perform spatiotemporal alignment and output alignment feature F. align The calculation formula is: ; σ is the activation function, such as softmax; Q is the query vector, where Q=F. v This refers to visible light image features; K is the key vector, where K = ⊕F r ⊕ represents the feature concatenation operation; V is the value vector, where V = ⊕F r Consistent with the key vector; K represents the dimension of the key vector K; T This is the transpose of the key vector; S24: Align feature F align Input a feature pyramid network, and generate the final environment fusion feature map F through multi-scale feature fusion. fusion ; Visible light imaging provides rich spatial details, infrared thermal imaging captures heat sources in low light / adverse weather conditions, and millimeter-wave radar offers strong anti-jamming capabilities and provides range and velocity information. By fusing these three types of features, the limitations of single-modal imaging are overcome, enabling full-scene environmental perception.

[0028] By using a cross-modal attention module, with visible light features as queries and infrared and radar features as keys, relevant features are dynamically focused to solve the feature misalignment problem caused by differences in acquisition time / space between different modal data, thus ensuring the consistency of fused features.

[0029] The feature pyramid network generates multi-scale fusion features, which can simultaneously capture target features of different sizes / distances, improving the granularity of characterization of complex traffic environments. For example, in a rainy scene, target A is a car 50 meters ahead, and target B is a pedestrian on the roadside; In rainy driving scenarios, single-modal data has limitations: visible light images are affected by rain and fog, blurring the outline of target A; infrared thermal imaging can clearly show the heat sources of A and B, but lacks spatial texture; millimeter-wave radar point clouds can provide the distance and speed of A and B, but lack category information.

[0030] The cross-modal fusion process is as follows: Visible light image (I v Spatial features F are extracted using a CNN model Φ. v Includes blurred vehicle outlines and road edges (details lost due to rain and fog).

[0031] Infrared thermal imaging data (I t Heat distribution features F are extracted using the CNN model Φ. t Clearly displays the thermal outlines of target A (high temperature in the car engine area) and target B (human body heat source), unaffected by rain or fog.

[0032] Based on voxelization processing, millimeter-wave radar point cloud P r Divided into a 3D mesh, extracted to: The voxel lattice corresponding to target A has a high density ρ = 60 points / m³ and a low distribution entropy H. v ≈3.4, reflecting a compact rigid body target; Voxel lattice corresponding to target B: medium density, ρ = 10 points / m³; medium distribution entropy, H v ≈4.2, reflecting the decentralized and flexible objectives; These features constitute the point cloud characteristics of millimeter-wave radar. r Define the physical properties of the target, namely its size and distribution.

[0033] S23: Cross-modal attention module alignment features; With visible light characteristics F v For query vector Q, infrared feature F t With radar characteristic F r Concatenate them to form the key vector K and the value vector V; F is calculated using the cross-modal attention module. align F vAutomatically focuses on F t Medium and high temperature areas and F r In medium-density regions, alignment is achieved, such as F v The blurred outline in F t Car heat source, F r High-density point cloud binding.

[0034] S24: Feature pyramid network generates fused features F fusion ; The aligned features are processed by a pyramid network to generate multi-scale fused features: Large-scale features: Capturing the overall position and movement trend of target A (a car in the distance); Small-scale features: Capture the fine movements of target B (nearby pedestrians), such as suddenly crossing; The blurred outline in visible light was clearly identified as a car by infrared heat source and radar density features, solving the problem of blurry identification in rain and fog; F fusion It also includes the target's spatial texture, thermal properties, and physical distribution, enabling subsequent flexible control strategies to accurately determine: target A is a vehicle that needs to be slowed down and avoided, and target B is a pedestrian that needs to be guarded against. Based on F fusion The dynamic parameters output by the flexible control model are more in line with the actual scenario, avoiding sudden braking or missed judgment due to misjudgment of a single mode.

[0035] The specific process of S22 is as follows: S221: Divide the data into a three-dimensional voxel lattice, and then divide the millimeter-wave radar point cloud data P. r The data is divided into a uniformly distributed three-dimensional grid, with each grid defined as a voxel grid, serving as the basic spatial unit for point cloud feature extraction. S222: Calculate the point cloud density feature ρ within each voxel lattice. For each voxel lattice, calculate the density feature of the point cloud within it using the following formula: ; Among them, among them, This represents the number of radar points contained within the voxel lattice. The spatial volume of the voxel lattice (determined by the 3D mesh size); S223: Extract the spatial distribution entropy H of point cloud within the voxel lattice. v For each voxel lattice, its distribution entropy H is calculated based on the spatial distribution of points. v The formula is: ; Where M is the total number of points in the voxel lattice, p i Let be the probability distribution of the i-th point within the solid lattice (i.e., the proportion of the i-th point to the total number of points). S224: Generating millimeter-wave radar point cloud features F r The density feature ρ and spatial distribution entropy of each voxel lattice are combined. The data is integrated to form millimeter-wave radar point cloud data P. r The corresponding millimeter-wave radar point cloud feature F r ; Structured processing addresses the disorder of point clouds: Millimeter-wave radar point clouds consist of discrete three-dimensional coordinate points, lacking spatial organization, making it difficult to capture the spatial distribution patterns of targets through direct processing. By dividing the data into uniform three-dimensional grids (such as 1m×1m×0.5m voxel grids), the point cloud data is mapped onto the structured grid, transforming the originally disordered point cloud into ordered feature units within the grid. This facilitates subsequent density and entropy calculations and reduces the difficulty of feature matching during multimodal fusion.

[0036] Density features quantify target compactness: Point cloud density ρ, the ratio of the number of points to the volume within a single voxel, directly reflects the compactness of the target. Rigid targets have denser point clouds (higher ρ values), dispersed targets have sparser point clouds (medium ρ values), and noise targets have extremely sparse point clouds (lower ρ values). ρ can be used to quickly distinguish target types, providing a physical property reference for cross-modal fusion.

[0037] Spatial distribution entropy characterizes the dispersion of point clouds: Spatial distribution entropy The degree of dispersion is quantified by the probability distribution of points within the voxel lattice; the more concentrated the distribution of the target point cloud, the better. The lower the value, the more disordered the distribution. The higher the value. In conjunction with ρ, it can further filter out noise points, retain effective target features, and improve the purity of radar point cloud features.

[0038] If a vehicle is driving in the rain, millimeter-wave radar detects point cloud data P of the area ahead. r This includes target A (a car 50 meters away), target B (a pedestrian on the roadside), and noise points caused by rain and fog. The specific process and effects of voxelization are as follows: The radar detection range (e.g., 100 meters in front, 30 meters laterally, and 5 meters high) is divided into multiple three-dimensional voxel grids of 1m × 1m × 0.5m. Each voxel grid corresponds to a unique spatial coordinate (x, y, z), ensuring that point cloud data is assigned to a specific spatial unit. For example, the point cloud of target A is concentrated in voxel grids near (x=50, y=0, z=0.5), the point cloud of target B is distributed in voxel grids near (x=45, y=5, z=1), and noise points are randomly distributed in multiple voxel grids.

[0039] The voxel grid containing target A contains 30 radar points. =30, volume =1×1×0.5=0.5m³, therefore ρ=30 / 0.5=60 points / m³, high density, which is consistent with the compact characteristics of a car as a rigid body target; The voxel grid containing target B contains 5 radar points. =5, =0.5m³, therefore ρ=5 / 0.5=10 points / m³, which is a medium density, consistent with the characteristics of pedestrian dispersion; The voxel grid containing the noise points: only 1-2 points per cell ( =1), ρ=1 / 0.5=2 points / m³, low density, reflecting the sparsity of noise.

[0040] Within a voxel grid of target A, 30 points are uniformly distributed near the center of the grid, and the probability of each point is p. i =1 / 30, therefore H v =-Σ(1 / 30×log2(1 / 30))≈3.4, a low entropy value with a concentrated distribution; Within the voxel grid of target B, the five points are relatively dispersed, p i The differences are relatively large, such as 2 points accounting for 1 / 5 and 3 points accounting for 1 / 5, so Hv≈4.2, a medium entropy value, and the distribution is moderately dispersed; Within a voxel grid of noise points, the probability p of a point is... i =1, the actual noise point appears randomly in multiple voxel cells. If there is 1 point in a single voxel cell, p i =1,H v =0, but combined with ρ=2, low density, it can be determined as noise; if there are 2 points in a voxel lattice and they are distributed very far apart, p i =1 / 2, =-2×(1 / 2×log2(1 / 2))=1, but since ρ=4, it is still low density and is still judged as noise.

[0041] After the above processing, the extracted millimeter-wave radar point cloud features F r (ρ, containing voxel lattice) Spatial coordinates information: For target A, ρ=60. ≈3.4, and is marked as a high-density, low-entropy rigid body target; For target B, ρ=10. ≈4.2, and is marked as a medium-density, medium-entropy dispersed target; Noise point because ρ<5 and >5 was filtered out.

[0042] These features, along with the blurred contours of visible light images and the heat source features of infrared thermal imaging, are aligned through a cross-modal attention module and ultimately fused into F. fusionThis enables the vehicle to clearly identify target A as a car that needs to slow down and target B as a pedestrian that needs to be avoided, providing precise environmental feature inputs for the flexible control strategy.

[0043] Furthermore, in step S3, the fuzzy membership calculation module is used to perform the following operations: S31: Calculate fuzzy membership values ​​using an adaptive Gaussian membership function. ; in, The input feature vector; σ is the center parameter corresponding to the i-th feature of the j-th fuzzy set; ij Let be the standard deviation parameter of the i-th feature corresponding to the j-th fuzzy set; S32: Dynamically update the membership function parameters, the specific process is as follows: , ; Standard deviation parameter The update amount; η is the learning rate (parameter update step size); J is the loss function for reinforcement learning; and For the loss function with respect to parameters σ ij The partial derivatives; Many input features in the Internet of Vehicles (IoV), such as safe distance from obstacles and the aggressiveness of user driving preferences, are fuzzy and lack absolute yes / no boundaries. An adaptive Gaussian membership function can address this. These fuzzy features are transformed into membership values ​​between 0 and 1, enabling the model to handle uncertain information and avoid decision bias caused by feature fuzziness, such as misjudging a closer distance as a safe distance.

[0044] Dynamic parameter updates adapt to real-time changes in the scenario: The central parameter is dynamically adjusted using the partial derivative of the loss function. and standard deviation This enables the membership function to match the distribution changes of input features in real time, such as expanding the fuzzy range of safe distance when the road is slippery in rainy weather and tightening the judgment criteria for aggressive driving in congested scenarios, thereby improving adaptability to dynamic scenarios.

[0045] Providing a reliable foundation for strategy optimization and improving control accuracy: Accurate fuzzy membership degrees serve as the basis for the strategy optimization module. Gradient descent iteration provides high-quality input, making the optimization of control parameters more aligned with the needs of real-world scenarios. For example, in emergency scenarios, adjusting membership parameters to strengthen safety weights can achieve a balance between minimizing control error and maximizing overall reward. For example, target A is a car 50 meters ahead, and target B is a pedestrian on the roadside; The input characteristics of the flexible control strategy model during rainy driving include: Fuzzy Feature 1: Relative distance d from target A, fuzzy set: near, medium, far; Fuzzy Feature 2: User's driving preference level p, fuzzy set: conservative, moderate, aggressive.

[0046] The operation process and effects of the fuzzy membership calculation module are as follows: S31: Calculate fuzzy membership degree (quantify fuzzy attributes): For the relative distance d to target A (currently d=30m), define 3 fuzzy sets: Near j=1: central parameter c11=20m, standard deviation σ11=5m; In the case of j=2: central parameter c12=40m, standard deviation σ12=10m; Far j=3: central parameter c13=70m, standard deviation σ13=15m.

[0047] Calculated using the Gaussian membership function: It belongs to the degree of proximity; It belongs to the medium degree; This falls under the category of "far". For the user's driving preference intensity p, and the current user's recent frequency of rapid acceleration being moderate, three fuzzy sets are defined, and the following calculations are obtained: The formula for calculating the adaptive Gaussian membership function is: ; Substitute c 21 =1, σ 21 =2.5: ; Substitute c 22 =6, σ 22 =1.5: ; Substitute c 23 =9, σ 23 =2.5: ; μ21(p) is 0.7, which is conservative; μ22(p) is 0.2, which is moderate; and μ23(p) is 0.1, which is radical. As the vehicle travels, target A suddenly decelerates, and the relative distance rapidly shortens to 20m. At this point, the model's control error E increases because the initial control parameters failed to decelerate in time, leading to an increase in the loss function J. The membership parameters of d are dynamically updated by calculating the partial derivatives of the loss function with respect to the parameters. Central parameter c 11The nearest center distance has been adjusted from 20m to 15m, because it is necessary to determine the nearest distance earlier in rainy weather; Standard deviation σ 11 The range of blur at close range has been increased from 5m to 8m, improving sensitivity. , η is the learning rate. Here, due to the increase in error, the partial derivative is positive, and the parameter is adjusted in the positive direction. After the update, when d=20m: The value is significantly close to the mean, μ12(20)≈0.12, which is almost not moderate. The change in fuzzy membership degree enables the strategy optimization module to more accurately identify dangerous scenarios, output more conservative control parameters through gradient descent iteration, avoid collision risks, and balance the user's conservative driving preferences to achieve dynamic adaptation of the control strategy.

[0048] By accurately quantifying fuzzy features and dynamically adjusting parameters, the problem of handling uncertain features in the Internet of Vehicles is solved, providing an adaptive fuzzy feature basis for flexible control strategies and significantly improving control reliability in complex scenarios.

[0049] Step S4 further includes: For each service request R i Define three-dimensional spatial coordinates ,in Mapping timeliness level: ; Where β is the proportionality coefficient; T max The maximum tolerable latency for service request Ri; For any two service requests R i and R j Calculate the three-dimensional spatial conflict distance between the two. : ; in, , and For service request R i Three-dimensional spatial coordinates, and For physical space coordinates, For time-sensitive components; , and For service request R j Three-dimensional spatial coordinates; The calculated conflict distance With service request R i The conflict radius ri (reflecting R)i Compare it with the influence range of the required resources); If <ri, then it is determined that R i And R j There is a resource conflict (both compete for the same resource in terms of space or timeliness); If ≥ri, then it is determined that there is no resource conflict between the two; Combine the physical space position (x, y) of the service request with the timeliness level (z) through three-dimensional coordinates (x, y, z), taking into account both the spatial resource competition of services within the same area and the time-sensitive requirements of different services, and avoiding the one-sidedness of judging conflicts only based on physical location or only based on time.

[0050] Quantify the conflict distance and improve the judgment accuracy: By calculating the three-dimensional Euclidean distance d ij , convert the abstract resource conflict into a quantifiable value, and combine it with the conflict radius ri, the influence range of the resources required by the service, to clearly determine whether there is resource competition between service requests, replacing subjective experience judgment, and improving the objectivity and accuracy of conflict recognition.

[0051] Support dynamic resource allocation and optimize service response efficiency: Precise conflict judgment enables the conflict resolution module to preferentially allocate resources to high-priority services, reduce the latency of critical services, and at the same time avoid resource waste, improving the utilization efficiency of computing and communication resources in the vehicle networking.

[0052] For example, in a rainy day scenario, a vehicle needs to process three types of service requests; During rainy day driving, the service collaboration management platform of the vehicle receives three service requests and needs to determine conflicts and allocate resources: R1: Emergency braking assistance service. Since it detects that the target A in front suddenly decelerates, it needs to calculate braking parameters in real time, with a maximum tolerance delay T max1 =0.5s and a conflict radius r1 = 3; R2: Navigation road condition update service, which needs to obtain real-time road conditions, with a maximum tolerance delay T max2 =3s and a conflict radius r2 = 2; R3: Entertainment music push service, non-urgent, with a maximum tolerance delay T max3 =10s and a conflict radius r3 = 1.

[0053] Step 1: Define three-dimensional space coordinates; Physical location: Since all three services come from the same vehicle, so x1 = x2 = x3 = 100m, y1 = y2 = y3 = 50m (the same physical location); Timeliness level , let β = 10; z1 = 10 × 1 / 0.5 = 20, high timeliness, because T max1 Minimum; z2 = 10 × 1 / 3 ≈ 3.33, medium timeliness; z3=10×1 / 10=1, low timeliness.

[0054] Therefore, the three-dimensional coordinates are: R1=(100,50,20), R2=(100,50,3.33), R3=(100,50,1).

[0055] Step 2: Calculate the conflict distance d ij ; Distance between R1 and R2: ; Distance between R1 and R3: ; Distance between R2 and R3: ; Step 3: Resource allocation; The conflict resolution module prioritizes the timeliness of R1 (allocating 60% of communication resources); For the conflicting R2 and R3, since R2 is more timely, z2>z3, allocate 30% of resources to R2 and 10% of resources to R3.

[0056] By using 3D coordinate quantization and conflict determination, navigation delays caused by resource contention between R2 and R3 are avoided, while ensuring priority response to emergency services of R1, thus improving the stability and efficiency of service collaboration in complex rainy scenarios.

[0057] In summary, by quantifying and accurately determining conflicts across multiple dimensions, a scientific basis is provided for resource allocation in vehicle-to-everything (V2X) services. This effectively balances the priorities and resource needs of different services, and significantly improves the efficiency and reliability of service collaboration management.

[0058] Also includes: S5: Construct a service dependency graph G=(V, E), where node V represents a service instance and edge E represents data flow between services; S6: Calculate and dynamically optimize the collaborative cost function C: ; Let e ​​be the communication delay weighting coefficient for edge e; The communication delay (inter-service data transmission time) corresponding to edge e; Calculate the load weight coefficient for node v; The computing load (computing resource consumption of the service instance) corresponding to node v. Constructing a service dependency graph G=(V,E) makes service instances and data flows between services explicit, intuitively showing the dependencies between services. This avoids focusing only on a single service when allocating resources while ignoring the impact of relationships between services. For example, allocating too much computing resources to a certain service may lead to a decrease in overall efficiency due to high communication latency between related services.

[0059] Multi-dimensional cost balancing optimizes overall collaboration efficiency: The collaboration cost function C comprehensively considers communication latency and computational load, balancing these two types of costs by dynamically adjusting weight coefficients. This avoids the drawbacks of single-dimensional optimization and minimizes overall collaboration costs.

[0060] Dynamically adapt to changing scenarios and enhance resource elasticity: By updating the service dependency graph and cost function parameters in real time, resource allocation can quickly respond to dynamic scenarios and enhance the flexibility of vehicle network service collaboration. For example, in a rainy weather scenario: vehicle service instances and dependencies; In rainy driving scenarios, the vehicle's service collaboration management platform runs the following service instances, which require optimization of resource allocation: V1: Emergency braking assistance service node, which needs to calculate braking parameters in real time and calculate the load Lᵥ1=8 units; V2: Navigation traffic update service node, which needs to receive real-time traffic conditions and update the route, with a calculated load of Lᵥ2=5 units; V3: Millimeter-wave radar data processing service node, processes point cloud data and outputs target features, with a computational load of Lᵥ3=12 units; Inter-service data flow (edge), E1: from V3 to V1, transmitting target distance and velocity data, communication latency D e 1 = 0.8 seconds, weight λ e1 =0.6, because emergency services are sensitive to delays; E2: From V3 to V2, transmitting road condition and obstacle data, communication delay D. e 2 = 1.2 seconds, weight λ e 2 = 0.3; E3: Transmitting braking status data from V1 to V2, communication delay D e 3 = 0.5 seconds, weight λ e 3 = 0.1; Calculate the load weights: μᵥ1=0.5, emergency services are highly important, μᵥ2=0.3, μᵥ3=0.4.

[0061] Construct a service dependency graph G=(V,E); Node V = {V1, V2, V3}, edge E = {E1, E2, E3}. Visually, V3 is the upstream service (providing data) for V1 and V2. V1 and V2 have data interaction. To clarify resource allocation, it is necessary to consider that insufficient computing power of V3 will lead to increased latency of E1, which in turn will affect V1 and V2.

[0062] Calculate and optimize the collaborative cost function C; Initial cost calculation: Cooperative cost function C=Σ(λ) e •D e )+Σ(μᵥ・Lᵥ); Communication delay cost: λ e 1・D e 1+λ e 2・D e 2+λ e 3・D e 3 = 0.6 × 0.8 + 0.3 × 1.2 + 0.1 × 0.5 = 0.48 + 0.36 + 0.05 = 0.89; Calculate load cost: μᵥ1・Lᵥ1+μᵥ2・Lᵥ2+μᵥ3・Lᵥ3=0.5×8+0.3×5+0.4×12=4+1.5+4.8=10.3; Initially, C = 0.89 + 10.3 = 11.19.

[0063] Dynamic optimization based on rainy day scenario requirements: Due to increased data processing volume and higher point cloud noise during rainy weather, Lᵥ3 increased to 15 units, E1 latency increased to 1.0 second, and C increased to 13.2. The service collaboration management platform, based on the service dependency graph, discovered: The high load on V3 is the main bottleneck, and its latency also affects V1 and emergency services. Adjustment strategy: Migrate some V3 computing tasks to roadside units, reducing Lᵥ3 to 9 units; optimize E1 transmission paths, using 5G low-latency channels, reducing D e , 1 to 0.6 seconds.

[0064] Optimized cost: Communication delay cost: 0.6×0.6+0.3×1.2+0.1×0.5=0.36+0.36+0.05=0.77; Calculate the load cost: 0.5×8+0.3×5+0.4×9=4+1.5+3.6=9.1; After optimization, C = 0.77 + 9.1 = 9.87, a reduction of approximately 25%.

[0065] The service dependency graph clarifies the strong dependency between V3 and V1, avoiding isolated optimization of V1 while ignoring the bottlenecks of V3; the dynamic optimization of the collaborative cost function C balances communication latency and computational load, ensuring the real-time performance of emergency braking assistance services even under increased data processing pressure in rainy weather, reducing E1 latency, controlling overall resource consumption, and improving the efficiency and stability of vehicle network service collaboration.

[0066] The process of obtaining the multi-objective reward function in step S3 is as follows: ; , and The weights of each objective are (summing up to 1); r safety For safety rewards (such as negative feedback for collision risk); r energy For energy consumption rewards (such as negative feedback on vehicle energy consumption); r latency For delayed rewards (such as negative feedback for service response delays); Among them, the weighting coefficient Dynamically adjust according to service scenarios The dynamic weight coefficient for the i-th objective is defined as follows: ; q i τ represents the service quality evaluation value for the i-th objective, such as safety score, energy consumption score, etc.; τ is a temperature parameter that controls the degree of weight differentiation, and the smaller the value, the more significant the weight difference; exp(∙) is an exponential function. Traditional control strategies often focus on a single objective, while multi-objective reward functions achieve a dynamic balance between safety priority, optimal energy consumption, and low-latency response by weightedly integrating three types of rewards: safety, energy consumption, and latency. This avoids sacrificing one aspect for another. For example, in emergency scenarios, safety is prioritized, while in regular commutes, energy consumption is reduced.

[0067] The weighting coefficient ωᵢ is derived from the service quality evaluation value q. i The temperature parameter τ is dynamically adjusted to enable the strategy to match the needs of the scenario in real time, such as the safety score q when the road is slippery in rainy weather. i As ω decreases, ω1 automatically increases; energy consumption score q for unobstructed road sections. i As ω2 increases, ω2 increases accordingly, solving the problem that fixed weights are difficult to adapt to complex scenarios.

[0068] By transforming abstract service quality into calculable reward values ​​and combining dynamic weights, a clear optimization direction is provided for the strategy optimization module, enabling gradient descent iterations to converge more accurately to the overall optimal solution.

[0069] For example, in a rainy weather scenario: the vehicle avoids collision with target A and evades target B; In rainy driving scenarios, vehicles need to balance the following objectives through flexible control strategies: Security rsafety Collision risk with target A, negative feedback, the lower the risk, the higher the reward, range [-5, 5]; Energy consumption r energy Energy consumption during vehicle braking and acceleration; negative feedback; the lower the energy consumption, the higher the reward; range [-3, 3]. Delay r latency : Ancillary services, such as braking warning response delay, negative feedback, the lower the delay, the higher the reward, range [-2, 2].

[0070] The process of the multi-objective reward function is as follows: The initial reward value and service quality score qi are determined. The vehicle has just detected target A (50 meters) and target B (30 meters). Initial state: moderate collision risk r safety =2, service quality score qi1=60 out of 100, higher risk in rainy weather; Moderate energy consumption energy =1.5, service quality score qi2=70, normal energy consumption level; The response latency is slightly high. latency =0.5, service quality score qi3=65, communication is affected by rain and fog; The temperature parameter τ=20 controls the degree of weight differentiation; a smaller value results in more significant weight differences.

[0071] Calculate the dynamic weighting coefficient ωᵢ; According to the formula ωᵢ=exp(qi / τ) / [exp(q1 / τ)+exp(q2 / τ)+exp(q3 / τ)]: exp(q1 / τ)=exp(60 / 20)=exp(3)≈20.085; exp(q2 / τ)=exp(70 / 20)=exp(3.5)≈33.115; exp(q3 / τ)=exp(65 / 20)=exp(3.25)≈25.790; Denominator = 20.085 + 33.115 + 25.790 ≈ 78.99; Weight: ω1≈20.085 / 78.99≈0.254, ω2≈33.115 / 78.99≈0.419, ω3≈25.790 / 78.99≈0.327.

[0072] Due to changes in the scene and adjustments in weights, target A suddenly decelerates, increasing the risk of collision. Target A brakes suddenly, drastically increasing the risk of collision: rsafety Dropped to -1, high risk, service quality rating q i As the number of points drops from 1 to 40, safety deteriorates. Energy consumption and latency remain essentially unchanged: q i 2=70, q i 3 = 65; Recalculate the weights: exp(q1 / τ)=exp(40 / 20)=exp(2)≈7.389; Denominator = 7.389 + 33.115 + 25.790 ≈ 66.294; Weight: ω1≈7.389 / 66.294≈0.111. Note here: security score q i A decrease in q indicates increased risk, but according to the reward mechanism, the safety weight should be increased at this point. In practice, q i 1 represents the service quality rating; the better the security, the higher the rating. i The higher the value of 1, the more security deteriorates. i To prioritize improving security, the weight ω1 needs to be adjusted in the opposite direction, since exp(qi / τ) varies with q. i When r decreases, it is necessary to optimize the strategy to improve it. safety To improve q i 1.

[0073] Correction logic: When the collision risk increases, the model uses the loss function to provide feedback, causing the policy optimization to shift towards improving r. safety The weights are tilted, and the actual weights are adjusted to ω1=0.6. The simplified manual example reflects the dynamic emphasis, so ω2=0.2 and ω3=0.2.

[0074] Overall reward calculation and strategy optimization: Overall reward before adjustment: r=0.254×2+0.419×1.5+0.327×0.5≈0.508+0.629+0.164≈1.301; The adjustment places greater emphasis on safety: the strategy optimization module outputs more conservative control parameters, such as decelerating 0.8 seconds earlier and increasing braking intensity by 15%. safety Rising to 4 reduces risk. energy When the value is reduced to 1, energy consumption increases slightly, r latency Upgrade to 1.5, latency reduced, overall benefits: r = 0.6 × 4 + 0.2 × 1 + 0.2 × 1.5 = 2.4 + 0.2 + 0.3 = 2.9, a significant improvement.

[0075] By dynamically adjusting the weights, the multi-objective reward function guides the strategy to prioritize reducing collision risks in the event of sudden risks in rainy weather, while also taking into account energy consumption and latency. This makes the output dynamic control parameters, such as braking timing and deceleration magnitude, more aligned with the safety-first scenario requirements in rainy weather, avoiding the sacrifice of safety for energy saving or the neglect of risks for low latency caused by fixed weights.

[0076] In summary, through multi-objective collaboration and dynamic weight adjustment, the flexible control strategy can accurately adapt to the core needs of complex traffic scenarios, balancing energy consumption and service latency while ensuring safety, and significantly improving the overall quality of vehicle-to-everything (V2X) auxiliary services.

[0077] The specific optimization process of the strategy optimization module in step S3 includes: SS1: Initialize the policy parameters θ, including the control parameter weight matrix and bias terms; SS2: Fuzzy membership degree μ output by the fuzzy membership degree calculation module. ij The initial control parameters π(θ) are generated by combining the input vectors (Xf, Xs, Xp) of the flexible control strategy model. SS3: Calculate the control error E and the overall reward r: The control error E = |π(θ) - π*|, where π* is the ideal control parameter, set based on historical best data or expert experience; The overall reward r is calculated using a multi-objective reward function; SS4: Construct the policy loss function L=α・E-(1-α)・r, where α is the balance coefficient (0<α<1); SS5: Update policy parameters using gradient descent. θnew=θ-η・∂L / ∂θ, where η is the learning rate and ∂L / ∂θ is the gradient of the loss function with respect to the parameter θ; SS6: Repeat steps SS1-SS5 until |θnew-θ|<ε, where ε is the convergence threshold, and output the final dynamic control parameter π(θnew).

[0078] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0079] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0080] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A vehicle-to-everything (V2X) AI flexible control method based on assisted services, characterized in that, Includes the following steps: S1: Collect raw environmental data through multimodal sensors deployed in vehicles, roadside units, and the cloud. The raw environmental data includes raw vehicle operation data, raw road environment data, raw user behavior data, visible light images, infrared thermal imaging data, and millimeter-wave radar point cloud data. S2: Construct a cross-modal feature alignment network to perform spatiotemporal alignment and feature fusion on visible light images, infrared thermal imaging data and millimeter-wave radar point cloud data to generate an environment fusion feature map; S3: Generate a flexible control strategy model based on a deep reinforcement learning model, wherein the deep reinforcement learning model includes a fuzzy membership calculation module, a policy optimization module, and a multi-objective reward function; The fuzzy membership calculation module is used to quantify the fuzzy attributes of the input features, and its output is the membership value of the input features to each fuzzy set. The strategy optimization module is used to iteratively optimize the strategy based on membership values ​​and multi-objective reward functions using gradient descent, minimizing control error and maximizing comprehensive reward, and finally outputting dynamic control parameters that meet the needs of the scenario. The inputs to the flexible control strategy model include: The compressed vector Xf extracted from the environmental fusion feature map; The vehicle state vector Xs is obtained by preprocessing the raw vehicle operation data. The user preference vector Xp is extracted from raw user behavior data; The output is dynamic control parameters for auxiliary services; S4: Deploy a conflict resolution module in the service collaboration management platform to dynamically allocate computing and communication resources based on a flexible control strategy model.

2. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 1, characterized in that: The raw vehicle operation data in step S1 includes real-time sensor readings of vehicle speed, acceleration, steering angle, braking status, and fault codes. The data sampling frequency is 10Hz. The braking status is represented by binary 0 to indicate no braking and 1 to indicate braking. The fault codes are represented by digital codes to indicate the fault type and level. The vehicle state vector Xs in step S3 is generated through the following steps: The raw vehicle operation data is preprocessed to obtain the preprocessed data; Features are extracted from the preprocessed data, including time-domain features and frequency-domain features; Time-domain features are calculated based on the time series data of the original data: The dynamic rate of change of vehicle speed is calculated by the ratio of the difference in vehicle speed between two adjacent sampling periods to the sampling time interval, i.e., 0.1s, i.e., (v(t)−v(t−1)) / 0.1; v(t) is the vehicle speed value in the current sampling period; v(t−1) is the vehicle speed value in the previous sampling period; The peak acceleration is the maximum acceleration value extracted within a sliding window of 10 sampling periods, i.e., 1 second. The steering angle fluctuation frequency is the number of times the steering angle change direction switches within 10 sampling periods, that is, the number of times it switches between positive and negative directions; The duration of the braking state is the product of the number of consecutive samples and the sampling period when the braking state is 1. Fault code frequency: Count the number of times a fault code has a non-zero value within 600 sampling periods, i.e., within 60 seconds; Frequency domain features are obtained by performing a Fourier transform on the preprocessed raw data: The dominant frequency of engine vibration is determined by performing a Fourier transform on the acceleration time series and selecting the frequency value with the highest power spectral density. The chassis resonance frequency is obtained by performing a Fourier transform on the steering angle time series and taking the frequency value corresponding to the second largest power spectral density. The vehicle's real-time operating status is classified according to preset rules to obtain the vehicle status classification results; Feature selection algorithm is used to filter features: the vehicle state classification result is used as the decision attribute set D, and features with γi>τ are selected to form the dimension-compressed vehicle state vector Xs. Grey relational analysis was used to classify vehicle status as the decision attribute set D. Calculate the correlation γi between the extracted time-domain features and frequency-domain features and D; The features selected from γi>τ constitute the dimension-compressed vehicle state vector Xs.

3. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 2, characterized in that: The feature selection algorithm uses grey relational analysis to obtain Xs. The specific process is as follows: The vehicle status classification results are used as the decision attribute set D; Calculate the correlation between time-domain or frequency-domain features and D. : ； Where n is the number of samples; Let be the decision attribute value of the k-th sample; Let i be the value of the i-th feature in the k-th sample; It is the minimum difference between two levels; The maximum difference between the two levels; ρ is the resolution coefficient; τ is the correlation threshold; choose The features constitute Xs.

4. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 1, characterized in that: Step S2 further includes: S21: A dual-stream convolutional neural network encoder is used to process two types of image data respectively: Visible light image data I v Spatial features are extracted using a two-stream convolutional neural network model Φ to obtain visible light features. Specifically, ; Infrared thermal imaging data I t Infrared features were obtained by extracting thermal distribution features using the same two-stream convolutional neural network model Φ. Specifically: ; in, This is a convolutional neural network model used to extract features from visible light images and infrared thermal imaging. S22: P for millimeter-wave radar point cloud data r Voxelization is performed to generate millimeter-wave radar point cloud features F. r Specifically, this includes: P r The data is divided into a uniform three-dimensional grid, i.e., a voxel grid; the point cloud density and spatial distribution entropy of each voxel grid are calculated and integrated into the millimeter-wave radar point cloud feature F. r ; S23: Through the cross-modal attention module , F r Perform spatiotemporal alignment and output alignment feature F. align The calculation formula is: ； σ is the activation function; Q is the query vector; K is the key vector; V is the value vector; K represents the dimension of the key vector K; T This is the transpose of the key vector; S24: Align feature F align Input a feature pyramid network, and generate the final environment fusion feature map F through multi-scale feature fusion. fusion .

5. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 4, characterized in that: The specific process of S22 is as follows: S221: Divide the data into a three-dimensional voxel lattice, and then divide the millimeter-wave radar point cloud data P. r The data is divided into a uniformly distributed three-dimensional grid, with each grid defined as a voxel grid, serving as the basic spatial unit for point cloud feature extraction. S222: Calculate the point cloud density feature ρ within each voxel lattice. For each voxel lattice, calculate the density feature of the point cloud within it using the following formula: ； in, This represents the number of radar points contained within the voxel lattice. This represents the spatial volume of the voxel lattice. S223: Extracting the spatial distribution entropy of point cloud within a voxel lattice For each voxel lattice, its distribution entropy is calculated based on the spatial distribution of points. The formula is: ； Where M is the total number of points within the voxel lattice. Let be the probability distribution of the i-th point within the voxel lattice; S224: Generating millimeter-wave radar point cloud features F r The density feature ρ and spatial distribution entropy of each voxel lattice are combined. The data is integrated to form millimeter-wave radar point cloud data P. r The corresponding millimeter-wave radar point cloud feature F r .

6. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 1, characterized in that: The fuzzy membership calculation module in step S3 is used to perform the following operations: S31: Calculate fuzzy membership values ​​using an adaptive Gaussian membership function. ； in, The input feature vector; σ is the center parameter corresponding to the i-th feature of the j-th fuzzy set; ij Let be the standard deviation parameter of the i-th feature corresponding to the j-th fuzzy set; S32: Dynamically update the membership function parameters, the specific process is as follows: ， ； Standard deviation parameter The update amount; η is the learning rate; J is the loss function for reinforcement learning; and For the loss function with respect to parameters σ ij The partial derivatives of .

7. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 1, characterized in that: Step S4 further includes: For each service request R i Define three-dimensional spatial coordinates ,in Mapping timeliness level: ; Where β is the proportionality coefficient; T max For service request R i Maximum tolerable delay; For any two service requests R i and R j Calculate the three-dimensional spatial conflict distance between the two. : ； in, , and For service request R i Three-dimensional spatial coordinates, and For physical space coordinates, For time-sensitive components; , and For service request R j Three-dimensional spatial coordinates; The calculated conflict distance With service request R i The conflict radius ri is compared; If j < ri, then it is determined that R i and R j have a resource conflict; like If ≥ri, then it is determined that there is no resource conflict between the two.

8. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 1, characterized in that: Also includes: S5: Construct a service dependency graph G=(V, E), where node V represents a service instance and edge E represents data flow between services; S6: Calculate and dynamically optimize the collaborative cost function C: ； Let e ​​be the communication delay weighting coefficient for edge e; The communication delay corresponding to edge e; Calculate the load weight coefficient for node v; This represents the computational load corresponding to node v.

9. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 1, characterized in that: The process of obtaining the multi-objective reward function in step S3 is as follows: ； , and r represents the weighting coefficient for each objective. safety For security rewards; r energy Energy consumption reward; r latency For delayed rewards; Among them, the weighting coefficient Dynamically adjust according to service scenarios The dynamic weight coefficient for the i-th objective is defined as follows: ； Let be the service quality evaluation value for the i-th target; τ be the temperature parameter; and exp(∙) be the exponential function.

10. The vehicle-to-everything (V2X) AI flexible control method based on assisted services according to claim 1, characterized in that: The specific strategy optimization process of the strategy optimization module in step S3 includes: SS1: Initialize policy parameters θ; SS2: Fuzzy membership degree μ based on the output of the fuzzy membership degree calculation module ij The initial control parameters π(θ) are generated by combining the input vectors (Xf, Xs, Xp) of the flexible control strategy model. SS3: Calculate the control error E and the overall reward r: The control error E = |π(θ) - π*|, where π* is the ideal control parameter; The overall reward r is calculated using a multi-objective reward function; SS4: Construct the policy loss function L=α・E-(1-α)・r, where α is the balance coefficient; SS5: Update policy parameters using gradient descent. θnew=θ-η・∂L / ∂θ, where η is the learning rate and ∂L / ∂θ is the gradient of the loss function with respect to the parameter θ; SS6: Repeat steps SS1-SS5 until |θnew-θ|<ε, where ε is the convergence threshold, and output the final dynamic control parameter π(θnew).

Images