Vehicle-cloud collaborative perception defense method, electronic device, storage medium and program product

Abstract

This application provides a vehicle-cloud collaborative perception and defense method, electronic device, storage medium, and program product. It relates to the field of vehicle-cloud collaborative technology. The method is applied to a vehicle client, on which a local model is deployed. This local model is used for attack detection and proactive defense response. The vehicle-cloud collaborative perception and defense method includes: extracting and inferring features from multi-source sensor data using the local model to obtain predicted logical values ​​and corresponding local prediction probability distributions; calculating variational free energy based on the global prior knowledge distribution and local prediction probability distribution received from a cloud server; sending the variational free energy and predicted logical values ​​to the cloud server and receiving the global logical values ​​and global prediction probability distribution returned by the cloud server; and optimizing the parameters of the local model with the goal of minimizing the composite loss. This application aims to improve the detection performance of the global model, thereby enhancing the vehicle-cloud collaborative perception and defense capabilities.

Description

Technical Field

[0001] This application relates to the field of vehicle-cloud collaborative technology, and in particular to a vehicle-cloud collaborative perception and defense method, electronic device, storage medium and program product. Background Technology

[0002] In modern intelligent transportation systems, vehicles exchange multimodal sensor data via edge-cloud connections to achieve collaborative perception, which increases the attack surface of vehicular networks and poses a risk of privacy breaches. Federated learning is used for distributed training of intrusion detection models, but its parameter transmission faces challenges due to limitations in vehicular network bandwidth and computing power.

[0003] To protect privacy and ensure collaborative security, the relevant technologies employ a federated distillation defense approach. Specifically, a locally deployed model within the vehicle client extracts features and infers from multimodal sensor data to obtain predicted logical values, which are then uploaded to a cloud server. The cloud server typically aggregates global knowledge using a weighted average based on sample size or a preset fixed threshold rule, and then broadcasts the aggregated global knowledge to the vehicle client to guide local model updates. This approach effectively reduces communication overhead by exchanging model output knowledge to replace parameters, but it struggles to adapt to the complex scenarios in vehicle-to-everything (V2X) environments, where data is not independently and identically distributed, the environment is dynamically changing, and attack patterns are diverse. This results in a decline in the detection performance of the global model and fails to meet the security requirements of intelligent transportation systems in complex adversarial environments. Summary of the Invention

[0004] This application provides a vehicle-cloud collaborative perception defense method, electronic device, storage medium, and program product to improve the detection performance of the global model and enhance the vehicle-cloud collaborative perception defense capability.

[0005] In a first aspect, embodiments of this application provide a vehicle-cloud collaborative perception and defense method, applied to a vehicle client. A local model is deployed on the vehicle client, and this local model is used for attack detection and proactive defense response. The vehicle-cloud collaborative perception and defense method includes:

[0006] By using a local model to extract features and infer from multi-source sensor data, the predicted logical value and the local prediction probability distribution corresponding to the predicted logical value are obtained.

[0007] Based on the global knowledge prior distribution and the local prediction probability distribution received from the cloud server, the variational free energy is calculated. The variational free energy is used to indicate the confidence level of the predicted logical value.

[0008] Send variational free energy and predicted logical value to the cloud server, and receive global logical value and global predicted probability distribution returned by the cloud server. The global logical value is obtained by the cloud server through global knowledge weighted aggregation of the predicted logical value based on variational free energy, and the global predicted probability distribution is the probability distribution of the global logical value.

[0009] With the goal of minimizing the composite loss, the parameters of the local model are optimized. The composite loss is obtained based on the global logistic value, the global prediction probability distribution, and the local prediction probability distribution.

[0010] In one possible implementation, the variational free energy is calculated based on the global prior knowledge distribution received from the cloud server and the local prediction probability distribution, including:

[0011] Based on the local prediction probability distribution, the information entropy term corresponding to the local prediction probability distribution is obtained;

[0012] Based on the local prediction probability distribution and the global knowledge prior distribution, the divergence term corresponding to the local prediction probability distribution is obtained;

[0013] The variational free energy is obtained by adding the product of the information entropy term and the entropy weight to the divergence term.

[0014] In one possible implementation, the local prediction probability distribution includes multiple categories and a probability value corresponding to each category; based on the local prediction probability distribution, the information entropy term corresponding to the local prediction probability distribution is obtained, including:

[0015] Each category in the local predicted probability distribution is taken as the target category in turn;

[0016] The product of the probability value corresponding to the target category and the natural logarithm of the probability value corresponding to the target category is used as the information entropy of the target category.

[0017] The information entropy is accumulated and its negative value is taken to obtain the information entropy term corresponding to the local prediction probability distribution.

[0018] In one possible implementation, the local prediction probability distribution includes multiple categories and a probability value corresponding to each category; the global knowledge prior distribution includes multiple categories identical to the local prediction probability distribution and a prior value corresponding to each category; and based on the local prediction probability distribution and the global knowledge prior distribution, the divergence term corresponding to the local prediction probability distribution is obtained, including:

[0019] Each category in the local predicted probability distribution is taken as the target category in turn;

[0020] The first value of the target category is obtained by dividing the probability value corresponding to the target category by the prior value corresponding to the target category.

[0021] The product of the probability value of the target class and the natural logarithm of the first value is used as the first divergence of the target class;

[0022] The first divergence is accumulated to obtain the divergence term corresponding to the local predicted probability distribution.

[0023] In one possible implementation, the local prediction probability distribution includes an attack probability distribution and a defense strategy probability distribution, and the composite loss is obtained through the following methods:

[0024] The distillation loss is obtained based on the local and global prediction probability distributions.

[0025] The inference loss is obtained based on the negative log-likelihood value of the attack probability distribution;

[0026] The defense loss is obtained based on the negative log-likelihood value of the probability distribution of the defense strategy;

[0027] The weighted sum of distillation loss, reasoning loss, and defense loss yields the composite loss.

[0028] In one possible implementation, the distillation loss is obtained based on the local prediction probability distribution and the global prediction probability distribution, including:

[0029] The classification cross-entropy is obtained by measuring the difference between the local predicted probability distribution and the true label;

[0030] Using the distillation temperature parameter, the second divergence between the local predicted probability distribution and the global predicted probability distribution is calculated;

[0031] The distillation loss is obtained by weighting and summing the classification cross-entropy and the second divergence.

[0032] One possible implementation also includes:

[0033] The confidence score is the probability value corresponding to the attack probability distribution in the local predicted probability distribution.

[0034] If the confidence score is greater than the security threshold, then active defense is executed based on the defense strategy, which includes adjusting the weight values ​​of the corresponding attack modality data in the feature fusion module of the local model.

[0035] In one possible implementation, after calculating the variational free energy based on the global prior knowledge distribution and the local prediction probability distribution received from the cloud server, the method further includes:

[0036] If the variational free energy exceeds a preset threshold, an active defense response is triggered on the vehicle client. The active defense response includes: discarding the predicted logical value and the local predicted probability distribution corresponding to the predicted logical value, and / or triggering an anomaly alarm.

[0037] Secondly, this application provides a vehicle-cloud collaborative perception defense method, applied to a cloud server. The vehicle-cloud collaborative perception defense method includes:

[0038] It receives variational free energy and predicted logic value sent by multiple vehicle clients respectively. The variational free energy is used to indicate the confidence level of the predicted logic value.

[0039] Based on the aggregated temperature parameters, the variational free energy of each vehicle client is exponentially calculated to obtain the credibility score of each vehicle client.

[0040] Based on the credibility score corresponding to each vehicle client, the predicted logical values ​​of multiple vehicle clients are aggregated using global knowledge weighting to obtain the global logical value and the global predicted probability distribution of the global logical value.

[0041] Global logical values ​​and global predicted probability distributions are sent to multiple vehicle clients. Both global logical values ​​and global predicted probability distributions are used for parameter optimization of the local models deployed in multiple vehicle clients.

[0042] In one possible implementation, based on the aggregated temperature parameter, an exponential calculation is performed on the variational free energy of each vehicle client to obtain a credibility score for each vehicle client, including:

[0043] The exponential energy of each vehicle client is obtained by performing a natural exponential operation on the quotient of the inverse of the variational free energy of each vehicle client divided by the aggregation temperature parameter.

[0044] The normalization factor is obtained by summing the exponential energies.

[0045] The credibility score of each vehicle client is obtained by dividing the exponential energy of each vehicle client by the normalization factor.

[0046] In one possible implementation, based on the credibility score corresponding to each vehicle client, the predicted logical values ​​of multiple vehicle clients are globally weighted and aggregated to obtain a global logical value, including:

[0047] Multiply the credibility score corresponding to each vehicle client by the predicted logical value of the vehicle client to obtain the aggregate logical value corresponding to each vehicle client.

[0048] Aggregate the aggregated logical values ​​from multiple vehicle clients to obtain a global logical value.

[0049] One possible implementation further includes: updating the global knowledge prior distribution based on the global logical value, and sending the updated global knowledge prior distribution to multiple vehicle clients respectively.

[0050] One possible implementation involves updating the global knowledge prior distribution based on a global logical value, including:

[0051] The global logical values ​​are exponentialized and normalized to obtain the global prediction probability distribution;

[0052] Based on the global prediction probability distribution, the global knowledge prior distribution is updated to obtain the updated global knowledge prior distribution.

[0053] Thirdly, embodiments of this application provide a vehicle client for implementing the vehicle-cloud collaborative perception and defense method as described in the first aspect and / or various possible implementations of the first aspect.

[0054] Fourthly, embodiments of this application provide a cloud server for implementing the vehicle-cloud collaborative perception and defense method as described in the second aspect and / or various possible implementations of the second aspect.

[0055] Fifthly, embodiments of this application provide a vehicle-cloud collaborative perception and defense system, including a cloud server as described in the fourth aspect and / or various possible implementations of the fourth aspect, and at least one vehicle client as described in the third aspect and / or various possible implementations of the third aspect.

[0056] Sixthly, embodiments of this application provide a vehicle having a vehicle client deployed with the third aspect and / or various possible implementations of the third aspect as described above.

[0057] In a seventh aspect, embodiments of this application provide an electronic device, including: a memory and a processor;

[0058] The memory stores instructions that the computer executes;

[0059] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0060] Eighthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0061] Ninthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0062] The vehicle-cloud collaborative perception defense method, electronic device, storage medium, and program product provided in this application extract and infer features from multi-source sensor data using a local model to obtain predicted logical values ​​and corresponding local predicted probability distributions. This fully utilizes the rich sensor information available on the vehicle, quickly obtaining the local predicted probability distributions corresponding to the predicted logical values, reducing the time delay of multi-source sensor data transmission to the cloud, and improving the real-time performance of feature extraction and inference. Based on the global knowledge prior distribution and local predicted probability distribution received from the cloud server, variational free energy is calculated. This can be combined with global knowledge to evaluate the local predicted probability distribution, better quantifying its reliability and better responding to various complex attack scenarios and environmental changes. Sending variational free energy and predicted logical values ​​to the cloud server and receiving the global logical values ​​and global predicted probability distribution returned by the cloud server allows the vehicle client to obtain the global logical values ​​and global predicted probability distributions obtained by considering variational free energy from the cloud server. This enhances the vehicle client's attack detection and proactive defense response capabilities, better adapting to complex and ever-changing traffic environments and attack methods. Optimizing the parameters of the local model with the goal of minimizing the composite loss enables the local model to continuously learn and improve, enhancing its ability to extract features and infer from multi-source sensor data. This makes the inference results of the local model closer to the global logistic value and the global prediction probability distribution, thereby improving the model's performance. At the same time, by considering the global logistic value and the global prediction probability distribution, the local model can also better adapt to different scenarios and data changes, enhancing its robustness. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0064] Figure 1 A schematic diagram of a scenario for the vehicle-cloud collaborative perception and defense method provided in the embodiments of this application;

[0065] Figure 2 A flowchart illustrating the vehicle-cloud collaborative perception and defense method provided in this application embodiment. Figure 1 ;

[0066] Figure 3 A flowchart illustrating the vehicle-cloud collaborative perception and defense method provided in this application embodiment. Figure 2 ;

[0067] Figure 4 A flowchart illustrating the vehicle-cloud collaborative perception and defense method provided in this application embodiment. Figure 3 ;

[0068] Figure 5A flowchart illustrating the vehicle-cloud collaborative perception and defense method provided in this application embodiment. Figure 4 ;

[0069] Figure 6 This is a schematic diagram of the system structure of the vehicle-cloud collaborative perception and defense system provided in the embodiments of this application;

[0070] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0071] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0072] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0073] In related technologies, a federated distillation defense approach is used to achieve privacy protection and collaborative security. Specifically, a locally deployed model within the vehicle client extracts features and infers from multimodal sensor data to obtain predicted logical values, which are then uploaded to a cloud server. The cloud server typically receives the predicted logical values ​​from the vehicle client and aggregates global knowledge using a weighted average based on sample size or a preset fixed threshold rule. This aggregated global knowledge is then broadcast back to the vehicle client to guide local model updates. However, the knowledge aggregation process on the vehicle client, the core of federated distillation, lacks an effective mechanism for evaluating the credibility of knowledge signals. This makes it vulnerable to malicious attacks that "poison" knowledge uploaded by vehicle clients, leading to performance degradation and security risks for the global model. This approach struggles to effectively eliminate malicious interference in complex scenarios such as non-independent and identically distributed data, dynamic environmental changes, and diverse attack patterns in the Internet of Vehicles (IoV), resulting in decreased detection performance of the global model and low security. It fails to meet the security requirements of intelligent transportation systems in complex adversarial environments.

[0074] The vehicle-cloud collaborative perception defense method provided in this application extracts and infers features from multi-source sensor data using a local model, obtaining predicted logical values ​​and corresponding local predicted probability distributions. This fully utilizes the rich sensor information available on the vehicle, quickly obtaining the local predicted probability distributions corresponding to the predicted logical values, reducing the time delay of multi-source sensor data transmission to the cloud, and improving the real-time performance of feature extraction and inference. Based on the global knowledge prior distribution and the local predicted probability distribution received from the cloud server, the variational free energy is calculated. This can be combined with global knowledge to evaluate the local predicted probability distribution, better quantifying its reliability and better responding to various complex attack scenarios and environmental changes. By sending the variational free energy and predicted logical values ​​to the cloud server and receiving the global logical values ​​and global predicted probability distribution returned by the cloud server, the vehicle client can obtain the global logical values ​​and global predicted probability distributions obtained by the cloud server after considering the variational free energy. This enhances the vehicle client's attack detection and proactive defense response capabilities, better adapting to complex and ever-changing traffic environments and attack methods. Optimizing the parameters of the local model with the goal of minimizing the composite loss enables the local model to continuously learn and improve, enhancing its ability to extract features and infer from multi-source sensor data. This makes the inference results of the local model closer to the global logistic value and the global prediction probability distribution, thereby improving the model's performance. At the same time, by considering the global logistic value and the global prediction probability distribution, the local model can also better adapt to different scenarios and data changes, enhancing its robustness.

[0075] Figure 1 This is a schematic diagram of a scenario for the vehicle-cloud collaborative perception and defense method provided in the embodiments of this application, such as... Figure 1 As shown, the specific application scenario of this application embodiment includes multiple vehicles 11 and a cloud server 12, wherein each vehicle 11 is equipped with a vehicle client 13. Wherein:

[0076] The vehicle client 13 includes a data input module 131, a multimodal feature fusion module 132, a local model inference module 133, a model update module 134, a variational free energy calculation module 135, and an active defense module 136. Specifically, the data input module 131 receives multi-source sensor data and transmits it to the multimodal feature fusion module 132. The multimodal feature fusion module 132 extracts features from the received multi-source sensor data to obtain multi-source feature data, which is then transmitted to the local model inference module 133. Based on the multi-source feature data, the local model inference module 133 performs corresponding inference tasks to obtain the predicted logical value and the corresponding local prediction probability distribution. Then, the variational free energy calculation module 135 calculates the variational free energy based on the global prior knowledge distribution received from the cloud server and the local prediction probability distribution. When the variational free energy exceeds a preset threshold, the vehicle client 13 can control the active defense module 136 to trigger an active defense response against the vehicle client 13. The vehicle client 13 can also control the model update module 134 to update the multimodal feature fusion module 132 and the local model inference module 133 when the variational free energy does not exceed a preset threshold.

[0077] Furthermore, the cloud server 12 includes a credibility assessment module 121 and an aggregation model 123.

[0078] The credibility assessment module 121 receives variational free energy and predicted logical values ​​sent by multiple vehicle clients, and performs exponential calculation on the variational free energy of each vehicle client based on the aggregated temperature parameter to obtain a credibility score for each vehicle client. The aggregation model 123 performs global knowledge weighted aggregation on the predicted logical values ​​of multiple vehicle clients based on the credibility scores corresponding to each vehicle client, obtaining a global logical value and a global predicted probability distribution of the global logical value; it then sends the global logical value and global predicted probability distribution to multiple vehicle clients, both of which are used for parameter optimization of the local models deployed in the multiple vehicle clients.

[0079] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0080] Figure 2 A flowchart illustrating the vehicle-cloud collaborative perception and defense method provided in this application embodiment. Figure 1 ,like Figure 2As shown, this method is applied to the vehicle client, where a local model is deployed. This local model is used for attack detection and proactive defense response. The vehicle-cloud collaborative perception and defense method includes:

[0081] S201. The local model is used to extract features and infer from multi-source sensor data to obtain the predicted logical value and the local prediction probability distribution corresponding to the predicted logical value.

[0082] Multi-source sensor data refers to data collected by various types of sensors equipped on a vehicle. This data is obtained by acquiring information about the vehicle's surrounding environment and the interior environment from different angles and with varying physical characteristics using multiple types of sensors.

[0083] A local model is a model deployed on the vehicle client that has the capability to process and analyze data from multiple sensor sources. By performing feature extraction and inference on multi-source sensor data using a local model, tasks related to attack detection and proactive defense response can be accomplished.

[0084] The predicted logical value is a preliminary judgment made by the local model based on feature extraction and inference of multi-source sensor data, regarding the category or state of the multi-source sensor data. Optionally, the predicted logical value can be represented in a logical form.

[0085] The local prediction probability distribution is the probability distribution of different categories obtained by the local model for the predicted logical value. Optionally, the predicted logical value can be exponentially normalized using preset temperature parameters to generate the local prediction probability distribution for the vehicle client. For example, this exponentialization and normalization can be performed by executing a Softmax operation.

[0086] First, data from various types of multi-source sensors is collected to obtain multi-source sensor data. Then, this multi-source sensor data is input into a local model deployed on the vehicle's client. The local model performs feature extraction on the input multi-source sensor data, extracting valuable feature information from the raw data using specific algorithms and structures. Next, inference calculations are performed based on the extracted features to derive predicted logical values. Finally, the local predicted probability distribution corresponding to the predicted logical values ​​is calculated. By using a local model for feature extraction and inference on multi-source sensor data, the rich sensor information available on the vehicle can be fully utilized, quickly obtaining the local predicted probability distribution corresponding to the predicted logical values, reducing the time latency of multi-source sensor data transmission to the cloud, and improving the real-time performance of feature extraction and inference.

[0087] In vehicle-cloud collaborative perception and defense scenarios, timely and accurate preliminary analysis of multi-source sensor data is crucial for more precise judgment and defense. Rapid processing of local models provides an efficient starting point for the entire defense process, helping to promptly detect potential attack threats and ensuring the safe operation of vehicles.

[0088] Optionally, the multi-source sensor data includes image data and LiDAR point cloud data. Further, to process the image data and LiDAR point cloud data from the multi-source sensor data, the local model includes a visual encoder for extracting features from the image data, a point cloud encoder for extracting features from the LiDAR point cloud data, a feature fusion module for the two different modalities, and a logic layer. For example, the local model receives input from the multi-source sensor data, passes the image data to the visual encoder to obtain the visual semantic features output by the visual encoder, and passes the LiDAR point cloud data to the point cloud encoder to obtain the geometric spatial features output by the point cloud encoder. Then, the feature fusion module receives the visual semantic features and geometric spatial features and generates a unified fused feature representation through a concatenation operation. Then, the logic layer receives the fused feature representation and performs forward computation, outputting a predicted logical value for a preset category. By utilizing preset temperature parameters, the predicted logical value is exponentially normalized to generate a local predicted probability distribution for the vehicle client.

[0089] S202. Based on the global knowledge prior distribution and the local prediction probability distribution received from the cloud server, the variational free energy is calculated. The variational free energy is used to indicate the confidence level of the predicted logical value.

[0090] Global prior knowledge distribution is a representation of global information obtained by a cloud server by integrating a large amount of historical data and information from multiple vehicle clients. This pre-learned probability distribution under different conditions reflects the general patterns and prior knowledge within the entire vehicle-cloud collaborative perception and defense scenario.

[0091] Variational free energy (VFE) is a metric calculated by combining the local prediction probability distribution and the global prior knowledge distribution. It measures the reliability of the prediction logic value and the local prediction probability distribution. Generally, the smaller the VFE value, the more reliable the local prediction probability distribution obtained by the vehicle client, and the lower the possibility of poisoning the cloud server.

[0092] After the vehicle client completes the inference of the local model and obtains the local predicted probability distribution, it receives the global knowledge prior distribution from the cloud server. Then, according to the variational free energy calculation method, the local predicted probability distribution and the global knowledge prior distribution are used as input parameters to calculate the variational free energy value. This quantifies the reliability of the local predicted probability distribution, which helps to filter out predictions with low reliability and avoid security risks caused by erroneous predictions.

[0093] In vehicle-cloud collaborative perception and defense scenarios, relying solely on feature extraction and inference from local models for attack detection and proactive defense response may have limitations. Introducing global knowledge prior distribution and calculating variational free energy allows for the evaluation of local prediction probability distributions in conjunction with global knowledge, better quantifying the reliability of local prediction probability distributions, and better responding to various complex attack scenarios and environmental changes.

[0094] S203. Send the variational free energy and the predicted logical value to the cloud server, and receive the global logical value and the global predicted probability distribution returned by the cloud server. The global logical value is obtained by the cloud server through global knowledge weighted aggregation of the predicted logical value based on the variational free energy. The global predicted probability distribution is the probability distribution of the global logical value.

[0095] The global logical value is the variational free energy received by the cloud server from vehicle clients. It combines the variational free energy and predicted logical values ​​from multiple vehicle clients, performing a global knowledge-weighted aggregation of the predicted logical values. Compared to the predicted logical values ​​themselves, the global logical value offers better globality, accuracy, and reliability. The global prediction probability distribution is the probability distribution of the global logical value.

[0096] The vehicle client sends the calculated variational free energy and predicted logical value to the cloud server. Upon receiving the variational free energy and predicted logical value, the cloud server utilizes its internal global knowledge and algorithms to perform a weighted aggregation operation on the predicted logical value based on the variational free energy, obtaining the global logical value. Simultaneously, it calculates the global predicted probability distribution corresponding to the global logical value. Then, the cloud server returns the global logical value and global predicted probability distribution to the vehicle client, enabling the vehicle client to obtain the globally processed logical value and global predicted probability distribution. This improves the vehicle client's attack detection and proactive defense response capabilities, allowing it to better adapt to complex and ever-changing traffic environments and attack methods.

[0097] Meanwhile, the cloud server uses variational free energy to perform global knowledge weighted aggregation on the predicted logical value to obtain the global logical value. It can detect and defend against malicious behavior of vehicle clients through variational free energy, so as to effectively identify and suppress the impact of malicious and / or unreliable vehicle clients on global knowledge and defend against malicious vehicles uploading "poisoning" knowledge signals.

[0098] S204. To minimize the composite loss, the parameters of the local model are optimized. The composite loss is obtained based on the global logistic value, the global prediction probability distribution, and the local prediction probability distribution.

[0099] Composite loss is a loss function value calculated by comprehensively considering factors such as global logistic value, global prediction probability distribution, and local prediction probability distribution. Composite loss is used to measure the degree of difference between the prediction results of the local model and global information, and to guide the parameter optimization of the local model.

[0100] After receiving the global logistic value and global predicted probability distribution from the cloud server, the vehicle client calculates the composite loss using the global logistic value, global predicted probability distribution, and local predicted probability distribution as input parameters, according to a pre-defined composite loss calculation formula. Then, aiming to minimize the composite loss, an optimization algorithm is used to adjust the parameters of the local model. This allows the local model to continuously learn and improve, enhancing its feature extraction and inference capabilities from multi-source sensor data. The inference results of the local model are made closer to the global logistic value and global predicted probability distribution, improving model performance. Simultaneously, by considering the global logistic value and global predicted probability distribution, the local model can better adapt to different scenarios and data changes, enhancing its robustness. Optionally, the optimization algorithm can be gradient descent.

[0101] In vehicle-cloud collaborative perception and defense scenarios, local models need to continuously improve their performance to cope with various complex attacks and environmental changes. By comparing the local model with the global logical values ​​and global predicted probability distribution returned by the cloud server and optimizing the model parameters, the local model can better integrate global knowledge, improve attack detection and proactive defense capabilities, and ensure the safe operation of vehicles.

[0102] The vehicle-cloud collaborative perception defense method provided in this application extracts and infers features from multi-source sensor data using a local model, obtaining predicted logical values ​​and corresponding local predicted probability distributions. This fully utilizes the rich sensor information available on the vehicle, quickly obtaining the local predicted probability distributions corresponding to the predicted logical values, reducing the time delay of multi-source sensor data transmission to the cloud, and improving the real-time performance of feature extraction and inference. Based on the global knowledge prior distribution and the local predicted probability distribution received from the cloud server, the variational free energy is calculated. This can be combined with global knowledge to evaluate the local predicted probability distribution, better quantifying its reliability and better responding to various complex attack scenarios and environmental changes. By sending the variational free energy and predicted logical values ​​to the cloud server and receiving the global logical values ​​and global predicted probability distribution returned by the cloud server, the vehicle client can obtain the global logical values ​​and global predicted probability distributions obtained by the cloud server after considering the variational free energy. This enhances the vehicle client's attack detection and proactive defense response capabilities, better adapting to complex and ever-changing traffic environments and attack methods. Optimizing the parameters of the local model with the goal of minimizing the composite loss enables the local model to continuously learn and improve, enhancing its ability to extract features and infer from multi-source sensor data. This makes the inference results of the local model closer to the global logistic value and the global prediction probability distribution, thereby improving the model's performance. At the same time, by considering the global logistic value and the global prediction probability distribution, the local model can also better adapt to different scenarios and data changes, enhancing its robustness.

[0103] Figure 3 A flowchart illustrating the vehicle-cloud collaborative perception and defense method provided in this application embodiment. Figure 2 ,like Figure 3 As shown, in this embodiment... Figure 2 Based on the embodiments, the vehicle-cloud collaborative perception defense method is described in detail, which includes:

[0104] In one possible implementation, step S202 may further include:

[0105] S2021. Based on the local prediction probability distribution, obtain the information entropy term corresponding to the local prediction probability distribution.

[0106] The information entropy term measures the degree of uncertainty of random variables in a local prediction probability distribution.

[0107] In vehicle-cloud collaborative perception and defense scenarios, when a malicious vehicle client uploads a "poisoning" knowledge signal, the local prediction probability distribution of the vehicle client often exhibits abnormal uncertainty. For example, a malicious vehicle client might intentionally create confusing prediction results, making the prediction probability distribution of the vehicle client more dispersed and the information entropy value higher. By employing a specific method for determining the information entropy term, it is calculated from the local prediction probability distribution to obtain the information entropy term. A higher information entropy term indicates higher uncertainty in local prediction, meaning there is a higher probability that the vehicle client has uploaded a "poisoning" knowledge signal.

[0108] Optionally, if the local prediction probability distribution includes multiple possible outcomes, the information entropy term can express the degree of uncertainty of each possible outcome of the local prediction probability distribution.

[0109] In technical problems involving prediction and decision-making, accurately assessing the uncertainty of predictions is crucial. Information entropy, as an indicator of the uncertainty of the local prediction probability distribution, can assist in making rational decisions in the face of complex and uncertain environments, thereby improving the timeliness and accuracy of attack detection and proactive defense responses.

[0110] S2022. Based on the local prediction probability distribution and the global knowledge prior distribution, obtain the divergence term corresponding to the local prediction probability distribution.

[0111] The divergence term measures the degree of difference between the local predicted probability distribution and the global prior knowledge distribution. Normally, the local predicted probability distribution of a legitimate vehicle client should have a certain similarity to the global prior knowledge distribution, with a small divergence value between the two. However, malicious vehicle clients uploading "poisoning" knowledge signals will cause a significant difference between the local predicted probability distribution and the global prior knowledge distribution, resulting in a larger divergence value. The divergence term can help identify malicious vehicle clients, improving the attack detection and proactive defense response capabilities of vehicle clients.

[0112] By employing a specific divergence term determination method, the difference between the local predicted probability distribution and the global prior knowledge distribution is measured, and the divergence term corresponding to the local predicted probability distribution is obtained. This can quantify the uniqueness and consistency of the global prior knowledge distribution. The divergence term can be used to determine whether the local predicted probability distribution deviates from the general perception of the global knowledge, thereby allowing for reasonable adjustments and optimizations to the local model.

[0113] S2023. Add the product of the information entropy term and the entropy weight to the divergence term to obtain the variational free energy.

[0114] Multiplying the information entropy term by a pre-defined entropy weight yields a weighted information entropy term, which highlights the impact of the uncertainty of the local prediction probability distribution on the variational free energy calculation. Then, the weighted information entropy term is added to the divergence term to combine the information entropy term reflecting the uncertainty of the local prediction probability distribution with the divergence term reflecting the consistency between the local prediction probability distribution and the global knowledge prior distribution, resulting in the final variational free energy. The variational free energy allows for coordination and optimization with the global knowledge prior distribution while considering the local prediction probability distribution, improving overall performance and decision accuracy. The entropy weight is an arbitrary value set according to the application scenario of vehicle-cloud collaborative perception defense. If a greater sensitivity to the uncertainty of the local prediction probability distribution is desired, the entropy weight can be appropriately increased to more easily identify malicious vehicle clients.

[0115] Variational free energy (VFE) can comprehensively evaluate vehicle clients from both uncertainty and consistency dimensions, enabling more accurate assessment of their trustworthiness and effective identification of malicious "poisoning" knowledge signals uploaded by such clients. By promptly identifying and eliminating these malicious "poisoning" signals, VFE prevents negative impacts on the updating of the global prior knowledge distribution, avoiding interference from malicious signals that could lead to the global model learning incorrect knowledge, thus ensuring the stability and accuracy of the global model's performance.

[0116] In one possible implementation, the local predicted probability distribution includes multiple categories and a probability value corresponding to each category, and step S2021 may further include:

[0117] Step A: Sequentially select each category in the local predicted probability distribution as the target category.

[0118] From the multiple categories included in the local predicted probability distribution, each category is identified as the target category in a specific order to ensure that no category is missed. This specific order can be from the first category to the last.

[0119] For example, if the local prediction probability distribution contains categories A, B, and C, category A is first selected as the target category for subsequent related calculations. After processing category A, category B is selected as the target category, and finally category C is selected as the target category.

[0120] Step B: The product of the probability value corresponding to the target category and the natural logarithm of the probability value corresponding to the target category is used as the information entropy of the target category.

[0121] Information entropy is used as an indicator to measure the uncertainty of a target category.

[0122] First, based on the local predicted probability distribution, the probability value corresponding to the target category is obtained, thus obtaining the target probability value. Then, a logarithmic operation with the natural constant e is performed on the target probability value to obtain the natural logarithmic value. Next, the target probability value is multiplied by its natural logarithmic value to obtain the information entropy of the target category. The larger the target probability value, the lower its relative uncertainty. The combination of natural logarithmic and multiplication operations ensures that the information entropy can reasonably reflect the uncertainty of the target category.

[0123] Step C: Accumulate the information entropy and take the opposite number to obtain the information entropy term corresponding to the local prediction probability distribution.

[0124] After obtaining the information entropy for each target category, the information entropies of all categories are summed to obtain a total information entropy. Then, the negative value of the total information entropy is taken to obtain the information entropy term corresponding to the local predicted probability distribution. By summing and taking the negative value, the information entropies of multiple categories are integrated into a single overall information entropy term, which comprehensively reflects the uncertainty of the local predicted probability distribution. Summing ensures that the uncertainty of all categories is taken into account, while taking the negative value gives the value of the information entropy term a more intuitive physical meaning; a larger value indicates higher uncertainty. During attack detection and proactive defense response, the defense strategy can be adjusted based on the magnitude of the information entropy term to cope with different environmental changes.

[0125] For example, if the local prediction probability distribution includes three categories, and the information entropy of each category is -0.3, -0.4 and -0.2 respectively, then the sum of information entropy is -0.9. Taking the negative of the sum of information entropy gives 0.9, that is, the information entropy term corresponding to the local prediction probability distribution is 0.9.

[0126] In one possible implementation, the local prediction probability distribution includes multiple categories and a probability value corresponding to each category, and the global knowledge prior distribution includes multiple categories identical to the local prediction probability distribution and a prior value corresponding to each category. Step S2022 may further include:

[0127] Step A: Sequentially select each category in the local predicted probability distribution as the target category.

[0128] Similar to the above implementation method, each category is identified as the target category in a certain order from the multiple categories included in the local predicted probability distribution to ensure that no category is missed.

[0129] Step B: Divide the probability value corresponding to the target category by the prior value corresponding to the target category to obtain the first value of the target category.

[0130] The prior value corresponding to the target category is the prior value corresponding to the target category in the global knowledge prior distribution.

[0131] First, the probability value corresponding to the target category is obtained from the local predicted probability distribution, and the prior value corresponding to the target category is obtained from the global knowledge prior distribution. Then, the probability value corresponding to the target category is divided by the prior value corresponding to the target category; the result is the first value for that target category. This first value is obtained by associating and comparing the probability values ​​divided by the prior values, reflecting the degree of difference between the probability values ​​and prior values ​​of the target category.

[0132] Step C: The product of the probability value of the target class and the natural logarithm of the first value is used as the first divergence of the target class.

[0133] Perform a logarithmic operation to base e on the first value to obtain its natural logarithm. Then, multiply the probability value of the target class by the natural logarithm of the first value to obtain the first divergence of the target class, which quantifies the difference between the local predicted probability distribution and the global prior knowledge distribution for the target class. By multiplying the probability value of the target class by the natural logarithm of the first value, we can consider the degree of difference between the probability value of the target class in the local probability prediction distribution and the prior value of the target class in the global prior knowledge distribution, and analyze the deviation between the local predicted probability distribution and the global prior knowledge distribution of the local model in more detail.

[0134] For example, if the probability value of the target category is 0.4 and the first value is 2, the first value is logarithmically calculated to base e to obtain the natural logarithm of the first value as 0.693; then, the probability value of the target category is multiplied by the natural logarithm of the first value to obtain the first divergence as 0.2772.

[0135] Step D: Accumulate the first divergence to obtain the divergence term corresponding to the local predicted probability distribution.

[0136] By treating each category in the local prediction probability distribution as the target category and calculating the first divergence of the target category, we can obtain the first divergence for multiple categories. By summing the first divergences for multiple categories, we obtain the total divergence. Using this total divergence as the divergence term for the local prediction probability distribution integrates the first divergences of each target category into a single overall divergence term, thus comprehensively reflecting the difference between the local prediction probability distribution and the global prior knowledge distribution.

[0137] In one possible implementation, the local prediction probability distribution includes an attack probability distribution and a defense strategy probability distribution, and the composite loss is obtained through the following methods:

[0138] Step A: Obtain the distillation loss based on the local and global prediction probability distributions.

[0139] Distillation loss is a loss function used to measure the difference between the local predicted probability distribution and the global predicted probability distribution.

[0140] Using a specific method to determine distillation loss, the local and global predicted probability distributions are used as inputs. The difference between the local and global predicted probability distributions is calculated, ultimately yielding the distillation loss. The distillation loss can be determined using a distillation loss function. Optionally, the distillation loss function can be a cross-entropy loss function.

[0141] By calculating the distillation loss, the difference between the local and global predicted probability distributions is quantified. When optimizing the parameters of the local model, considering the distillation loss allows the use of information from the global predicted probability distribution to optimize the local model. This helps address the problem of inaccurate predictions caused by limited data or computational limitations in the local model, improving the accuracy of the local model in predicting attack and defense strategies, and enhancing the local model's attack detection and proactive defense response capabilities.

[0142] Step B: Obtain the inference loss based on the negative log-likelihood value of the attack probability distribution.

[0143] The attack probability distribution represents the probability of various attack behaviors occurring. Inference loss is a loss value obtained based on the negative log-likelihood of the attack probability distribution, used to measure the model's performance in inferring attack behaviors. The negative log-likelihood measures the degree of match between the probability distribution and actual observations. The smaller the negative log-likelihood value, the smaller the inference loss, indicating a higher accuracy of the attack probability distribution.

[0144] First, the actual observed attack behaviors are identified, resulting in observed events. Then, the probability of each observed event occurring is determined based on the attack probability distribution, yielding the attack event probability. Next, the negative logarithm of the attack event probability is taken to obtain the negative log-likelihood of the attack probability distribution. This negative log-likelihood is used as the inference loss. By calculating the inference loss, the accuracy of the local model's inference about attack behaviors can be intuitively measured.

[0145] Optionally, in scenarios where a local model is trained using a training dataset, the actual observed event is the attack label corresponding to the current training data.

[0146] Optionally, in practical applications of vehicle-cloud collaborative perception defense, the actual observed events are attack tags obtained based on the global predicted probability distribution. For example, the attack tag with the highest probability value corresponding to the attack probability distribution in the global predicted probability distribution is taken as the actual observed event.

[0147] Optionally, if the attack probability distribution generated by the local model can be used to generate a textual description of the attack behavior, the attack probability distribution includes the probability of the occurrence of terms related to the attack behavior in the output textual description. The actual observed events represent the terminological description of the observed attack behavior. In this case, the negative log-likelihood value of the attack probability distribution is used to represent the error between the terms in the attack probability distribution and the terms in the actual observed events.

[0148] Optionally, if the attack probability distribution generated by the local model can be used to generate a textual description of the attack behavior, then the attack probability distribution includes the probability of the occurrence of attack-related terms in the output textual description. Actual observed events, on the other hand, represent the terminological description of the attack behavior observed through actual monitoring, analysis, and other means. Actual observed events reflect the textual characteristics of real-world attack behavior. Terminology includes at least one of words, phrases, and / or specific symbols. Attack-related terms collectively constitute the textual content describing the attack behavior.

[0149] For example, in a scenario where a local model is trained using a training dataset, the actual observed events are the word sequences corresponding to the current training data. Each word in the word sequence is taken as a target word. The inference loss is obtained by calculating the negative log-likelihood of the probability of the target word appearing in the attack probability distribution, effectively measuring the error between the words in the attack probability distribution and the target words in the actual observed events. Specifically, the smaller the negative log-likelihood value, the closer the words in the attack probability distribution generated by the model are to the target words of the actual observed attack behavior; that is, the more accurate the textual description of the attack behavior by the local model. Conversely, the larger the negative log-likelihood value, the greater the error between the two.

[0150] Step C: Obtain the defense loss based on the negative log-likelihood value of the probability distribution of the defense strategy.

[0151] The defense strategy probability distribution represents the probability that various defense strategies will be adopted. The defense loss is a loss value obtained based on the negative log-likelihood of the defense strategy probability distribution, used to measure the model's performance in selecting defense strategies.

[0152] First, the probability distribution of defense strategies is obtained to determine the actual defense strategy to be adopted. This actual defense strategy is determined based on multi-source sensor data and represents the defensive measures that should be taken in the current environment. Then, the probability of the actual defense strategy being selected is calculated based on the probability distribution, yielding the actual defense probability. Next, the negative logarithm of the actual defense probability is taken to obtain the negative log-likelihood value of the defense strategy probability distribution. This negative log-likelihood value is used as the defense loss. A smaller defense loss indicates a closer match between the local model's predicted defense strategy probability distribution and the actual defense strategy adopted. By analyzing the defense loss, it is helpful to identify deficiencies in the local model's defense strategy selection, thereby optimizing the local model to select defense strategies more rationally and improve the system's defense capabilities.

[0153] Optionally, in scenarios where a local model is trained using a training dataset, the actual defense strategy is the defense strategy label corresponding to the current training data.

[0154] Optionally, in practical application scenarios of vehicle-cloud collaborative perception defense, the actual defense strategy is a defense strategy label obtained based on the global predicted probability distribution. For example, the defense strategy label with the highest probability value corresponding to the defense strategy probability distribution in the global predicted probability distribution is taken as the actual defense strategy.

[0155] Optionally, if the defense strategy probability distribution generated by the local model can be used to generate a textual description of the defense strategy behavior, the defense strategy probability distribution includes the probability of the occurrence of terms related to the defense strategy behavior in the output textual description. The actual defense strategy represents the terminological description of the observed defense strategy behavior. In this case, the negative log-likelihood value of the defense strategy probability distribution is used to represent the error between the terms in the defense strategy probability distribution and the terms in the actual defense strategy.

[0156] Optionally, if the probability distribution of defense strategies generated by the local model can be used to generate a textual description of defense strategy behavior, then the probability distribution of defense strategies includes the probability of occurrence of terms related to defense strategy behavior in the output textual description. The actual defense strategy, on the other hand, represents the terminological description of defense strategy behavior observed through actual monitoring, analysis, and other means. The actual defense strategy reflects the textual characteristics of the actual defense strategy behavior. Terms include at least one of words, phrases, and / or specific symbols. The terms related to defense strategy behavior collectively constitute the textual content describing the defense strategy behavior.

[0157] For example, in a scenario where a local model is trained using a training dataset, the actual defense strategy is the word sequence corresponding to the current training data. Each word in the word sequence is taken as the target word. The defense loss is obtained by calculating the negative log-likelihood of the probability of the target word appearing in the defense strategy probability distribution. This effectively measures the error between the words in the defense strategy probability distribution and the target words in the actual defense strategy. Specifically, the smaller the negative log-likelihood value, the closer the words in the model-generated defense strategy probability distribution are to the target words of the actually observed defense strategy behavior; that is, the more consistent the defense strategy probability distribution predicted by the local model is with the actual defense strategy adopted. Conversely, the larger the negative log-likelihood value, the greater the error between the local model's predicted defense strategy probability distribution and the actual defense strategy adopted.

[0158] Step D: Weight the distillation loss, reasoning loss, and defense loss to obtain the composite loss.

[0159] First, based on the vehicle-cloud collaborative perception and defense scenario, the distillation loss weight, inference loss weight, and defense loss weight are determined. These weights can be set according to actual needs and the impact of the corresponding losses on the overall performance of the local model.

[0160] Then, the distillation loss is multiplied by its weight to obtain the weighted distillation loss. The inference loss is multiplied by its weight to obtain the weighted inference loss. The defense loss is multiplied by its weight to obtain the weighted defense loss. Finally, the weighted distillation loss, weighted inference loss, and weighted defense loss are summed to obtain the composite loss. By using a weighted summation of the distillation loss, inference loss, and defense loss to obtain the composite loss, the one-sided influence of a single loss value on the local model optimization can be avoided. This more comprehensively reflects the model's performance in attack detection and proactive defense response, guides the local model to perform more effective optimization, improves the model's overall performance in complex scenarios, and better addresses the challenges of attack detection and proactive defense response.

[0161] In one possible implementation, the distillation loss is obtained based on the local prediction probability distribution, the global prediction probability distribution, and the global logistic value, including:

[0162] Step 1: Obtain the classification cross-entropy by measuring the difference between the local predicted probability distribution and the true label.

[0163] Optionally, the ground truth labels are predefined classification labels. During local model updates, the client uses local multi-source sensor data labeled with classification labels for supervised training. Classification cross-entropy is the error between the local predicted distribution and the ground truth labels corresponding to the classification labels. Optionally, the ground truth labels include at least one of the following classification labels: normal, adversarial patch, sensor spoofing, physical attack, and data poisoning.

[0164] First, the true labels are converted into a form suitable for calculating the classification cross-entropy, resulting in true label values. Then, according to the formula for calculating classification cross-entropy, the classification cross-entropy between the local predicted probability distribution and the true label values ​​is calculated. For example, if there are three types of true labels, and the true label corresponding to the classification label is the second type, then the true label value encoding can be represented by [0, 1, 0].

[0165] Classification cross-entropy can accurately measure the difference between the local predicted probability distribution and the true label. The greater the difference, the larger the classification cross-entropy value, indicating that the local model's prediction is less accurate.

[0166] Step 2: Using the distillation temperature parameter, calculate the second divergence between the local predicted probability distribution and the global predicted probability distribution.

[0167] The distillation temperature parameter is a hyperparameter introduced during the knowledge distillation process to control the smoothness of the probability distribution. Specifically, the distillation temperature parameter is set according to the vehicle-cloud collaborative perception and defense scenario. A higher distillation temperature parameter makes the second divergence smoother, highlighting the relative relationship between the local and global prediction probability distributions; a lower distillation temperature parameter makes the second divergence sharper.

[0168] The local and global predicted probability distributions represent the probability distributions obtained by the local and global models for the same task, respectively. The second divergence measures the difference between the local and global predicted probability distributions, providing a more nuanced reflection of their differences. Optionally, the second divergence is the Kullback-Leibler (KL) divergence.

[0169] First, based on the set distillation temperature parameters, the local and global prediction probability distributions are softened respectively, resulting in softened local and global prediction probability distributions. Then, according to the divergence determination formula, a second divergence is calculated between the softened local and global prediction probability distributions. This second divergence allows the local model to not only focus on the local data characteristics of multi-source sensor data and ground truth labels during training, but also learn the generalization ability of the global prior knowledge distribution, thereby improving the overall performance of the local model.

[0170] Step 3: Weight the classification cross-entropy and the second divergence to obtain the distillation loss.

[0171] First, the weights of the classification cross-entropy and the second divergence are determined. Based on the requirements of attack detection and proactive defense response, the classification cross-entropy weight and the second divergence weight are set. Then, the classification cross-entropy is multiplied by its weight to obtain the weighted classification cross-entropy, and the second divergence is multiplied by its weight to obtain the weighted second divergence. Finally, the weighted classification cross-entropy and the weighted second divergence are added together to obtain the distillation loss. By obtaining the distillation loss through the weighted summation of the classification cross-entropy and the second divergence, the performance of the local model in terms of classification accuracy and global knowledge learning can be comprehensively considered. This allows the local model to achieve a better balance during training, effectively addressing the technical problem of poor performance of the local model in complex tasks.

[0172] In one possible implementation, the vehicle-cloud collaborative perception defense method further includes: using the probability value corresponding to the attack probability distribution in the local prediction probability distribution as a confidence score; if the confidence score is greater than a security threshold, then active defense is executed based on the defense strategy, which includes: adjusting the weight values ​​of the corresponding attack modality data in the feature fusion module of the local model.

[0173] The confidence score, which is a score derived by using the probability value corresponding to the attack probability distribution as a metric, is used to determine whether the vehicle client is likely to be attacked and the likelihood of such an attack.

[0174] The security threshold is a pre-set numerical limit used to compare with the confidence score to determine whether to implement active defense on the vehicle client.

[0175] First, the attack probability distribution is extracted from the local predicted probability distribution, and the corresponding probability value of the attack probability distribution is used as the confidence score. The confidence score is then compared with a pre-set security threshold. If the confidence score is less than or equal to the security threshold, it indicates that the vehicle perception system is unlikely to be attacked, and no active defense is implemented on the vehicle client side; normal attack detection and active defense response continue. If the confidence score is greater than the security threshold, it indicates that the vehicle client is very likely under attack, and active defense on the vehicle client side needs to be implemented.

[0176] Furthermore, based on the defense strategy, proactive defense is implemented. The defense strategy is used to defend against attacks that the vehicle client may suffer and to prevent attacked vehicle clients from negatively impacting the cloud server. Optionally, the defense strategy includes adjusting the weight values ​​of the corresponding attack modality data in the feature fusion module of the local model. For example, if the confidence score is greater than a security threshold, the weight values ​​of the corresponding attack modality data in the feature fusion module of the local model are adjusted. By reducing the weight of the attack modality data, the impact of the attack data is effectively weakened, reducing the impact of the attack modality data on the overall perception result, thereby improving the vehicle client's ability to defend against attacks.

[0177] Optionally, the defense strategy includes the vehicle client initiating preset security control commands. These security control commands are used to defend against potential attacks and prevent attacked vehicle clients from negatively impacting the cloud server.

[0178] In one possible implementation, after step S202, the method further includes: if the variational free energy exceeds a preset threshold, triggering an active defense response to the vehicle client, wherein the active defense response includes: discarding the predicted logical value and the local predicted probability distribution corresponding to the predicted logical value, and / or triggering an anomaly alarm.

[0179] The preset threshold is a pre-defined maximum value for the variational free energy under normal conditions, based on the vehicle-cloud collaborative perception and defense scenario. If the variational free energy exceeds the preset threshold, it indicates that the variational free energy is beyond the acceptable range, and there may be an anomaly in the vehicle client.

[0180] The variational free energy is compared with a preset threshold. If the variational free energy is less than or equal to the preset threshold, it indicates that the variational free energy is within an acceptable range, and the vehicle client continues to perform attack detection and proactive defense response normally without triggering a proactive defense response. If the variational free energy exceeds the preset threshold, it indicates that the variational free energy exceeds the acceptable range, and the vehicle client may be experiencing abnormalities or being attacked, triggering a proactive defense response for the vehicle client. Specifically, the proactive defense response includes: discarding the predicted logical value and the corresponding local predicted probability distribution, and / or triggering an anomaly alarm.

[0181] By discarding the predicted logical value and its corresponding local predicted probability distribution, the misleading influence of anomalous data on model optimization can be reduced. Triggering anomaly alerts can then be sent to relevant personnel and / or the system, enabling them to promptly understand the anomaly and take appropriate action.

[0182] By monitoring variational free energy and triggering proactive defense responses in a timely manner, it is possible to effectively identify and respond to anomalies or potential attacks in the system, prevent abnormal data from causing further impact on the system, and ensure the safe and stable operation of the vehicle client and the entire vehicle-cloud collaborative perception system.

[0183] Figure 4 A flowchart illustrating the vehicle-cloud collaborative perception and defense method provided in this application embodiment. Figure 3 ,like Figure 4 As shown, this method is applied to a cloud server. The vehicle-cloud collaborative perception and defense method includes:

[0184] S401. Receive variational free energy and prediction logic value sent by multiple vehicle clients respectively. The variational free energy is used to indicate the confidence level of the prediction logic value.

[0185] A cloud server is a server with computing and storage capabilities that can receive variational free energy and predicted logic values ​​from multiple vehicle clients, centrally process and analyze these values, and provide collaborative perception support for vehicles.

[0186] Multiple vehicle clients send variational free energy and predicted logical values ​​to the cloud server. The cloud server then receives the variational free energy and predicted logical values ​​from each vehicle client. Optionally, the cloud server continuously monitors the communication channel between itself and the multiple vehicle clients; once it receives the variational free energy and predicted logical values ​​from a vehicle client, it stores them in the corresponding database.

[0187] By receiving variational free energy and predicted logical values ​​from multiple vehicle clients via a cloud server, centralized data management is achieved. This avoids the inaccuracy of analysis caused by limited data from a single vehicle client, and provides a comprehensive understanding of the defense status of multiple vehicle clients. It also reduces the risk of inaccurate variational free energy and predicted logical values ​​due to local interference or attacks on a single vehicle client.

[0188] S402. Based on the aggregated temperature parameter, perform an exponential calculation on the variational free energy of each vehicle client to obtain the credibility score of each vehicle client.

[0189] The aggregation temperature parameter is used to control the weight allocation during the data aggregation process, determining the degree of emphasis on different data within the variational free energy when calculating the credibility score.

[0190] Based on the pre-set aggregation temperature parameters, the variational free energy of each vehicle client is exponentially calculated to obtain the credibility score of each vehicle client.

[0191] By converting variational free energy into a confidence score through exponential operations, a quantitative representation of the confidence level of the predicted logical value for vehicle clients is achieved, making the confidence levels of data from different vehicles comparable. Simultaneously, the aggregated temperature parameter can be adjusted according to actual conditions, thereby flexibly controlling the influence of different variational free energies on the confidence score to adapt to different traffic scenarios and data characteristics.

[0192] S403. Based on the credibility score corresponding to each vehicle client, perform global knowledge weighted aggregation on the predicted logical values ​​of multiple vehicle clients to obtain the global logical value and the global predicted probability distribution of the global logical value.

[0193] Based on the credibility score, corresponding weights are assigned to the predicted logical values ​​of each vehicle client. By considering the credibility score, the impact of unreliable data in the vehicle client is reduced, and the diversity and complementarity of data are fully utilized to solve the problems of inaccurate and one-sided data from individual vehicle clients, thereby improving the overall perception capability of the traffic environment. Subsequently, the predicted logical values ​​of the vehicle clients are globally weighted and aggregated using the weights assigned to the vehicle clients to obtain global logical values. This reduces the impact of unreliable data within the vehicle clients on the global logical values, thus identifying and suppressing the impact of "poisoning" attacks by vehicle clients, and improving the security and stability of vehicle-cloud collaborative perception defense.

[0194] Furthermore, by performing indexation and normalization on the global logic values, the global prediction probability distribution of the global logic values ​​is obtained.

[0195] S404. Send global logical values ​​and global prediction probability distributions to multiple vehicle clients. Both global logical values ​​and global prediction probability distributions are used for parameter optimization of the local models deployed in multiple vehicle clients.

[0196] After the cloud server completes the calculation of the global logical value and the global predicted probability distribution, it packages the global logical value and the global predicted probability distribution and sends them to multiple vehicle clients. This allows the vehicle clients to use the global logical value and the global predicted probability distribution to optimize the parameters of the local model, thereby continuously improving the performance of the local model, enabling the local model to better adapt to complex traffic environments, and improving perception accuracy.

[0197] Figure 5 A flowchart illustrating the vehicle-cloud collaborative perception and defense method provided in this application embodiment. Figure 4 ,like Figure 5 As shown, in this embodiment... Figure 4 Based on the embodiments, the vehicle-cloud collaborative perception defense method is described in detail, which includes:

[0198] In one possible implementation, step S402 may further include:

[0199] S4021. Perform a natural exponential operation on the quotient of the negative of the variational free energy of each vehicle client divided by the aggregation temperature parameter to obtain the exponential energy of each vehicle client.

[0200] For each vehicle client, the inverse of its variational free energy is calculated, i.e., the variational free energy is negativeed to obtain the inverted variational free energy. Then, the inverted variational free energy is divided by the polymerization temperature parameter to obtain the target quotient. Finally, the target quotient is exponentially calculated with the natural constant e as the base to obtain the exponentialized energy of each vehicle client.

[0201] S4022. Accumulate the exponential energy to obtain the normalization factor.

[0202] The indexed energies of multiple vehicle clients are summed to obtain a normalization factor, which allows the indexed energies of different vehicle clients to be compared and analyzed under a unified normalization factor, thus achieving the function of normalization processing.

[0203] S4023. Divide the indexed energy of each vehicle client by the normalization factor to obtain the credibility score of each vehicle client.

[0204] Dividing the exponential energy corresponding to each vehicle client by the normalization factor yields the vehicle client's credibility score. Converting the exponential energy of each vehicle client into a relative proportional credibility score allows for comparison and analysis of the credibility of different vehicle clients within the range of 0 to 1, clearly demonstrating the relative credibility of each vehicle client within the entire system.

[0205] In one possible implementation, step S403 may further include:

[0206] S4031. Multiply the credibility score corresponding to each vehicle client by the predicted logical value of the vehicle client to obtain the aggregate logical value corresponding to each vehicle client.

[0207] The aggregated logical value for each vehicle client is obtained by multiplying its credibility score by the predicted logical value. By multiplying the credibility score by the predicted logical value, the aggregated logical value includes not only the local predicted probability distribution but also the credibility of that local predicted probability distribution. This results in a greater weight being given to the local predicted probability distribution of vehicle clients with high credibility scores, while the local predicted probability distribution of vehicle clients with low credibility scores has a relatively smaller impact on the overall value. This approach more accurately reflects the contribution of each vehicle client's prediction results to the global logical value.

[0208] S4032. Aggregate the aggregated logical values ​​of multiple vehicle clients to obtain a global logical value.

[0209] A global logical value is obtained by aggregating the aggregated logical values ​​from multiple vehicle clients. This aggregation can be achieved by summing the aggregated logical values ​​and then normalizing them according to certain rules; or by directly applying a weighted average to the aggregated logical values. For example, using a simple weighted average, the aggregated logical values ​​from multiple vehicle clients are directly added together and then divided by the number of vehicle clients to obtain the global logical value. By aggregating the aggregated logical values ​​from multiple vehicle clients to obtain the global logical value, information from multiple vehicle clients can be effectively integrated to provide an overall prediction and judgment for a specific event.

[0210] In one possible implementation, the global knowledge prior distribution is updated based on the global logical value, and the updated global knowledge prior distribution is sent to multiple vehicle clients respectively.

[0211] Based on the global logical value, and combined with a certain probability model and algorithm, the existing global knowledge prior distribution is updated to obtain the updated global knowledge prior distribution. Then, the updated global knowledge prior distribution is sent to multiple vehicle clients.

[0212] Optionally, the global logical value can be directly used as the updated global knowledge prior distribution.

[0213] Optionally, the weighted average of the global logical value and the global knowledge prior distribution can be used as the updated global knowledge prior distribution. Specifically, the global logical value is multiplied by a first weighting value to obtain a weighted global logical value; the global knowledge prior distribution is multiplied by a second weighting value to obtain a weighted global knowledge prior distribution. Then, the average of the weighted global logical value and the weighted global knowledge prior distribution is taken to obtain the updated global knowledge prior distribution.

[0214] In one possible implementation, updating the global knowledge prior distribution based on the global logical value includes: performing indexation and normalization on the global logical value to obtain a global prediction probability distribution; and updating the global knowledge prior distribution based on the global prediction probability distribution to obtain an updated global knowledge prior distribution.

[0215] The global logical values ​​are exponentialized and normalized to obtain the global prediction probability distribution. Then, based on the global prediction probability distribution, the global knowledge prior distribution is updated to obtain the updated global knowledge prior distribution.

[0216] Optionally, the global predicted probability distribution can be directly used as the updated global knowledge prior distribution.

[0217] Optionally, the weighted average of the global prediction probability distribution and the global knowledge prior distribution is used as the updated global knowledge prior distribution. Specifically, the global prediction probability distribution is multiplied by a first weighting value to obtain a weighted global prediction probability distribution; the global knowledge prior distribution is multiplied by a second weighting value to obtain a weighted global knowledge prior distribution. Then, the average of the weighted global prediction probability distribution and the weighted global knowledge prior distribution is taken to obtain the updated global knowledge prior distribution.

[0218] This application provides a vehicle client that can execute the vehicle-cloud collaborative perception and defense method provided in the above method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0219] This application provides a cloud server that can execute the vehicle-cloud collaborative perception and defense method provided in the above method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0220] Figure 6 This is a schematic diagram of the system structure of the vehicle-cloud collaborative perception and defense system provided in an embodiment of this application. Figure 6 As shown, the vehicle-cloud collaborative perception and defense system includes: a cloud server 12 as described in the above embodiments or any possible implementation of the above embodiments, and at least one vehicle client 13 as described in the above embodiments or any possible implementation of the above embodiments.

[0221] To facilitate understanding of the vehicle-cloud collaborative perception and defense system, this application also provides an initialization example of the vehicle-cloud collaborative perception and defense system, including:

[0222] First, the cloud server is initialized, building a collection of vehicle clients to manage and identify each client communicating with the cloud server. Then, the edge cloud server initializes the global knowledge model deployed in the cloud. During the initialization phase of the vehicle-cloud collaborative perception and defense system, local models are deployed on the vehicle clients. These local models extract features and perform inference from multi-source sensor data. By deploying local models on the vehicle clients, they gain the ability to perceive and process multi-source sensor data.

[0223] Once the cloud server completes the construction of the vehicle client set and initializes the global knowledge model, and the vehicle clients complete the deployment of their local models, the initialization process of the vehicle-cloud collaborative perception and defense system ends, and the system enters a runnable state. Afterward, the vehicle-cloud collaborative perception and defense system can be used to implement the above embodiments or any possible implementation of the above embodiments.

[0224] The vehicle-cloud collaborative perception and defense system provided in this application has at least the following advantages when used to implement the above embodiments or any possible implementation of the above embodiments:

[0225] (1) By actively inferring the variational free energy from the vehicle client and the global knowledge weighted aggregation based on the variational free energy of the cloud server, the vehicle-cloud collaborative perception defense system can automatically identify and suppress the poisoning knowledge signals uploaded from malicious or intruded vehicle clients, significantly reducing the destructive impact of abnormal vehicle clients on the global knowledge model in the cloud server, thereby training a more robust and secure global knowledge model and improving the resilience of the entire Internet of Vehicles in adversarial environments.

[0226] (2) The global knowledge weighted aggregation based on variational free energy of the cloud server does not rely on fixed thresholds or static rules. The weight allocation of the global knowledge weighted aggregation of the cloud server can be dynamically adjusted according to the specific performance of the variational free energy and predicted logic value uploaded by each vehicle client. This enables the vehicle-cloud collaborative perception defense system to effectively cope with the complex scenarios in the Internet of Vehicles, such as highly non-independent and identically distributed data, time-varying vehicle client status, and dynamic evolution of attack modes.

[0227] (3) By transmitting only variational free energy and predicted logic values ​​rather than multi-source sensor data through federated distillation, the high efficiency of communication is enhanced, and the communication overhead between the vehicle and the cloud is significantly reduced. At the same time, since the entire global knowledge weighted aggregation is performed only on the knowledge uploaded by the vehicle client on the cloud server side, without accessing multi-source sensor data, the data privacy protection capability of the federated learning framework is fully maintained.

[0228] (4) The vehicle-cloud collaborative perception and defense system links the abnormal local prediction probability distribution of the vehicle client with the security response layer through the triggering mechanism, so as to realize end-to-end security enhancement from threat perception, assessment, model update to active protection, and provides a system-level solution for intelligent connected vehicles.

[0229] This application provides a vehicle in which a vehicle client is deployed as described in the above embodiments or any possible implementation of the above embodiments.

[0230] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 70 provided in this embodiment includes at least one processor 701 and a memory 702. Optionally, the electronic device 70 further includes a communication component 703. The processor 701, memory 702, and communication component 703 are connected via a bus 704.

[0231] In a specific implementation, at least one processor 701 executes computer execution instructions stored in memory 702, causing at least one processor 701 to perform the above-described method.

[0232] The specific implementation process of processor 701 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0233] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0234] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0235] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0236] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0237] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0238] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0239] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0240] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0241] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0242] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0243] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0244] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0245] Finally, it should be noted that other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and alterations may be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A vehicle-cloud collaborative perception defense method, characterized in that, Applied to a vehicle client, on which a local model is deployed for attack detection and proactive defense response, the vehicle-cloud collaborative perception and defense method includes: The local model is used to extract features and infer from multi-source sensor data to obtain the predicted logical value and the local prediction probability distribution corresponding to the predicted logical value. Based on the global knowledge prior distribution received from the cloud server and the local prediction probability distribution, the variational free energy is calculated, and the variational free energy is used to indicate the confidence level of the prediction logic value. The variational free energy and the predicted logical value are sent to the cloud server, and the global logical value and global predicted probability distribution returned by the cloud server are received. The global logical value is obtained by the cloud server through global knowledge weighted aggregation of the predicted logical value based on the variational free energy, and the global predicted probability distribution is the probability distribution of the global logical value. With the goal of minimizing the composite loss, the parameters of the local model are optimized. The composite loss is obtained based on the global logistic value, the global prediction probability distribution, and the local prediction probability distribution.

2. The vehicle-cloud collaborative perception and defense method according to claim 1, characterized in that, The variational free energy is calculated based on the global prior knowledge distribution received from the cloud server and the local prediction probability distribution, including: Based on the local prediction probability distribution, the information entropy term corresponding to the local prediction probability distribution is obtained; Based on the local prediction probability distribution and the global knowledge prior distribution, the divergence term corresponding to the local prediction probability distribution is obtained; The variational free energy is obtained by adding the product of the information entropy term and the entropy weight to the divergence term.

3. The vehicle-cloud collaborative perception and defense method according to claim 2, characterized in that, The local prediction probability distribution includes multiple categories and a probability value corresponding to each category; the step of obtaining the information entropy term corresponding to the local prediction probability distribution based on the local prediction probability distribution includes: Each category in the local predicted probability distribution is sequentially taken as the target category; The product of the probability value corresponding to the target category and the natural logarithm of the probability value corresponding to the target category is taken as the information entropy of the target category; The information entropy is accumulated and its negative value is taken to obtain the information entropy term corresponding to the local prediction probability distribution.

4. The vehicle-cloud collaborative perception and defense method according to claim 2, characterized in that, The local prediction probability distribution includes multiple categories and a probability value corresponding to each category. The global knowledge prior distribution includes multiple categories identical to the local prediction probability distribution and a prior value corresponding to each category. Based on the local prediction probability distribution and the global knowledge prior distribution, the divergence term corresponding to the local prediction probability distribution is obtained, including: Each category in the local predicted probability distribution is sequentially taken as the target category; The first value of the target category is obtained by dividing the probability value corresponding to the target category by the prior value corresponding to the target category. The product of the probability value of the target category and the natural logarithm of the first value is used as the first divergence of the target category; The first divergence is accumulated to obtain the divergence term corresponding to the local prediction probability distribution.

5. The vehicle-cloud collaborative perception and defense method according to any one of claims 1-4, characterized in that, The local prediction probability distribution includes an attack probability distribution and a defense strategy probability distribution. The composite loss is obtained through the following methods: The distillation loss is obtained based on the local prediction probability distribution and the global prediction probability distribution; The inference loss is obtained based on the negative log-likelihood value of the attack probability distribution. The defense loss is obtained based on the negative log-likelihood value of the probability distribution of the defense strategy. The composite loss is obtained by weighted summation of the distillation loss, the inference loss, and the defense loss.

6. The vehicle-cloud collaborative perception and defense method according to claim 5, characterized in that, The process of obtaining the distillation loss based on the local prediction probability distribution and the global prediction probability distribution includes: The classification cross-entropy is obtained by measuring the difference between the local predicted probability distribution and the true label; Using the distillation temperature parameter, calculate the second divergence between the local predicted probability distribution and the global predicted probability distribution; The distillation loss is obtained by weighting and summing the classification cross-entropy with the second divergence.

7. The vehicle-cloud collaborative perception and defense method according to any one of claims 1-4, characterized in that, Also includes: The probability value corresponding to the attack probability distribution in the local predicted probability distribution is used as the confidence score; If the confidence score is greater than the security threshold, then active defense is executed based on the defense strategy, which includes adjusting the weight values ​​of the corresponding attack modality data in the feature fusion module of the local model.

8. The vehicle-cloud collaborative perception and defense method according to any one of claims 1-4, characterized in that, After calculating the variational free energy based on the global knowledge prior distribution received from the cloud server and the local prediction probability distribution, the method further includes: If the variational free energy exceeds a preset threshold, an active defense response is triggered on the vehicle client. The active defense response includes: discarding the predicted logical value and the local predicted probability distribution corresponding to the predicted logical value, and / or triggering an anomaly alarm.

9. A vehicle-cloud collaborative perception and defense method, characterized in that, The vehicle-cloud collaborative perception and defense method, applied to cloud servers, includes: The system receives variational free energy and predicted logic value sent by multiple vehicle clients, wherein the variational free energy is used to indicate the confidence level of the predicted logic value. Based on the aggregated temperature parameter, the variational free energy of each vehicle client is exponentially calculated to obtain the credibility score of each vehicle client. Based on the credibility score corresponding to each vehicle client, the predicted logical values ​​of the multiple vehicle clients are globally knowledge-weighted and aggregated to obtain the global logical value and the global predicted probability distribution of the global logical value. The global logical value and the global predicted probability distribution are sent to the plurality of vehicle clients. Both the global logical value and the global predicted probability distribution are used for parameter optimization of the local model deployed in the plurality of vehicle clients.

10. The vehicle-cloud collaborative perception and defense method according to claim 9, characterized in that, The process of performing an exponential calculation on the variational free energy of each vehicle client based on the aggregated temperature parameter to obtain a credibility score for each vehicle client includes: The exponential energy of each vehicle client is obtained by performing a natural exponential operation on the quotient of the inverse of the variational free energy of each vehicle client divided by the polymerization temperature parameter. The exponentialized energies are accumulated to obtain the normalization factor; The credibility score of each vehicle client is obtained by dividing the exponential energy of each vehicle client by the normalization factor.

11. The vehicle-cloud collaborative perception and defense method according to claim 9, characterized in that, The global logical value is obtained by globally weighting and aggregating the predicted logical values ​​of the multiple vehicle clients based on the credibility score corresponding to each vehicle client, including: Multiply the credibility score corresponding to each vehicle client by the predicted logical value of the vehicle client to obtain the aggregate logical value corresponding to each vehicle client. The aggregated logical values ​​of the multiple vehicle clients are aggregated to obtain a global logical value.

12. The vehicle-cloud collaborative perception and defense method according to any one of claims 9-11, characterized in that, Also includes: The global knowledge prior distribution is updated based on the global logical value, and the updated global knowledge prior distribution is sent to multiple vehicle clients respectively.

13. The vehicle-cloud collaborative perception and defense method according to claim 11, characterized in that, The step of updating the global knowledge prior distribution based on the global logical value includes: The global logical value is exponentialized and normalized to obtain the global prediction probability distribution. Based on the global prediction probability distribution, the global knowledge prior distribution is updated to obtain the updated global knowledge prior distribution.

14. A vehicle client, characterized in that, include: Perform the vehicle-cloud collaborative perception and defense method as described in any one of claims 1-8.

15. A cloud server, characterized in that, include: Perform the vehicle-cloud collaborative perception and defense method as described in any one of claims 9-13.

16. A vehicle-cloud collaborative perception and defense system, characterized in that, include: The cloud server as described in claim 15, and at least one vehicle client as described in claim 14.

17. A vehicle, characterized in that, The vehicle client as described in claim 14 is deployed.

18. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the vehicle-cloud collaborative perception and defense method as described in any one of claims 1-13.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the vehicle-cloud collaborative perception and defense method as described in any one of claims 1-13.

20. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the vehicle-cloud collaborative perception and defense method according to any one of claims 1-13.

Images