Federal visual autonomous driving-oriented data leakage risk rapid assessment method
By using a public information extractor and gradient matching optimization mechanism, environmental attributes can be rapidly reconstructed in federated vision autonomous driving, solving the problem of low evaluation efficiency in existing technologies. This enables efficient privacy risk assessment and security auditing, and is applicable to environments such as vehicle-mounted terminals and edge servers.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UESTC (SHENZHEN) ADVANCED RES INST
- Filing Date
- 2026-03-30
- Publication Date
- 2026-06-12
Smart Images
Figure CN121935927B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving technology, and in particular to a rapid data leakage risk assessment method for federal vision autonomous driving. Background Technology
[0002] With the development of autonomous driving technology, a vehicle's ability to perceive its environment has become one of the core capabilities for achieving efficient and safe driving. Among these capabilities, visual perception models, such as deep neural networks used for lane line recognition, obstacle detection, and traffic signal perception, are crucial components of autonomous driving systems. These models often rely on large amounts of high-dimensional image data from various driving environments (urban, suburban, highway, inclement weather, etc.) for training to enhance their generalization ability.
[0003] However, the raw image data involved in autonomous driving may contain a large amount of sensitive information, such as geographical location, infrastructure features, human activities, and driving trajectories. Direct sharing of this information not only poses risks to personal privacy but may also lead to data breaches and traffic safety hazards. Therefore, federated reinforcement learning technology has emerged. This type of method allows multiple participants (such as vehicles, traffic nodes, and urban systems) to collaboratively train the system by sharing only model gradients without exchanging raw data, thereby protecting data privacy to a certain extent.
[0004] Although federated reinforcement learning avoids the plaintext transmission of raw images, the model gradient information itself still carries multi-dimensional information such as data distribution, model response, and input features, making it vulnerable to potential attackers using "inversion reconstruction" attacks. In reinforcement learning scenarios, there is a high degree of coupling between the uploaded policy gradient or loss gradient and the input state (i.e., the image scene observed by the vehicle). By observing this gradient information, attackers may be able to reconstruct the victim's locally observed image, thereby recovering environmental attribute images, such as road type, building layout, and weather conditions. This type of "environmental attribute leakage" not only infringes on the privacy of data providers but may also be used for security threats such as simulation attacks and policy interference.
[0005] Current technologies primarily focus on assessing the risk of information leakage under full semantics and high resolution; however, due to their low execution efficiency, such assessments have been difficult to implement in practical engineering. Summary of the Invention
[0006] The purpose of this application is to provide a rapid data leakage risk assessment method for federal vision-based autonomous driving, addressing the technical problems of poor applicability and low efficiency of existing assessment methods. The various technical effects of the preferred technical solutions provided in this application are detailed below.
[0007] To achieve the above objectives, this application provides the following technical solutions:
[0008] This application provides a rapid data leakage risk assessment method for federated vision autonomous driving, comprising: extracting public environmental attributes from a set of state information using a public information extractor, wherein the state information is a visual image; randomly initializing several sets of pseudo-environment information; averaging the pseudo-state information in the pseudo-environment information element by element in the pixel space to obtain an initial alignment sample; locally updating the pseudo-environment information corresponding to the initial alignment sample until the number of local updates reaches a preset number of iterations; re-extracting alignment samples from the updated pseudo-environment information to update the initial alignment sample; continuing to locally update the pseudo-environment information corresponding to the updated alignment sample until the number of update samples reaches a preset number of alignments; obtaining the final alignment sample as a reconstructed environmental attribute; calculating the similarity between the reconstructed environmental attribute and the public environmental attribute using an image similarity algorithm; and outputting the similarity as a risk assessment index.
[0009] In some embodiments, the local update of the pseudo-environment information corresponding to the initial alignment sample includes: processing the pseudo-environment information using a global autonomous driving model and a preset loss function to obtain a training loss value; calculating the partial derivative vector of the training loss value with respect to the model parameters of the global autonomous driving model to obtain a pseudo gradient; calculating the difference between the pseudo gradient and the gradient information using the L2 norm formula; simultaneously calculating the total variation regularization loss and the alignment regularization loss of the state information in the pseudo-environment information; calculating the total gradient loss based on the difference value, the total variation regularization loss, and the alignment regularization loss; taking the partial derivative of the total gradient loss with respect to the pseudo-environment information to obtain the update amount of the pseudo-environment information; and correcting the pseudo-environment information according to the update amount and the preset update step size to complete the local update.
[0010] In some embodiments, the public environment attribute is represented by the following formula:
[0011]
[0012] in, For the public information extractor, B is the number of visual images. This refers to the state information.
[0013] In some embodiments, the initial alignment sample is represented by the following formula:
[0014]
[0015] Where M refers to the number of sets of the pseudo-environment information. Pixel position The values of the aligned samples on, It is the first The pseudo-environment information described in the group pseudo-state information At pixel position The value of .
[0016] In some embodiments, the difference value is represented by the following formula:
[0017]
[0018] in, The gradient information, This refers to the pseudo gradient.
[0019] In some embodiments, the total variation regularization loss is expressed by the following formula:
[0020]
[0021] in, It is an image In position The pixel value.
[0022] In some embodiments, the alignment regularization loss is expressed by the following formula:
[0023]
[0024] in, For the alignment sample, This refers to the pseudo-state information.
[0025] In some embodiments, the total gradient loss is expressed by the following formula:
[0026]
[0027] in, For total variation regularization weights, To align the regularization weights.
[0028] In some embodiments, the update amount is represented by the following formula:
[0029]
[0030] in, The total gradient loss is... This refers to the pseudo-environment information.
[0031] In some embodiments, the correction of the pseudo-environment information based on the update amount and the preset update step size is expressed by the following formula:
[0032]
[0033] in, The preset update step size.
[0034] Implementing one of the above-mentioned technical solutions in this application has the following advantages or beneficial effects: In this application, public environmental attributes, that is, public prior representations of environmental attributes, are extracted by a public information extractor. Combined with the optimization mechanism guided by gradient matching and regularization, local environmental attributes are quickly reconstructed at the low-resolution and incomplete semantic level to obtain reconstructed environmental attributes. Finally, by calculating the similarity between the reconstructed environmental attributes and public environmental attributes, a risk assessment index for evaluating the degree of leakage of environmental attributes is quantitatively output.
[0035] Compared with existing technologies, this application does not require precise reconstruction of the original input data. By obtaining the reconstructed environmental attributes, it can effectively characterize the risk of leakage of common environmental information, which can significantly reduce computational complexity and improve evaluation efficiency. At the same time, it has good reinforcement learning adaptability and cross-task versatility, and can provide technical support for privacy protection and security auditing of federal autonomous driving systems.
[0036] In addition, this application not only overcomes the problem of evaluation failure in reinforcement learning scenarios of existing technologies, but also has advantages in evaluation efficiency and stability. It can be widely used in resource-constrained environments such as vehicle terminals and edge servers to support the full-process privacy and security audit and protection effect verification of federated vision tasks. Attached Figure Description
[0037] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0038] Figure 1 This is a flowchart illustrating the rapid data leakage risk assessment method for federal vision-based autonomous driving according to an embodiment of this application;
[0039] Figure 2 This is a schematic diagram of the public environment attributes and reconstructed environment attributes in an embodiment of this application. Detailed Implementation
[0040] To make the objectives, technical solutions, and advantages of this application clearer, various exemplary embodiments described below will be referenced to the accompanying drawings, which form part of the exemplary embodiments and depict various exemplary embodiments that may be adopted to implement this application. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. It should be understood that they are merely examples of processes, methods, and apparatuses consistent with some aspects of this application disclosed as detailed in the appended claims, and other embodiments may be used, or structural and functional modifications may be made to the embodiments listed herein without departing from the scope and spirit of this application.
[0041] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," etc., indicate the orientation or positional relationship based on the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the referred element must have a specific orientation, or be constructed and operated in a specific orientation. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. The term "multiple" means two or more. The terms "connected" and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, integral connections, mechanical connections, electrical connections, communication connections, direct connections, indirect connections through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more of the related listed items. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0042] To illustrate the technical solutions described in this application, specific embodiments are provided below, showing only the parts related to the embodiments of this application.
[0043] like Figure 1 As shown, this application provides a rapid data leakage risk assessment method for federal vision-based autonomous driving, including the following steps (steps S1 to S3):
[0044] S1. Use a common information extractor to extract common environmental attributes from the state information set, where the state information is a visual image. Randomly initialize several sets of pseudo-environment information, and average the pseudo-state information in the pseudo-environment information element by element in the pixel space to obtain the initial alignment sample.
[0045] In some embodiments, a public information extractor can be used to extract public environment attributes from a set of original visual images, wherein the set of original visual images may be a set of images used locally by the victim, and the data information in the set of original visual images may be in the form of quadruples. The formal representation, that is, the state. ,action Instant feedback rewards and the next step status Status information Original visual image, state in the next moment This is the original visual image at the next moment.
[0046] This application utilizes long-term cumulative rewards, environmental states, and payoff estimates from reinforcement learning, such as Q-values or policy probability distributions, to reconstruct the victim's local environmental attributes. By combining a public information extractor to extract latent representations, this application avoids the problem of directly relying on label constraints in complex continuous state spaces and temporally dependent data, thereby enabling the effective quantification and reconstruction of environmental state information.
[0047] In some embodiments, the common information extractor can be an encoder-decoder feature extraction model used to extract low-dimensional latent representations from the input visual image and reconstruct features. The common information extractor can be implemented using a variational autoencoder (VAE).
[0048] Pseudo-environment information can be obtained through The quadruple form of the pseudo-environment information can have the same structure as the data information in the state information set. The pseudo-environment information has random values and different quantities. By utilizing the pseudo-environment information, the search space for reconstruction can be expanded.
[0049] In some embodiments, public environment attributes can be represented by the following formula:
[0050]
[0051] in, For public information extractors, B represents the number of visual images. This is status information.
[0052] In some embodiments, the initial alignment sample can be represented by the following formula:
[0053]
[0054] Where M refers to the number of sets of pseudo-environment information. Pixel position The values of the aligned samples on, It is the first Group pseudo-environment information pseudo-state information At pixel position The value of .
[0055] S2. Locally update the pseudo-environment information corresponding to the initial alignment sample until the number of local updates reaches the preset number of iterations. Extract the alignment sample again from the updated pseudo-environment information to update the initial alignment sample. Continue to locally update the pseudo-environment information corresponding to the updated alignment sample until the number of sample updates reaches the preset number of alignments. Obtain the final alignment sample as the reconstructed environment attribute.
[0056] Specifically, the pseudo-environment information can be locally updated T times. That is, it can be determined whether the number of local updates meets the requirement of T times. If it does not meet the requirement, the local update continues. If it does meet the requirement, the alignment sample is re-extracted from the updated pseudo-environment information. At this time, the initial alignment sample becomes the updated alignment sample.
[0057] The aligned sample can be updated Y times. That is, after updating the aligned sample, it can be determined whether the number of updates satisfies Y times. If not, the next round of local updates to the pseudo-environment information continues. In this round, T iterations of local updates can also be performed. This can be understood as updating the aligned sample Y times, with each round of updates including T iterations of local updates to the pseudo-environment information.
[0058] In other embodiments, the preset number of iterations can be adjusted in real time according to actual needs; that is, the preset number of iterations can be different for different rounds, which is not limited here. Correspondingly, the preset number of iterations and the preset number of alignments can be different or the same, which is not limited here.
[0059] In some embodiments, locally updating the pseudo-environment information corresponding to the initial aligned samples may include: processing the pseudo-environment information using a global autonomous driving model and a preset loss function to obtain a training loss value; calculating the partial derivative vector of the training loss value with respect to the model parameters of the global autonomous driving model to obtain a pseudo gradient; calculating the difference between the pseudo gradient and the gradient information using the L2 norm formula; simultaneously calculating the total variation regularization loss and the alignment regularization loss of the state information in the pseudo-environment information; calculating the total gradient loss based on the difference value, the total variation regularization loss, and the alignment regularization loss; taking the partial derivative of the total gradient loss with respect to the pseudo-environment information to obtain the update amount of the pseudo-environment information; and correcting the pseudo-environment information according to the update amount and a preset update step size to complete the local update.
[0060] In some embodiments, the training loss value can be expressed by the following formula:
[0061]
[0062] in, For a global autonomous driving model, The preset loss function is used. A global autonomous driving model can refer to a reinforcement learning model that completes the entire process from perceiving the environment to executing driving operations within a framework, such as the UniAD model and the WAM model. The preset loss function can be basic regression loss, multimodal loss, and policy loss, etc.
[0063] In some embodiments, the pseudo gradient can be represented by the following formula:
[0064]
[0065] in, These are the model parameters for the global autonomous driving model.
[0066] In some embodiments, the difference value can be expressed by the following formula:
[0067]
[0068] in, For gradient information, This is a pseudo gradient.
[0069] In some embodiments, the total variation regularization loss can be expressed by the following formula:
[0070]
[0071] in, It is an image In position pixel values, Used to calculate pixel differences between adjacent rows. This is used to calculate the pixel differences between adjacent columns. It can also be used to store pseudo-state information. For image By introducing total variation regularization loss, the pseudo-environment state can be smoothed, thereby improving the clarity of the reconstructed environment attributes.
[0072] In some embodiments, the alignment regularization loss can be expressed by the following formula:
[0073]
[0074] in, To align samples, This is pseudo-state information. By introducing alignment regularization loss, we can effectively bridge the gap between pseudo-environment information and aligned samples. Distance accelerates the convergence rate of reconstruction.
[0075] In some embodiments, the total gradient loss can be expressed by the following formula:
[0076]
[0077] in, For total variation regularization weights, To align the regularization weights.
[0078] In some embodiments, the update amount can be expressed by the following formula:
[0079]
[0080] in, For the total gradient loss, This is false environmental information.
[0081] In some embodiments, the correction of pseudo-environment information based on the update amount and a preset update step size can be expressed by the following formula:
[0082]
[0083] in, This is the preset update step size.
[0084] Existing evaluation methods often employ complex optimization processes, such as backpropagation and gradient descent, to reconstruct real input samples. While these methods may achieve high accuracy in reconstruction quality, they typically incur significant computational costs, making them unsuitable for real-time evaluation, especially in large-scale autonomous driving scenarios. Furthermore, existing evaluation methods overemphasize precise input reconstruction while neglecting the quantitative evaluation objective of preventing privacy breaches, namely, generating a representative public environment attribute.
[0085] Compared to existing evaluation methods, this application introduces total variation regularization and alignment regularization mechanisms to smooth pseudo-environment information during the reconstruction process, avoiding unnecessary noise interference and optimizing the reconstruction quality of pseudo-environment information. It also avoids high computational costs. Especially in scenarios involving batch data and high-resolution images, this application can significantly reduce the computational burden, improve evaluation throughput, and meet the real-time and batch processing requirements of practical deployments.
[0086] Unlike traditional methods that achieve high-precision reconstruction through numerous iterations and fine-grained optimization, the embodiments of this application focus more on extracting a set of common environmental attributes from multiple sets of pseudo-environment information. Through an iterative process of alignment and local optimization, the stability and quality of the reconstructed environmental attributes are improved while maintaining low computational overhead, thus ensuring the accuracy and real-time performance of privacy leakage assessment.
[0087] S3. Calculate the similarity between the reconstructed environmental attributes and the public environmental attributes using an image similarity algorithm, and output the similarity score as a risk assessment indicator. The similarity score is used to assess the risk of leakage of the victim's environmental attributes. A higher similarity score indicates a greater risk of leakage.
[0088] In some embodiments, image similarity algorithms can be structural similarity algorithms (SSIM), perceptual patch similarity algorithms (LPIPS), etc. Specifically, structural similarity algorithms can compare similarities from three dimensions (brightness, contrast, and structure), and are used in autonomous driving for the optimization of perception modules and data simulation. Perceptual patch similarity algorithms, on the other hand, calculate similarity by computing features, and are used to generate implicit constraints for model generation and end-to-end training.
[0089] Most existing evaluation methods rely on accurate input reconstruction, especially in reinforcement learning. These methods typically depend on label prediction loss functions or precise restoration of the input data. However, in reinforcement learning scenarios, the training objective of the model is to maximize long-term rewards through policy optimization, and the input features are continuous state spaces (such as image or sensor vectors), rather than finite label spaces. Traditional methods struggle to effectively extract and evaluate the implicit common environmental properties within complex state spaces, resulting in lower reconstruction quality and limiting their application in reinforcement learning tasks.
[0090] This application simulates a potential attacker by using a public information extractor to obtain reconstructed environmental attributes, i.e., reconstructed public environmental attributes, from uploaded gradient information and model training resources, rather than accurately reproducing the original input. This application extracts common, shared environmental features (public environmental attributes) from multiple images or state information of multiple participants, such as road type and weather conditions.
[0091] This application assesses privacy leakage risks by calculating the similarity between reconstructed environmental attributes and public environmental attributes, rather than reconstructing the exact original image. This approach effectively reduces computational overhead and improves evaluation efficiency, making it particularly suitable for reinforcement learning environments with temporal dependencies, continuous state spaces, and high-dimensional data.
[0092] Table 1 compares the embodiments of this application with existing methods, where PSNR and SSIM are image similarity indices, and the higher the index, the greater the similarity.
[0093]
[0094] Table 1
[0095] In this application, public environmental attributes, i.e., public prior representations of environmental attributes, are extracted through a public information extractor. Combined with an optimization mechanism guided by gradient matching and regularization, local environmental attributes are rapidly reconstructed at a low-resolution and incomplete semantic level to obtain reconstructed environmental attributes. Finally, by calculating the similarity between the reconstructed environmental attributes and public environmental attributes, a risk assessment index is quantitatively output to evaluate the degree of leakage of environmental attributes.
[0096] Compared with existing technologies, this application does not require precise reconstruction of the original input data. By obtaining the reconstructed environmental attributes, it can effectively characterize the risk of leakage of common environmental information, which can significantly reduce computational complexity and improve evaluation efficiency. At the same time, it has good reinforcement learning adaptability and cross-task versatility, and can provide technical support for privacy protection and security auditing of federal autonomous driving systems.
[0097] In addition, this application not only overcomes the problem of evaluation failure in reinforcement learning scenarios of existing technologies, but also has advantages in evaluation efficiency and stability. It can be widely used in resource-constrained environments such as vehicle terminals and edge servers to support the full-process privacy and security audit and protection effect verification of federated vision tasks.
[0098] Those skilled in the art will understand that all or part of the features / steps of the above-described method embodiments can be implemented by methods, data processing systems, or computer programs. These features may be implemented without hardware, entirely in software, or in a combination of hardware and software. The aforementioned computer program may be stored in one or more computer-readable storage media. When the computer program is executed (e.g., by a processor), it performs the steps of the above-described method embodiments for rapid data leakage risk assessment of federated vision autonomous driving.
[0099] The aforementioned storage media capable of storing program code include: static hard disks, solid-state hard disks, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks, and / or combinations of the above devices, that is, they can be implemented by any type of volatile or non-volatile storage devices or combinations thereof.
[0100] This application also provides a processing device embodiment, including one or more processors and a memory; wherein the memory is used to store one or more computer programs, and the one or more processors are used to execute the one or more computer programs stored in the memory, so that the processors perform the features / steps of the above-described embodiment of the method for rapid assessment of data leakage risks for federal vision autonomous driving.
[0101] The above description is merely a preferred embodiment of this application. Those skilled in the art will understand that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of this application. Furthermore, under the teachings of this application, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of this application. Therefore, this application is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of this application.
Claims
1. A rapid data leakage risk assessment method for federated vision-based autonomous driving, characterized in that, include: A common environment attribute is extracted from the state information set using a common information extractor, wherein the state information is a visual image. Several sets of pseudo-environment information are randomly initialized, and the pseudo-state information in the pseudo-environment information is averaged element by element in the pixel space to obtain the initial alignment sample. The pseudo-environment information corresponding to the initial alignment sample is locally updated until the number of local updates reaches the preset number of iterations. The alignment sample is then extracted again from the updated pseudo-environment information to update the initial alignment sample. The pseudo-environment information corresponding to the updated alignment sample is then locally updated until the number of sample updates reaches the preset number of alignments. Finally, the alignment sample is obtained as the reconstructed environment attribute. The similarity between the reconstructed environmental attributes and the public environmental attributes is calculated using an image similarity algorithm, and the similarity is output as a risk assessment indicator. The step of locally updating the pseudo-environment information corresponding to the initial alignment sample includes: The pseudo-environment information is processed using a global autonomous driving model and a preset loss function to obtain a training loss value. The partial derivative vector of the training loss value with respect to the model parameters of the global autonomous driving model is calculated to obtain a pseudo gradient. The difference between the pseudo gradient and the gradient information is calculated using the L2 norm formula. At the same time, the total variation regularization loss and the alignment regularization loss of the state information in the pseudo-environment information are calculated. Based on the difference value, the total variation regularization loss, and the alignment regularization loss, the total gradient loss is calculated. The partial derivative of the total gradient loss with respect to the pseudo-environment information is obtained to obtain the update amount of the pseudo-environment information. The pseudo-environment information is corrected according to the update amount and a preset update step size to complete the local update.
2. The rapid data leakage risk assessment method for federated vision-based autonomous driving according to claim 1, characterized in that, The public environment attributes are represented by the following formula: in, For the public information extractor, B is the number of visual images. This refers to the state information.
3. The rapid data leakage risk assessment method for federated vision-based autonomous driving according to claim 1, characterized in that, The initial alignment sample is represented by the following formula: Where M refers to the number of sets of the pseudo-environment information. Pixel position The values of the aligned samples on, It is the first The pseudo-environment information described in the group pseudo-state information At pixel position The value of .
4. The rapid data leakage risk assessment method for federated vision-based autonomous driving according to claim 1, characterized in that, The difference value is expressed by the following formula: in, The gradient information, This refers to the pseudo gradient.
5. The rapid data leakage risk assessment method for federated vision autonomous driving according to claim 4, characterized in that, The total variation regularization loss is expressed by the following formula: in, It is an image In position The pixel value.
6. The rapid data leakage risk assessment method for federated vision-based autonomous driving according to claim 5, characterized in that, The alignment regularization loss is expressed by the following formula: in, For the alignment sample, This refers to the pseudo-state information.
7. The rapid data leakage risk assessment method for federated vision-based autonomous driving according to claim 6, characterized in that, The total gradient loss is expressed by the following formula: in, For total variation regularization weights, To align the regularization weights.
8. The rapid data leakage risk assessment method for federated vision-based autonomous driving according to claim 1, characterized in that, The update amount is expressed by the following formula: in, The total gradient loss is... This refers to the pseudo-environment information.