Unmanned aerial vehicle anti-deception capability improvement method based on LLM multi-modal data generation

Abstract

The invention relates to the technical field of deception data generation, in particular to an unmanned aerial vehicle anti-deception capability improvement method based on LLM multi-modal data generation, and the method comprises the steps: collecting multi-modal original data of a target unmanned aerial vehicle in a flight task execution process, and carrying out the time synchronization and preprocessing of the multi-modal original data, so as to obtain corresponding multi-modal feature data, packaging the multi-modal feature data into uniformly structured text description information; constructing a corresponding standardized Prompt template, and inputting the corresponding standardized Prompt template into the large language fine tuning model to output multi-modal deception data in a target format; and performing consistency check on the multi-modal spoofing data to construct a spoofing data set, and further updating the spoofing data set based on a feedback dynamic optimization mechanism for the unmanned aerial vehicle to be subsequently used for enhancing anti-spoofing training or performance evaluation. According to the method, the attack scene coverage range can be effectively expanded, the manual sample design cost is reduced, and the generated deception sample is closer to the real attack logic.

Description

Technical Field

[0001] The present application relates to the technical field of deception data generation, and in particular to a method for improving the anti-deception capability of a drone based on LLM multimodal data generation. Background Art

[0002] In recent years, drone technology has been widely used in logistics and distribution, geographic mapping, security inspections, and other fields. As their operating environments become increasingly complex, the threats of electromagnetic attacks and data spoofing are becoming increasingly severe. Drone systems rely heavily on multimodal data, including GPS (Global Positioning System), IMU (Inertial Measurement Unit) inertial navigation, and visual sensors, for positioning and navigation. Attackers can launch spoofing attacks by spoofing GPS signals, interfering with IMU data, or tampering with visual information, causing drones to deviate from their intended routes, leading to mission failures and even accidents.

[0003] Traditional drone anti-spoofing training methods mainly rely on manually designed attack samples or virtual attack generation based on simulation platforms. However, manually designed samples are limited by human experience and cannot cover the hidden and diverse attack modes in complex electromagnetic environments. Simulation platforms usually use idealized models, which cannot truly restore the complex characteristics of attack signals in dynamic environments, such as satellite signal multipath effects and IMU sensor temperature drift. The above limitations lead to significant differences between the training data generated by traditional methods and real attack scenarios, making the trained drone systems still face a high risk of being deceived in actual applications.

[0004] In recent years, artificial intelligence technology has rapidly advanced, particularly large language models (LLMs), which have demonstrated remarkable capabilities in complex scene understanding and content generation. Currently, existing technologies can fine-tune LLMs to enable them to understand the characteristics of multimodal drone data, such as GPS trajectories, IMU dynamics, and visual scene information, enabling the automated generation and scenario-specific adaptation of attack samples.

[0005] However, the application of existing large language models is still limited to a single modality or open-loop generation mode, lacking real-time feedback and dynamic optimization of attack effects, making it difficult to continuously improve the anti-deception capabilities of drones, which needs to be urgently addressed. Summary of the Invention

[0006] This application provides a method for improving the anti-spoofing capability of drones based on LLM multimodal data generation to address the problems that the application of existing large language models is limited to a single modality or open-loop generation mode, lacks real-time feedback and dynamic optimization of attack effects, and is difficult to continuously improve the anti-spoofing capability of drones.

[0007] The first embodiment of the present application provides a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation, comprising the following steps: collecting multimodal raw data of a target drone during the execution of a preset flight mission, and performing time synchronization and preprocessing operations on the multimodal raw data to obtain corresponding multimodal feature data, and encapsulating the multimodal feature data into a unified structured text description information; based on the text description information and a preset attack type, constructing a corresponding standardized Prompt template, and fine-tuning the pre-constructed large language model to generate a corresponding large language fine-tuning model; inputting the standardized Prompt template into the large language fine-tuning model to output multimodal deception data in a target format, wherein the multimodal deception data includes a forged GPS trajectory point series, a perturbed IMU acceleration trend, and a disguised image description.

[0008] Optionally, in one embodiment of the present application, after inputting the standardized Prompt template into the large language fine-tuning model to output the multimodal spoofing data in the target format, it also includes: obtaining the actual flight trajectory corresponding to the target UAV, and calculating the average offset index between the forged GPS trajectory point column in the multimodal spoofing data and the actual flight trajectory, and based on the average offset index, determining the target forged GPS trajectory point column in the forged GPS trajectory point column that meets the preset speed change constraint detection requirement; obtaining each inertial data frame corresponding to the target UAV, and filtering and aligning the inertial data frame to obtain a corresponding filtered inertial data frame; calculating the Euclidean difference modulus corresponding to each inertial data frame according to the filtered inertial data frame, and using the Euclidean difference modulus to calculate the average disturbance of all inertial data frames, and performing a disturbance consistency check operation on the disturbance IMU acceleration trend based on the average disturbance and a preset maximum tolerance threshold to generate a target disturbance consistency check that meets the preset disturbance consistency check requirements. performing a saliency occlusion extraction operation on the camouflaged image description to generate a corresponding camouflaged mask area, and respectively calculating the camouflaged area area and the total image area corresponding to the camouflaged mask area and the camouflaged image description; calculating the corresponding occlusion ratio based on the camouflaged area area and the total image area, and performing a preset image camouflage rationality check operation on the camouflaged image description according to the occlusion ratio to obtain a target camouflaged image description that meets the image camouflage rationality check requirements; constructing a corresponding deception data set based on the target camouflaged image description, the target perturbation IMU acceleration trend, and the target forged GPS trajectory point sequence; optimizing the standardized Prompt template according to the deception data set and a preset dynamic optimization mechanism to generate multimodal deception samples, and performing enhanced anti-spoofing training and / or performance evaluation operations on the target UAV through the multimodal deception samples to obtain corresponding training data and / or evaluation results, so as to optimize the performance of the target UAV in executing the preset flight mission based on the training data and / or the evaluation results.

[0009] Optionally, in one embodiment of the present application, the standardized Prompt template is optimized according to the deception dataset and a preset dynamic optimization mechanism to generate multimodal deception samples, and the target UAV is subjected to enhanced anti-spoofing training and / or performance evaluation operations through the multimodal deception samples, including: real-time detection of whether the target UAV successfully implements a deception attack using the deception dataset, wherein, when it is detected that the target UAV successfully implements a deception attack, the multimodal sensor data and the status information of the target UAV when the deception attack is successfully implemented are recorded to generate attack feedback data based on the multimodal sensor data and the status information; based on the attack feedback data, the standardized Prompt template is dynamically adjusted, and multimodal deception samples are generated according to the dynamically adjusted standardized Prompt template to update the deception dataset through the multimodal deception samples; based on the preset anti-spoofing training strategy, performance evaluation strategy and updated deception dataset, the target UAV is subjected to enhanced anti-spoofing training and / or performance evaluation operations.

[0010] Optionally, in one embodiment of the present application, the multimodal raw data collected during the target UAV's execution of a preset flight mission, and the multimodal raw data are time synchronized and preprocessed to obtain corresponding multimodal feature data, including: obtaining the multimodal raw data collected during the target UAV's execution of the preset flight mission, wherein the multimodal raw data includes GPS position information, IMU inertial measurement data, and an airborne camera image sequence; performing a millisecond-level alignment operation on the GPS position information, the IMU inertial measurement data, and the airborne camera image sequence in the multimodal raw data to obtain GPS position alignment information, IMU inertial measurement alignment data, and an airborne camera image alignment sequence; performing low-pass filtering on the IMU inertial measurement alignment data to obtain corresponding filtered data, and extracting speed change features corresponding to the GPS position alignment information and semantic features corresponding to the airborne camera image alignment sequence, and performing dimensionality reduction on the semantic features to generate corresponding reduced-dimensionality semantic features; and constructing the multimodal feature data based on the filtered data, the speed change features, and the reduced-dimensionality semantic features.

[0011] The second embodiment of the present application provides a device for improving the anti-spoofing capability of a drone generated based on LLM multimodal data, including: a preprocessing module for collecting multimodal raw data of a target drone during the execution of a preset flight mission, and performing time synchronization and preprocessing operations on the multimodal raw data to obtain corresponding multimodal feature data, and encapsulating the multimodal feature data into a unified structured text description information; a fine-tuning module for constructing a corresponding standardized Prompt template based on the text description information and a preset attack type, and fine-tuning the pre-constructed large language model to generate a corresponding large language fine-tuning model; an inference module for inputting the standardized Prompt template into the large language fine-tuning model to output multimodal deception data in a target format, wherein the multimodal deception data includes a forged GPS trajectory point list, a perturbed IMU acceleration trend, and a camouflaged image description. Optionally, in one embodiment of the present application, it further includes: a speed change constraint module, which is used to obtain the actual flight trajectory corresponding to the target UAV after inputting the standardized Prompt template into the large language fine-tuning model to output the multimodal spoofing data in the target format, and calculate the average offset index between the forged GPS trajectory point column in the multimodal spoofing data and the actual flight trajectory, and based on the average offset index, determine the target forged GPS trajectory point column in the forged GPS trajectory point column that meets the preset speed change constraint detection requirements; a filtering module, which is used to obtain each inertial data frame corresponding to the target UAV, and filter and align each inertial data frame to obtain a corresponding filtered inertial data frame; a disturbance consistency check module, which is used to calculate the Euclidean difference modulus corresponding to each inertial data frame according to the filtered inertial data frame, and use the Euclidean difference modulus to calculate the average disturbance of all inertial data frames, so as to perform a disturbance consistency check operation on the disturbance IMU acceleration trend based on the average disturbance and a preset maximum tolerance threshold to generate a target disturbance I that meets the preset disturbance consistency check requirements. MU acceleration trend; a calculation module, used to perform a saliency occlusion extraction operation on the camouflaged image description to generate a corresponding camouflaged mask area, and respectively calculate the camouflaged area area and the total image area corresponding to the camouflaged mask area and the camouflaged image description; a camouflage rationality verification module, used to calculate the corresponding occlusion ratio based on the camouflaged area area and the total image area, and perform a preset image camouflage rationality verification operation on the camouflaged image description according to the occlusion ratio to obtain a target camouflaged image description that meets the image camouflage rationality verification requirements; a construction module, used to construct a corresponding deception data set based on the target camouflaged image description, the target perturbed IMU acceleration trend and the target forged GPS trajectory point sequence; an optimization module, used to optimize the standardized Prompt template according to the deception data set and a preset dynamic optimization mechanism to generate multimodal deception samples, and perform enhanced anti-spoofing training and / or performance evaluation operations on the target UAV through the multimodal deception samples to obtain corresponding training data and / or evaluation results, so as to optimize the performance of the target UAV in executing the preset flight mission based on the training data and / or the evaluation results.

[0012] Optionally, in one embodiment of the present application, the optimization module includes: a detection unit, configured to detect in real time whether the target UAV successfully implements a deception attack using the deception data set, wherein, when it is detected that the target UAV successfully implements a deception attack, the multimodal sensor data and the status information of the target UAV when the deception attack is successfully implemented are recorded, so as to generate attack feedback data based on the multimodal sensor data and the status information; an updating unit, configured to dynamically adjust the standardized Prompt template based on the attack feedback data, and generate a multimodal deception sample based on the dynamically adjusted standardized Prompt template, so as to update the deception data set through the multimodal deception sample; a performance evaluation unit, configured to perform enhanced anti-spoofing training and / or performance evaluation operations on the target UAV based on a preset anti-spoofing training strategy, a performance evaluation strategy and an updated deception data set.

[0013] Optionally, in one embodiment of the present application, the preprocessing module includes: an acquisition unit for acquiring multimodal raw data collected by the target UAV during the execution of the preset flight mission, wherein the multimodal raw data includes GPS position information, IMU inertial measurement data and an airborne camera image sequence; an alignment unit for performing millisecond-level alignment operations on the GPS position information, the IMU inertial measurement data and the airborne camera image sequence in the multimodal raw data to obtain GPS position alignment information, IMU inertial measurement alignment data and an airborne camera image alignment sequence; a dimensionality reduction unit for performing low-pass filtering on the IMU inertial measurement alignment data to obtain corresponding filtered data, and extracting speed change features corresponding to the GPS position alignment information and semantic features corresponding to the airborne camera image alignment sequence, and performing dimensionality reduction on the semantic features to generate corresponding reduced-dimensionality semantic features; and an establishment unit for constructing the multimodal feature data based on the filtered data, the speed change features and the reduced-dimensionality semantic features.

[0014] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation as described in the above embodiment.

[0015] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned method for improving the anti-deception capability of a drone based on LLM multimodal data generation.

[0016] The fifth aspect of the present application provides a computer program product, including a computer program, which is executed to implement the above-mentioned method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation.

[0017] Therefore, the embodiments of the present application have the following beneficial effects: The embodiments of the present application can obtain corresponding multimodal feature data by collecting multimodal raw data of the target UAV during the flight mission, and performing time synchronization and preprocessing on the data, and encapsulating the multimodal feature data into a unified structured text description information; constructing a corresponding standardized Prompt template, and inputting it into the large language fine-tuning model to output multimodal deception data in the target format; performing consistency verification on the multimodal deception data to construct a deception dataset, and then updating the deception dataset based on a dynamic optimization mechanism based on feedback for subsequent use by the UAV to enhance anti-deception training or performance evaluation. The present application can effectively expand the coverage of attack scenarios, reduce the cost of manually designed samples, and the generated deception samples are closer to the real attack logic. Thus, it solves the problems that the application of existing large language models is limited to a single modality or open-loop generation mode, lacks real-time feedback and dynamic optimization of attack effects, and is difficult to continuously improve the anti-deception capabilities of UAVs.

[0018] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart of a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation according to an embodiment of the present application; Figure 2 A schematic diagram of the execution logic of a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation provided in one embodiment of the present application; Figure 3 This is an example diagram of a device for improving the anti-spoofing capability of a drone based on LLM multimodal data generation according to an embodiment of the present application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0020] Among them, 10-a device for improving the anti-deception capability of drones based on LLM multimodal data generation; 100-preprocessing module, 200-fine-tuning module, 300-inference module; 401-memory, 402-processor, 403-communication interface. DETAILED DESCRIPTION

[0021] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0022] The following describes, with reference to the accompanying drawings, a method for improving the anti-spoofing capabilities of drones based on LLM multimodal data generation, according to an embodiment of the present application. To address the issues mentioned in the background art, the present application provides a method for improving the anti-spoofing capabilities of drones based on LLM multimodal data generation. In this method, multimodal raw data from a target drone performing a flight mission is collected, time-synchronized, and preprocessed to obtain corresponding multimodal feature data. The multimodal feature data is then encapsulated into a uniformly structured text description. A corresponding standardized prompt template is constructed and input into a large language fine-tuning model to output multimodal deception data in the target format. A consistency check is then performed on the multimodal deception data to construct a deception dataset. This dataset is then updated through a dynamic optimization mechanism based on feedback for subsequent use by drones in enhanced anti-spoofing training or performance evaluation. This application effectively expands the coverage of attack scenarios, reduces the cost of manually designed samples, and generates deception samples that are closer to real-world attack logic. This method addresses the problems of existing large language models, which are limited to single modal or open-loop generation modes, lack real-time feedback and dynamic optimization of attack effects, and are difficult to continuously improve drone anti-spoofing capabilities.

[0023] Specifically, Figure 1 This is a flowchart of a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation provided in an embodiment of the present application.

[0024] like Figure 1 As shown, the method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation includes the following steps: In step S101, multimodal raw data of the target UAV during the execution of a preset flight mission is collected, and time synchronization and preprocessing operations are performed on the multimodal raw data to obtain corresponding multimodal feature data, and the multimodal feature data is encapsulated into a unified structured text description information.

[0025] The embodiment of the present application can first collect multimodal raw data from a normal flight mission of a drone, and perform millisecond-level alignment and standardization preprocessing on each modal data to obtain corresponding multimodal feature data.

[0026] Afterwards, the embodiment of the present application can abstract the multimodal feature data obtained after the above processing into unified structured text description information, thereby providing a reliable data basis for the subsequent generation of multimodal deception data.

[0027] Optionally, in one embodiment of the present application, multimodal raw data of the target UAV during the execution of a preset flight mission is collected, and time synchronization and preprocessing operations are performed on the multimodal raw data to obtain corresponding multimodal feature data, including: obtaining the multimodal raw data collected by the target UAV during the execution of the preset flight mission, wherein the multimodal raw data includes GPS position information, IMU inertial measurement data and an airborne camera image sequence; performing millisecond-level alignment operations on the GPS position information, IMU inertial measurement data and the airborne camera image sequence in the multimodal raw data to obtain GPS position alignment information, IMU inertial measurement alignment data and an airborne camera image alignment sequence; performing low-pass filtering on the IMU inertial measurement alignment data to obtain corresponding filtered data, and extracting speed change features corresponding to the GPS position alignment information and semantic features corresponding to the airborne camera image alignment sequence, and performing dimensionality reduction on the semantic features to generate corresponding reduced-dimensionality semantic features; constructing multimodal feature data based on the filtered data, speed change features and reduced-dimensionality semantic features.

[0028] Specifically, the embodiment of the present application can first collect multi-modal raw data from the normal flight mission of the drone, including GPS position information, IMU inertial measurement data and image sequences collected by the onboard camera; secondly, the embodiment of the present application can adopt a unified time synchronization mechanism (such as timestamp synchronization based on the ROS system) to align the data of each modality at the millisecond level to ensure the consistency of subsequent fusion processing.

[0029] Furthermore, the embodiments of the present application may adopt standardized preprocessing technology, perform first-order differential encoding on the GPS trajectory data to extract speed change characteristics, perform Butterworth low-pass filtering on the IMU data to reduce noise, and extract high-level semantic features from the image data through a lightweight convolutional neural network (such as MobileNetV2), and combine it with principal component analysis for dimensionality reduction, thereby obtaining corresponding multimodal feature data.

[0030] Afterwards, the embodiment of the present application can abstract the multimodal feature data obtained after the above processing into a unified structured text description information, that is, encapsulate all processed features into a structured natural language text description, thereby providing a reliable data basis for the subsequent generation of multimodal deception data.

[0031] In step S102, based on the text description information and the preset attack type, a corresponding standardized Prompt template is constructed, and the pre-constructed large language model is fine-tuned to generate a corresponding large language fine-tuning model.

[0032] In step S103, the standardized Prompt template is input into the large language fine-tuning model to output multimodal spoofing data in a target format, wherein the multimodal spoofing data includes a forged GPS trajectory point sequence, a perturbed IMU acceleration trend, and a disguised image description.

[0033] Furthermore, embodiments of the present application also require designing standardized prompt templates for specific attack types (such as GPS location spoofing, IMU inertial perturbations, or visual camouflage) to clearly describe the current environment state, the desired spoofing target, and the constraints for generating data. The prompt text is organized in a standard format, including an environment summary, an attack target description, and output format requirements, to ensure that the LLM can accurately understand the task context and generate instructions. Secondly, the embodiment of the present application can select a large language model LLM based on the LLaMA or Mistral architecture, and use LoRA (Low-Rank Adaptation) technology to fine-tune the LLM, quickly adapt the ability to generate targeted deception samples, and control the inference temperature (such as temperature = 0.5) to balance the generation quality and diversity.

[0034] Afterwards, the embodiment of the present application can input the standardized Prompt template into the fine-tuned LLM (i.e., large language fine-tuning model) to infer and generate multimodal deception data. The multimodal deception data includes forged GPS trajectory point series, perturbed IMU acceleration trends, and disguised image descriptions. The embodiment of the present application can use a unified JSON structure (i.e., target format) to format and output the multimodal deception data, thereby providing reliable data support for subsequent analysis and verification of the deception data.

[0035] Optionally, in one embodiment of the present application, after the standardized Prompt template is input into the large language fine-tuning model to output the multimodal deception data in the target format, it also includes: obtaining the actual flight trajectory corresponding to the target UAV, and calculating the average offset index between the forged GPS trajectory point column in the multimodal deception data and the actual flight trajectory, and based on the average offset index, determining the target forged GPS trajectory point column in the forged GPS trajectory point column that meets the preset speed change constraint detection requirements; obtaining each inertial data frame corresponding to the target UAV, and filtering and aligning each inertial data frame to obtain the corresponding filtered inertial data frame; calculating the Euclidean difference modulus corresponding to each inertial data frame according to the filtered inertial data frame, and using the Euclidean difference modulus to calculate the average disturbance of all inertial data frames, and performing a disturbance consistency check operation on the disturbance IMU acceleration trend based on the average disturbance and the preset maximum tolerance threshold to generate a disturbance consistency check that meets the preset disturbance consistency check requirements. The target perturbs the IMU acceleration trend; performs a saliency occlusion extraction operation on the camouflage image description to generate a corresponding camouflage mask area, and calculates the camouflage area area and the total image area corresponding to the camouflage mask area and the camouflage image description respectively; calculates the corresponding occlusion ratio based on the camouflage area area and the total image area, and performs a preset image camouflage rationality check operation on the camouflage image description according to the occlusion ratio to obtain a target camouflage image description that meets the image camouflage rationality check requirements; constructs a corresponding deception data set based on the target camouflage image description, the target perturbation IMU acceleration trend and the target forged GPS trajectory point sequence; optimizes the standardized Prompt template according to the deception data set and the preset dynamic optimization mechanism to generate multimodal deception samples, and performs enhanced anti-deception training and / or performance evaluation operations on the target UAV through the multimodal deception samples to obtain corresponding training data and / or evaluation results, so as to optimize the performance of the target UAV in performing the preset flight mission based on the training data and / or evaluation results.

[0036] It should be noted that the embodiments of the present application can design a consistency verification mechanism for multi-modal output to ensure the physical consistency and application reliability of the generated deception samples. The consistency verification mechanism includes speed change constraint detection of GPS trajectory continuity to ensure that the trajectory offset is within the physically feasible range; consistency analysis of IMU acceleration changes and trajectory offset directions to prevent logical contradictions; and environmental rationality verification of image camouflage descriptions to ensure that the visual disturbance content matches the original flight scene.

[0037] In actual implementation, the embodiment of the present application uses the above consistency verification mechanism to perform a constrained evaluation of the disturbances of GPS trajectory, IMU inertial information, and image data, and only includes data that passes all consistency checks into the spoofing dataset, as described below: (1) GPS track offset test: In terms of trajectory consistency, the embodiment of the present application can first calculate the average offset index D between the forged trajectory and the real flight trajectory (ie, the actual flight trajectory). offset , as shown below:

[0038] in, ( x i , y i )and( x i ′, y i ′) represent the i The original GPS coordinates and generated GPS coordinates of the time points; N represents the total number of GPS data points contained in the trajectory.

[0039] GPS trajectory data is automatically collected through GPS message recording of the ROS system, and the original and generated trajectories are paired using timestamp alignment. In the embodiment of the present application, the geodetic distance between two points can be calculated using the geopy or haversine library in Python, and the average offset value of each segment is finally obtained.

[0040] In order to ensure that the trajectory disturbance is both deceptive and does not exceed the acceptable range of the flight control system, the embodiment of the present application requires D offset Falling within the set range [D min , D max ], thereby ensuring that the trajectory disturbance amplitude is reasonable and only retaining the forged samples whose offsets are within this interval (i.e., the target forged GPS trajectory point sequence).

[0041] (2) IMU acceleration disturbance consistency test: Secondly, the embodiment of the present application can use the acceleration disturbance rate Raccel to characterize the degree of disturbance, which is used to measure the average difference between the generated data and the real data in the three-axis acceleration, as shown in the following formula:

[0042] in Indicates that the original IMU sensor is i The three-axis acceleration value of the frame; For the i The acceleration 3D vector of the frame forgery.

[0043] It should be noted that the above data comes from the inertial data frame in the IMU log, which is obtained by subscribing to / mavros / imu / data by ROS (Robot Operating System); the data is smoothed by Butterworth low-pass filtering and aligned with the generated samples, and the Euclidean difference modulus is calculated frame by frame, and the average value is the disturbance rate; the embodiment of the application requires the average disturbance amount R accel Less than the maximum tolerance threshold ε accel (usually set to within 0.5g). If it exceeds the set threshold, it is considered as an abnormal sample that does not conform to the flight physics law and is eliminated, thereby obtaining the target disturbance IMU acceleration trend.

[0044] (3) Image camouflage rationality test: As a feasible method, the embodiment of the present application can use YOLOv8 or DeepLabV3+ semantic segmentation model to extract the saliency occlusion of the forged image or the corresponding area of ​​the image description to form a camouflage mask area, C cover That is, the ratio of the camouflaged area to the total image area (i.e., the occlusion ratio), as shown below:

[0045] Among them, A cover is the number of pixels in the mask area (i.e., the area of ​​the camouflage area); A total is the total number of pixels in the entire image (i.e. the total image area).

[0046] Therefore, the embodiment of the present application can process the image through OpenCV or torchvision.transforms, extract the camouflage area and automatically calculate the occlusion ratio. In order to take into account both the effectiveness and perceptibility of interference, the embodiment of the present application requires the occlusion ratio C cover Control it between 10% and 40% to ensure that the camouflage interference is effective without destroying the overall image comprehension, avoid the situation where a too small ratio may not induce misjudgment, and a too large ratio may destroy the overall environmental perception logic.

[0047] During the deception sample generation process, the embodiment of the present application may introduce a dynamic optimization mechanism based on feedback. When the drone system detects a successful deception attack (such as mission abnormality, trajectory deviation, sensor abnormality), the multimodal sensor data and system status at the corresponding moment may be used as feedback information to record the deception attack characteristics. The feedback information includes the attack sample type, deception disturbance pattern, and environment description.

[0048] In addition, the embodiments of the present application can dynamically adjust the deception sample generation strategy based on feedback information, including optimizing the prompt content, increasing the attack intensity, introducing new attack combinations, or adjusting the scene complexity to guide the LLM to generate more diverse and challenging multimodal deception samples; the generated improved deception samples are updated to the output deception dataset after consistency verification, for subsequent use by the drone system to enhance anti-spoofing training and performance evaluation, thereby improving the performance of the drone in executing preset flight missions.

[0049] Optionally, in one embodiment of the present application, the standardized Prompt template is optimized according to the deception data set and a preset dynamic optimization mechanism to generate multimodal deception samples, and the target UAV is subjected to enhanced anti-spoofing training and / or performance evaluation operations through the multimodal deception samples, including: real-time detection of whether the target UAV successfully implements a deception attack using the deception data set, wherein, when it is detected that the target UAV successfully implements a deception attack, the multimodal sensor data and the status information of the target UAV when the deception attack is successfully implemented are recorded to generate attack feedback data based on the multimodal sensor data and status information; based on the attack feedback data, the standardized Prompt template is dynamically adjusted, and multimodal deception samples are generated according to the dynamically adjusted standardized Prompt template to update the deception data set through the multimodal deception samples; based on the preset anti-spoofing training strategy, performance evaluation strategy and updated deception data set, the target UAV is subjected to enhanced anti-spoofing training and / or performance evaluation operations.

[0050] It should be noted that during the flight or simulation of the drone, if a successful deception attack is detected (such as trajectory deviation, mission abnormality, or sensor status abnormality), the multimodal sensor data and system status information at the corresponding moment are recorded to form attack feedback data; the embodiment of the present application can dynamically adjust the standardized Prompt design based on the attack feedback data to enhance the attack intensity, combine attack types, or increase the complexity of the environment, thereby guiding the LLM to generate more diverse and challenging multimodal deception samples.

[0051] Afterwards, the embodiment of the present application can update the improved deception samples that have passed the consistency check into the output deception data set for subsequent use by the drone system for anti-deception capability training and performance verification.

[0052] As an achievable approach, to further demonstrate its significant superiority in anti-spoofing training, the embodiments of the present application can theoretically explain and verify the effectiveness of the generated multimodal deception samples and their positive effect on the training effect from the perspective of mathematical modeling and quantifiable indicators, as described below: 1. Entropy analysis of sample diversity improvement: In the specific implementation process, the traditional artificially constructed data distribution can be set as Phuman (x), the distribution of the multimodal deception samples generated by the embodiment of the present application is P LLM (x), then from the perspective of information entropy, it can be defined as:

[0053]

[0054] in, and They represent the information entropy of traditional artificially constructed deception samples and the information entropy of multimodal deception samples generated by large language models; P LLM (x) represents the probability distribution of multimodal deception samples generated by the large language model in the feature space; P human (x) represents the probability distribution of traditional artificially constructed deception samples in the feature space.

[0055] It should be noted that the embodiment of the present application can compare the information entropy H(P) of two samples in a discrete feature space. LLM )>H(P human ), reflecting that the LLM-generated samples are more diverse in the feature dimension space. In practical applications, the embodiment of the present application can estimate the entropy distribution using the bucketing method by discretely dividing the generated samples in the feature dimension.

[0056] 2. Sample distribution fitting authenticity index: In the embodiment of the present application, the generated multimodal deception samples and the real samples can be projected into a unified high-dimensional feature space (such as image embedding or trajectory embedding space), and then their means and covariances can be compared. The mean and covariance of the generated samples in the embedding space are recorded as (μ g ,Σ g ), the real attack sample distribution is (μ r ,Σ r ), you can calculate the Frechet Inception Distance (FID, a quality indicator of the image generated by the generative model):

[0057] in, Indicates the degree of similarity between the generated data and the real data distribution. The smaller the FID value, the closer the generated data is to the distribution of the real data. Represents the average feature vector of the real attack sample in the high-dimensional feature space; represents the average feature vector of the deceptive samples generated by LLM in the high-dimensional feature space; (·) represents the trace of the covariance matrix; Represents the covariance matrix of each dimension of the real sample in the high-dimensional feature space; Represents the covariance matrix of the generated sample in each dimension of the feature space.

[0058] Therefore, the embodiment of the present application can dynamically reduce the FID value by optimizing the prompt and feedback mechanism, so that the generated samples are closer to the actual attack data distribution.

[0059] 3. Training performance with improved anti-spoofing robustness: It is understandable that performance verification can be achieved by constructing an adversarial sample test set, training with and without LLM samples under the same model architecture and hyperparameters, and comparing the accuracy improvement.

[0060] In the model training stage, the embodiment of the present application can introduce LLM to generate samples, and assume that the accuracy of the anti-attack after training is from Acc baseline Promoted to Acc LLM , then the performance improvement measure is: ΔAcc = Acc LLM - Acc baseline in, Acc represents the adversarial attack detection accuracy of the model; LLM Acc represents the detection accuracy of the model against attacks after the introduction of LLM generated samples for training; baseline Denotes the adversarial attack detection accuracy of the baseline model.

[0061] Therefore, in the training set containing LLM samples, the embodiments of the present application can improve the data based on the detection rate of the drone system for complex deception scenarios such as camouflaged images and trajectory drift to determine the enhancement of the model robustness.

[0062] It can be understood that the embodiments of the present application can effectively expand the coverage of attack scenarios, and the generated deception samples are closer to the real attack logic, which reduces the cost of manually designed samples and is well applied to scenarios such as enhancing the anti-deception training set of drone systems and generating deception attack scripts in simulation environments.

[0063] The following describes the execution logic of a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation in conjunction with the accompanying drawings.

[0064] Figure 2 This is a logic diagram for executing a method for improving the anti-spoofing capability of drones based on LLM multimodal data generation in this application. Figure 2 As shown, the execution process of the method for improving the anti-spoofing capability of drones based on LLM multimodal data generation in this application is as follows: S201: Collect multimodal raw data (GPS position information, IMU inertial measurement data, and image sequences collected by the onboard camera); S202: Based on the preset difference, filtering and dimensionality reduction strategies, time synchronization and preprocessing are performed on the multimodal raw data to obtain multimodal feature data; S203: Encapsulating the multimodal feature data into unified structured text description information; S204: Build a corresponding standardized prompt template for the expected attack type; S205: Input the standardized prompt template into the fine-tuned large language model to generate multimodal deception data through inference; S206: Output multimodal deception data in a unified JSON structure; S207: Check whether the multimodal deception data passes the consistency check. If so, go to S208; otherwise, go to S204. S208: Constructing a deception sample dataset using the multimodal deception data that passes the consistency check; S209: Based on the deception sample data set, detect whether there is any attack success record feedback during the drone's mission execution. If there is any attack success record feedback, go to S2010, otherwise end the execution process of the method for improving the anti-spoofing capability of drones based on LLM multimodal data generation of this application; S2010: Conduct subsequent anti-spoofing capability training and performance verification of the UAV system based on feedback from successful attack records.

[0065] According to an embodiment of the present application, a method for improving the anti-spoofing capability of drones based on LLM multimodal data generation is proposed. By collecting multimodal raw data of the target drone during the flight mission, and performing time synchronization and preprocessing on it, the corresponding multimodal feature data is obtained, and the multimodal feature data is encapsulated into a unified structured text description information; a corresponding standardized Prompt template is constructed and input into the large language fine-tuning model to output multimodal deception data in the target format; the multimodal deception data is subjected to consistency verification to construct a deception dataset, and then based on a dynamic optimization mechanism based on feedback, the deception dataset is updated for subsequent use by drones to enhance anti-spoofing training or performance evaluation. This application can effectively expand the coverage of attack scenarios, reduce the cost of manually designed samples, and the generated deception samples are closer to the real attack logic.

[0066] Secondly, with reference to the accompanying drawings, a device for improving the anti-spoofing capability of a drone based on LLM multimodal data generation according to an embodiment of the present application is described.

[0067] Figure 3 It is a block diagram of a device for improving the anti-spoofing capability of a drone generated based on LLM multimodal data according to an embodiment of the present application.

[0068] like Figure 3 As shown, the device 10 for improving the anti-spoofing capability of a drone based on LLM multimodal data generation includes: a preprocessing module 100, a fine-tuning module 200 and an inference module 300.

[0069] Among them, the preprocessing module 100 is used to collect multimodal raw data of the target UAV during the execution of a preset flight mission, and perform time synchronization and preprocessing operations on the multimodal raw data to obtain corresponding multimodal feature data, and encapsulate the multimodal feature data into a unified structured text description information.

[0070] The fine-tuning module 200 is used to construct a corresponding standardized Prompt template based on the text description information and the preset attack type, and fine-tune the pre-built large language model to generate a corresponding large language fine-tuning model.

[0071] The inference module 300 is used to input the standardized Prompt template into the large language fine-tuning model to output multimodal spoofing data in a target format, wherein the multimodal spoofing data includes a forged GPS trajectory point sequence, a perturbed IMU acceleration trend, and a disguised image description.

[0072] Optionally, in one embodiment of the present application, a drone anti-deception capability enhancement device 10 based on LLM multimodal data generation in an embodiment of the present application further includes: a speed change constraint module, a filtering module, a disturbance consistency verification module, a calculation module, a camouflage rationality verification module, a construction module and an optimization module.

[0073] Among them, the speed change constraint module is used to obtain the actual flight trajectory corresponding to the target UAV after inputting the standardized Prompt template into the large language fine-tuning model to output multimodal deception data in the target format, and calculate the average offset index between the forged GPS trajectory point sequence in the multimodal deception data and the actual flight trajectory. Based on the average offset index, the target forged GPS trajectory point sequence in the forged GPS trajectory point sequence that meets the preset speed change constraint detection requirements is determined.

[0074] The filtering module is used to obtain each inertial data frame corresponding to the target UAV and perform filtering and alignment operations on each inertial data frame to obtain the corresponding filtered inertial data frame.

[0075] The disturbance consistency check module is used to calculate the Euclidean difference modulus corresponding to each inertial data frame based on the filtered inertial data frame, and use the Euclidean difference modulus to calculate the average disturbance amount of all inertial data frames, so as to perform a disturbance consistency check operation on the disturbance IMU acceleration trend based on the average disturbance amount and a preset maximum tolerance threshold, so as to generate a target disturbance IMU acceleration trend that meets the preset disturbance consistency check requirements.

[0076] The calculation module is used to perform a saliency occlusion extraction operation on the disguised image description to generate a corresponding disguised mask area, and respectively calculate the disguised area and the total image area corresponding to the disguised mask area and the disguised image description.

[0077] The camouflage rationality verification module is used to calculate the corresponding occlusion ratio based on the camouflage area area and the total image area, and perform a preset image camouflage rationality verification operation on the camouflage image description according to the occlusion ratio to obtain a target camouflage image description that meets the image camouflage rationality verification requirements.

[0078] A construction module is used to construct a corresponding deception dataset based on the target camouflage image description, the target perturbation IMU acceleration trend and the target forged GPS trajectory point list.

[0079] The optimization module is used to optimize the standardized Prompt template based on the deception dataset and the preset dynamic optimization mechanism to generate multimodal deception samples, and perform enhanced anti-spoofing training and / or performance evaluation operations on the target UAV through the multimodal deception samples to obtain corresponding training data and / or evaluation results, so as to optimize the performance of the target UAV in executing the preset flight mission based on the training data and / or evaluation results.

[0080] Optionally, in one embodiment of the present application, the optimization module includes: a detection unit, an update unit and a performance evaluation unit.

[0081] Among them, the detection unit is used to detect in real time whether the target drone has successfully implemented a deception attack using the deception data set. When it is detected that the target drone has successfully implemented a deception attack, the multimodal sensor data and the status information of the target drone when the deception attack is successfully implemented are recorded to generate attack feedback data based on the multimodal sensor data and status information.

[0082] The updating unit is used to dynamically adjust the standardized Prompt template based on the attack feedback data, and generate multimodal deception samples according to the dynamically adjusted standardized Prompt template to update the deception dataset through the multimodal deception samples.

[0083] The performance evaluation unit is used to perform enhanced anti-spoofing training and / or performance evaluation operations on the target UAV based on the preset anti-spoofing training strategy, performance evaluation strategy and updated deception dataset.

[0084] Optionally, in one embodiment of the present application, the preprocessing module 100 includes: an acquisition unit, an alignment unit, a dimensionality reduction unit, and a creation unit.

[0085] Among them, the acquisition unit is used to obtain the multimodal raw data collected by the target UAV during the execution of the preset flight mission, wherein the multimodal raw data includes GPS position information, IMU inertial measurement data and airborne camera image sequence.

[0086] The alignment unit is used to perform millisecond-level alignment operations on the GPS position information, IMU inertial measurement data and airborne camera image sequences in the multimodal raw data to obtain GPS position alignment information, IMU inertial measurement alignment data and airborne camera image alignment sequences.

[0087] The dimensionality reduction unit is used to perform low-pass filtering on the IMU inertial measurement alignment data to obtain the corresponding filtered data, extract the speed change features corresponding to the GPS position alignment information and the semantic features corresponding to the airborne camera image alignment sequence, and reduce the dimensionality of the semantic features to generate the corresponding reduced dimensionality semantic features.

[0088] A unit is established for constructing multimodal feature data based on filtered data, speed change features, and dimensionality reduction semantic features.

[0089] It should be noted that the above explanation of an embodiment of a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation is also applicable to an apparatus for improving the anti-spoofing capability of a drone based on LLM multimodal data generation in this embodiment, and will not be repeated here.

[0090] According to an embodiment of the present application, a device for enhancing the anti-spoofing capability of a drone based on LLM multimodal data generation includes a preprocessing module 100 for collecting multimodal raw data from a target drone during a preset flight mission, performing time synchronization and preprocessing operations on the multimodal raw data to obtain corresponding multimodal feature data, and encapsulating the multimodal feature data into a unified structured text description information; a fine-tuning module 200 for constructing a corresponding standardized Prompt template based on the text description information and a preset attack type, and fine-tuning the pre-built large language model to generate a corresponding large language fine-tuning model; and an inference module 300 for inputting the standardized Prompt template into the large language fine-tuning model to output multimodal deception data in a target format, wherein the multimodal deception data includes a forged GPS trajectory point series, a perturbed IMU acceleration trend, and a disguised image description. This application can effectively expand the coverage of attack scenarios, reduce the cost of manually designed samples, and generate deception samples that are closer to real attack logic.

[0091] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include: Memory 401 , processor 402 , and computer programs stored in the memory 401 and executable on the processor 402 .

[0092] When the processor 402 executes the program, a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation provided in the above embodiment is implemented.

[0093] Furthermore, the electronic device further includes: The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0094] The memory 401 is used to store computer programs that can be run on the processor 402 .

[0095] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0096] If the memory 401, processor 402, and communication interface 403 are implemented independently, the communication interface 403, memory 401, and processor 402 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0097] Optionally, in a specific implementation, if the memory 401 , the processor 402 and the communication interface 403 are integrated on a chip, the memory 401 , the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0098] The processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0099] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation is implemented as described above.

[0100] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation.

[0101] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0102] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0103] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0104] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" is any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (not exhaustive) of computer-readable media include: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0105] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logical functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0106] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0107] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0108] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for improving the anti-spoofing capability of drones based on LLM multimodal data generation, characterized in that: The following steps are involved: Collecting multimodal raw data from the target UAV during the execution of a preset flight mission, performing time synchronization and preprocessing operations on the multimodal raw data to obtain corresponding multimodal feature data, and encapsulating the multimodal feature data into unified structured text description information; Based on the text description information and the preset attack type, a corresponding standardized prompt template is constructed, and the pre-built large language model is fine-tuned to generate a corresponding large language fine-tuned model; The standardized Prompt template is input into the large language fine-tuning model to output multimodal spoofing data in a target format, wherein the multimodal spoofing data includes a forged GPS trajectory point sequence, a perturbed IMU acceleration trend, and a disguised image description.

2. The method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation according to claim 1 is characterized in that: After inputting the standardized Prompt template into the large language fine-tuning model to output the multimodal deception data in the target format, the method further includes: Obtaining an actual flight trajectory corresponding to the target UAV, calculating an average offset index between the forged GPS trajectory point sequence in the multimodal spoofing data and the actual flight trajectory, and determining, based on the average offset index, a target forged GPS trajectory point sequence in the forged GPS trajectory point sequence that meets a preset speed change constraint detection requirement; Acquire each inertial data frame corresponding to the target UAV, and perform filtering and alignment operations on each inertial data frame to obtain a corresponding filtered inertial data frame; Calculating a Euclidean difference modulus corresponding to each inertial data frame according to the filtered inertial data frame, and calculating an average disturbance amount of all inertial data frames using the Euclidean difference modulus, and performing a disturbance consistency check operation on the disturbance IMU acceleration trend based on the average disturbance amount and a preset maximum tolerance threshold to generate a target disturbance IMU acceleration trend that meets a preset disturbance consistency check requirement; performing a saliency occlusion extraction operation on the disguised image description to generate a corresponding disguised mask region, and respectively calculating the area of ​​the disguised region corresponding to the disguised mask region and the disguised image description and the total area of ​​the image; Calculating a corresponding occlusion ratio based on the camouflaged region area and the total image area, and performing a preset image camouflage rationality check operation on the camouflaged image description according to the occlusion ratio to obtain a target camouflaged image description that meets image camouflage rationality check requirements; Constructing a corresponding deception dataset based on the target disguised image description, the target perturbed IMU acceleration trend, and the target forged GPS trajectory point sequence; Based on the deception dataset and a preset dynamic optimization mechanism, the standardized Prompt template is optimized to generate multimodal deception samples, and the target drone is subjected to enhanced anti-spoofing training and / or performance evaluation operations using the multimodal deception samples to obtain corresponding training data and / or evaluation results, so as to optimize the performance of the target drone in performing the preset flight mission based on the training data and / or the evaluation results.

3. The method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation according to claim 2 is characterized in that: The step of optimizing the standardized prompt template based on the deception dataset and a preset dynamic optimization mechanism to generate multimodal deception samples, and performing enhanced anti-spoofing training and / or performance evaluation on the target drone using the multimodal deception samples, includes: detecting in real time whether the target UAV has successfully carried out a deception attack using the deception dataset, wherein, when detecting that the target UAV has successfully carried out the deception attack, recording multimodal sensor data and state information of the target UAV when the deception attack was successfully carried out, so as to generate attack feedback data based on the multimodal sensor data and the state information; Dynamically adjusting the standardized prompt template based on the attack feedback data, and generating a multimodal deception sample according to the dynamically adjusted standardized prompt template, so as to update the deception dataset through the multimodal deception sample; Based on the preset anti-spoofing training strategy, performance evaluation strategy and updated deception data set, enhanced anti-spoofing training and / or performance evaluation operations are performed on the target UAV.

4. The method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation according to claim 1 is characterized in that: The method includes collecting multimodal raw data of the target UAV during the execution of a preset flight mission, and performing time synchronization and preprocessing operations on the multimodal raw data to obtain corresponding multimodal feature data, including: Acquire multimodal raw data collected by the target UAV during the execution of the preset flight mission, wherein the multimodal raw data includes GPS position information, IMU inertial measurement data, and an onboard camera image sequence; Performing a millisecond-level alignment operation on the GPS position information, the IMU inertial measurement data, and the airborne camera image sequence in the multimodal raw data to obtain GPS position alignment information, IMU inertial measurement alignment data, and airborne camera image alignment sequence; Low-pass filtering is performed on the IMU inertial measurement alignment data to obtain corresponding filtered data, and speed change features corresponding to the GPS position alignment information and semantic features corresponding to the airborne camera image alignment sequence are extracted, and dimensionality reduction is performed on the semantic features to generate corresponding reduced-dimensionality semantic features; The multimodal feature data is constructed based on the filtered data, the speed change feature and the dimensionality reduction semantic feature.

5. A device for improving the anti-spoofing capability of drones based on LLM multimodal data generation, characterized in that: include: A preprocessing module is used to collect multimodal raw data from the target UAV during the execution of a preset flight mission, perform time synchronization and preprocessing operations on the multimodal raw data to obtain corresponding multimodal feature data, and encapsulate the multimodal feature data into unified structured text description information; A fine-tuning module is used to construct a corresponding standardized prompt template based on the text description information and the preset attack type, and fine-tune the pre-built large language model to generate a corresponding large language fine-tuning model; An inference module is configured to input the standardized Prompt template into the large language fine-tuning model to output multimodal spoofing data in a target format, wherein the multimodal spoofing data includes a forged GPS trajectory point sequence, a perturbed IMU acceleration trend, and a disguised image description.

6. The device for improving the anti-spoofing capability of a drone based on LLM multimodal data generation according to claim 5 is characterized in that: Also includes: a speed change constraint module, configured to, after inputting the standardized Prompt template into the large language fine-tuning model to output multimodal spoofing data in the target format, obtain the actual flight trajectory corresponding to the target UAV, calculate an average offset index between the forged GPS trajectory point sequence in the multimodal spoofing data and the actual flight trajectory, and determine, based on the average offset index, a target forged GPS trajectory point sequence in the forged GPS trajectory point sequence that meets a preset speed change constraint detection requirement; A filtering module is used to obtain each inertial data frame corresponding to the target UAV, and perform filtering and alignment operations on each inertial data frame to obtain a corresponding filtered inertial data frame; a disturbance consistency check module, configured to calculate a Euclidean difference modulus corresponding to each inertial data frame according to the filtered inertial data frame, and calculate an average disturbance amount of all inertial data frames using the Euclidean difference modulus, so as to perform a disturbance consistency check operation on the disturbance IMU acceleration trend based on the average disturbance amount and a preset maximum tolerance threshold, so as to generate a target disturbance IMU acceleration trend that meets preset disturbance consistency check requirements; a calculation module configured to perform a saliency occlusion extraction operation on the disguised image description to generate a corresponding disguised mask region, and respectively calculate the area of ​​the disguised mask region and the area of ​​the disguised region corresponding to the disguised image description and the total area of ​​the image; a camouflage rationality verification module, configured to calculate a corresponding occlusion ratio based on the camouflage region area and the total image area, and perform a preset image camouflage rationality verification operation on the camouflage image description according to the occlusion ratio to obtain a target camouflage image description that meets the image camouflage rationality verification requirements; A construction module is used to construct a corresponding deception dataset based on the target disguised image description, the target perturbed IMU acceleration trend and the target forged GPS trajectory point sequence; An optimization module is configured to optimize the standardized Prompt template based on the deception dataset and a preset dynamic optimization mechanism to generate multimodal deception samples, and perform enhanced anti-spoofing training and / or performance evaluation operations on the target drone using the multimodal deception samples to obtain corresponding training data and / or evaluation results, so as to optimize the performance of the target drone in executing the preset flight mission based on the training data and / or the evaluation results.

7. The device for improving the anti-spoofing capability of a drone based on LLM multimodal data generation according to claim 6 is characterized in that: The optimization module includes: a detection unit, configured to detect in real time whether the target UAV has successfully carried out a deception attack using the deception data set, wherein, upon detecting that the target UAV has successfully carried out a deception attack, recording multimodal sensor data and state information of the target UAV when the deception attack was successfully carried out, so as to generate attack feedback data based on the multimodal sensor data and the state information; an updating unit, configured to dynamically adjust the standardized Prompt template based on the attack feedback data, and generate a multimodal deception sample according to the dynamically adjusted standardized Prompt template, so as to update the deception dataset through the multimodal deception sample; A performance evaluation unit is configured to perform enhanced anti-spoofing training and / or performance evaluation operations on the target UAV based on a preset anti-spoofing training strategy, a performance evaluation strategy, and an updated deception dataset.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation as described in any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation as described in any one of claims 1 to 4.

10. A computer program product comprising a computer program, characterized in that The computer program is executed to implement a method for improving the anti-spoofing capability of a drone based on LLM multimodal data generation as described in any one of claims 1 to 4.

Images