A UAV flight safety assessment method, system, device and storage medium

Abstract

The present invention provides a method for assessing the flight safety of unmanned aerial vehicles, which belongs to the field of unmanned aerial vehicle safety assessment, including: obtaining multimodal data information in an actual battlefield environment, performing unmanned autonomous and safe multimodal representation learning based on the information, constructing a multimodal representation learning strategy based on variational autoencoder based on the learning results, and realizing the optimal extraction of interpretable features of multimodal information in the battlefield environment according to the multimodal representation learning strategy; storing the unlabeled information in the unmanned aerial vehicle in a knowledge-based manner, constructing a hierarchical knowledge base system based on symbolic reasoning based on the stored unlabeled information, and establishing a drone safety knowledge base; constructing a knowledge-driven and data-driven artificial intelligence safety assessment model based on the drone safety knowledge base; and performing multi-domain safety assessment based on the artificial intelligence safety assessment model. The present invention provides a comprehensive artificial intelligence safety assessment model that can assess the safety status of drones in different fields in real time and accurately.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle safety assessment, and in particular relates to a method, system, computer equipment and storage medium for evaluating the flight safety of an unmanned aerial vehicle. Background Art

[0002] With the expansion and increasing complexity of low-altitude application scenarios, the security threats faced by drone systems are also increasing. These threats include not only physical domain attacks, such as signal interference and hardware destruction, but also information domain attacks, such as network intrusion and data tampering.

[0003] When performing tasks, drones need to process a large amount of multimodal data information, including visual images, radar data, infrared signals, etc. The safety of drones is evaluated based on the processing results of multimodal data information. In addition to referring to the processing results of multimodal data information, traditional drone safety assessment methods mainly rely on experience and rules, lack dynamic adaptability and comprehensive safety assessment models, and cannot respond to complex and changing security threats in a timely and accurate manner, which restricts the intelligence level and safety improvement of drone systems. Summary of the invention

[0004] In order to solve the problem that existing UAV safety assessment methods lack dynamic adaptability and a comprehensive safety assessment model, the present invention provides a UAV flight safety assessment method, system, computer device and storage medium.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for evaluating the flight safety of an unmanned aerial vehicle comprises the following steps:

[0007] Obtain multimodal data information in historical battlefield environments and extract interpretable features of multimodal information in historical battlefield environments;

[0008] Feedback and interpretation of interpretable features are performed through the drone safety knowledge base to obtain relational attribute information about drone autonomous safety. Based on the relational attribute information, multiple relational attribute knowledge systems for drone autonomous safety are constructed, and different levels of safety knowledge are structured and stored according to relational attributes. An artificial intelligence safety assessment model is constructed based on multiple relational attribute knowledge systems for drone autonomous safety.

[0009] The multimodal data information obtained in the actual battlefield environment is input into the artificial intelligence safety assessment model, and the drone safety status parameters and drone safety threat parameters are output. The drone flight safety is assessed based on the drone safety status parameters and drone safety threat parameters.

[0010] Preferably, the extracting interpretable features of multimodal information in the historical battlefield environment is specifically:

[0011] The multimodal data information is subjected to unmanned, autonomous and safe multimodal representation learning, a multimodal representation learning strategy based on a variational autoencoder model is constructed through the learning results, and interpretable features of the multimodal information in the historical battlefield environment are extracted according to the multimodal representation learning strategy.

[0012] Preferably, the method for constructing the drone safety knowledge base is: using negative sampling to obtain a basic battlefield description from the unlabeled information existing in the drone, using the context information in the battlefield description to obtain the required battlefield environment knowledge pairs, and storing the battlefield environment knowledge pairs in a knowledge-based manner; constructing a hierarchical knowledge base system based on symbolic reasoning based on the stored unlabeled information to establish a drone safety knowledge base.

[0013] Preferably, in the variational autoencoder model, the interpretable and uninterpretable parts of the implicit variables are split, and corresponding constraints are added for different variables. It is assumed that the model implies two independent variables as non-entangled variables. and unexplained ,but;

[0014] ;

[0015] in, is the encoder in the variational autoencoder, x The safety status information of the drone; Given the safety status information of a drone x , non-entangled quantity v and unexplained z The posterior distribution of Given the safety status information of a drone x and the non-entangled quantity v The posterior distribution of Given the safety status information of a drone x and unexplained z The posterior distribution of .

[0016] Preferably, it also includes conducting adversarial attacks on the artificial intelligence security assessment model through a deep learning attack model based on black box attack and white box attack, specifically including:

[0017] The white-box attack uses an attack method targeting gradient information and other model information. The deep learning attack model directly obtains the gradient information of the artificial intelligence security assessment model and reconstructs the parameter information of the artificial intelligence security assessment model through the gradient information, thereby achieving the attack;

[0018] The black box attack uses a proxy model to verify the original model, and the deep learning attack model uses distillation to indirectly transfer the information of the artificial intelligence security assessment model to the proxy model, thereby realizing the attack on the artificial intelligence security assessment model.

[0019] Preferably, in a black box scenario, an attack method based on a proxy model is adopted, specifically, the proxy model is learned by using knowledge distillation. On the basis of basic distillation, a distillation method based on the two norm is constructed, and the specific form is:

[0020] ;

[0021] in, is the output of the proxy model, M is the target model, To learn the parameters, y is the sample label; is the objective function constructed, is the predicted output of the surrogate model, is the predicted output of the target model; on this basis, the generative model is used to learn the attachment area so that the attack point is attached to the image contour; finally, points with attack characteristics are selected according to the point area, and the generative model is used to learn the attachment area so that the attack point is attached to the image contour; wherein, the whole process uses the generative model to reconstruct the data, and the obtained image is used as the basis for selecting the attack point.

[0022] Preferably, in the white-box attack test, the intelligent model carried by the drone is attacked in an online learning manner; the training data set in the online learning arrives in the form of a data stream, which is represented as , the initial weight is ,exist The weight of the moment is expressed as ;exist The time is based on the current time data Perform iterative updates; the update formula using the online gradient descent algorithm is expressed as:

[0023] ;

[0024] in, It means taking partial derivative of a vector; is the loss function, represents the learning rate; represents the regularization process, middle , They are the initial safety status and the corresponding initial safety level: middle , They are the safety status at time t and the corresponding safety level at time t;

[0025] By iteratively updating the training data, the cumulative objective of the entire online learning window is maximized, that is: ;in , indicating that the iteratively updated training data causes the model to output incorrect results;

[0026] in, represents the intelligent model carried by the drone; T represents the sequence length of the time series, T' represents the transposition operation of the matrix, represents the expectation, D represents the training data set, Represents the data of the training set; Represents the model parameters of the intelligent model carried by the drone;

[0027] In the white-box attack test, a reverse verification set is also constructed , including each Counterexample , and then maximize the negative value of the logistic loss on this reverse validation set, thereby optimizing the objective function as:

[0028] ;

[0029] In the formula, S is the target subset to be solved by optimization, Represents the constructed loss function;

[0030] Indicates that the loss of all data in the reverse validation set is calculated and summed for optimization. Represents the constructed loss function;

[0031] Introduce the chain rule to calculate the gradient of the objective function, which is specifically expressed as:

[0032] ;

[0033] In the formula, x i represents the training data at the current moment, i Represents an index variable used to distinguish different input elements or different stages in the calculation process. t represents the tth moment, Express x The partial derivative of .

[0034] The present invention also provides a UAV flight safety assessment system, comprising:

[0035] The feature extraction module is used to obtain multimodal data information in the historical battlefield environment, conduct unmanned, autonomous and safe multimodal representation learning on the obtained multimodal data information, build a multimodal representation learning strategy based on the variational autoencoder model through the learning results, and extract interpretable features of multimodal information in the historical battlefield environment according to the multimodal representation learning strategy;

[0036] The evaluation model building module is used to interpret the interpretable features through the drone safety knowledge base, obtain the relational attribute information about the autonomous safety of drones, build multiple relational attribute knowledge systems for autonomous safety of drones based on the relational attribute information, and store the safety knowledge at different levels in a structured manner according to the relational attributes; build an artificial intelligence safety evaluation model based on multiple relational attribute knowledge systems for autonomous safety of drones;

[0037] The safety assessment module is used to input the multimodal data information obtained in the actual battlefield environment into the artificial intelligence safety assessment model, output the UAV safety status parameters and the UAV safety threat parameters, and perform the UAV flight safety assessment based on the UAV safety status parameters and the UAV safety threat parameters.

[0038] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement any one of the steps in the UAV flight safety assessment method.

[0039] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is loaded by a processor, it can execute any one of the steps in the UAV flight safety assessment method.

[0040] The UAV flight safety assessment method provided by the present invention has the following beneficial effects:

[0041] The present invention uses the drone safety knowledge base to interpret the interpretable features and obtains the relational attribute information about the autonomous safety of the drone. The interpretable features can be matched and associated with the relevant knowledge in the knowledge base through feedback interpretation, and then multiple relational attribute knowledge systems for autonomous safety of drones are constructed according to the relational attribute information. The knowledge system can store different levels of security knowledge in a structured manner according to their relational attributes with autonomous safety of drones. Based on this, an artificial intelligence security assessment model is constructed. Based on the knowledge obtained from the knowledge base and the constructed relational attribute knowledge system, the artificial intelligence security assessment model can make inferences and decisions based on this knowledge, no longer relying on experience and rules, improving the accuracy and reliability of system decisions, and also increasing the interpretability of the decision-making process. It can evaluate the safety status of drones in different fields in real time and accurately, identify and respond to security threats in a timely manner, and ensure the safety of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 A flow chart of the UAV flight safety assessment method provided by the present invention;

[0044] Figure 2 It is a schematic diagram of the relationship between the drone knowledge base and safety knowledge; Figure 2 (a) is a schematic diagram of the drone knowledge base. Figure 2 (b) is a schematic diagram of security knowledge relationships;

[0045] Figure 3 It is a schematic diagram of the attack model; Figure 3 (a) is a schematic diagram of the white box attack model. Figure 3 (b) is a schematic diagram of the black box attack model;

[0046] Figure 4 This is a logical block diagram of the UAV flight safety assessment system provided by the present invention. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0048] Example 1

[0049] The present invention aims to improve the information processing capability and safety assessment level of UAV systems in complex battlefield environments through advanced multimodal representation learning strategies, knowledge base construction methods, and artificial intelligence safety assessment models. Through deep learning and feature extraction of multimodal data, efficient fusion and interpretation of multimodal information of UAVs can be achieved; through knowledge storage and hierarchical knowledge base construction, systematic UAV safety knowledge support can be provided; through the combination of knowledge-driven and data-driven methods, multi-domain safety assessment of UAV systems can be achieved to ensure their safety and reliability in complex environments.

[0050] To achieve the above objectives, the overall concept of the present invention is: to obtain multimodal data information in an actual battlefield environment, and to perform unmanned, autonomous and safe multimodal representation learning on the multimodal data information. Through the learning results, a multimodal representation learning strategy based on variational autoencoders is constructed. The optimal extraction of interpretable features of multimodal information in the battlefield environment is achieved, and at the same time, the global interpretable strategy is studied to ensure the extraction of important interpretable features in the battlefield environment, and to achieve global representation and local interpretable representation learning in the drone battlefield environment. Through the constructed knowledge base system, the interpretable features are fed back and interpreted to achieve intelligent representation learning of drones.

[0051] The massive amount of unlabeled information in drones is stored in a knowledge-based manner, and a hierarchical knowledge base system model based on symbolic reasoning is constructed. It is iteratively expanded from meta-feature reasoning to construct multiple knowledge systems of relational attributes for autonomous safety of drones. This knowledge base stores safety knowledge at different levels in a structured manner, providing a solid foundation for subsequent safety assessments.

[0052] Based on the drone safety knowledge base, a knowledge-driven and data-driven AI safety assessment model is constructed. Explainability in this application refers to the ability to understand the decision-making process and output results of the AI ​​system. This AI safety assessment model combines symbolic reasoning and data-driven methods to improve the system's interpretability and reliability, so that drones can make more accurate and transparent decisions in complex environments. This AI safety assessment model is used for multi-domain safety assessments.

[0053] Specifically, the present invention proposes a method for evaluating the flight safety of an unmanned aerial vehicle, such as Figure 1 As shown, the following steps are included:

[0054] S1. Collect multimodal data information in historical battlefield environments, where the multimodal data information includes visual images, radar data, and infrared signals.

[0055] S2. Perform unmanned, autonomous and safe multimodal representation learning on the acquired multimodal data information, and construct a multimodal representation learning strategy based on the variational autoencoder model through the learning results. Optimally extract the interpretable features of multimodal information in the battlefield environment according to the multimodal representation learning strategy. Interpretable features refer to certain features that have a strong correlation with the target during the feature extraction process and can directly serve as the basis for reasoning about the target. This application defines such features as interpretable features, and interpretable features can effectively reason about the results. The interpretable features in the present invention include: environmental information (terrain, weather, surrounding environment), target identifiers (vehicles, personnel), target movement trajectories and related communication signals (electromagnetic signals, communication signals).

[0056] S3. Store the unlabeled information in the drone in a knowledgeable manner. Specifically, negative sampling is used to obtain the basic battlefield description from the drone's unlabeled information, and the required battlefield environment knowledge pairs are obtained using the context information in the battlefield description, and the battlefield environment knowledge pairs are stored in a knowledgeable manner. A hierarchical knowledge base system based on symbolic reasoning is constructed based on the stored unlabeled information to establish a drone safety knowledge base.

[0057] S4. Feedback and interpret the interpretable features through the drone safety knowledge base, match and associate the interpretable features with the knowledge in the knowledge base, determine the impact category and relationship attributes on the autonomous safety of drones, and obtain the relationship attribute information about the autonomous safety of drones. According to the relationship attribute information, multiple relationship attribute knowledge systems for autonomous safety of drones are constructed, and different levels of safety knowledge are structured according to the relationship attributes (that is, the relationship attribute knowledge system for autonomous safety of drones stores different levels of safety knowledge in a structured manner).

[0058] Specifically, the feedback interpretation of the interpretable features is to analyze the extracted interpretable features through the knowledge stored in the drone safety knowledge base. The knowledge in the knowledge base covers different levels of safety information, from basic battlefield descriptions to more abstract safety concepts. The interpretation content includes:

[0059] Safety knowledge association: clarify the association between the explainable features and the different levels of safety knowledge in the knowledge base. A certain explainable feature may be associated with specific environmental condition knowledge, and determining this association is part of the interpretation result.

[0060] Classification of impact on autonomous safety: Based on the knowledge in the knowledge base, determine the different impact categories of explainable features on the autonomous safety of drones. Some features may affect the flight stability of the drone (related to the device status factor), while others may affect its ability to perceive the environment (related to the environmental factor), etc.

[0061] Determination of relational attributes: Determine the relational attributes between each interpretable feature and between them and autonomous safety of drones. These relational attributes may include causal relationships, mutual constraints, etc., thus providing a basis for building a relational attribute knowledge system for autonomous safety of drones.

[0062] During the interpretation process, the interpretable features are matched and associated with the relevant knowledge in the knowledge base. If the interpretable features involve a certain environmental condition, the relevant environmental safety knowledge and the impact of environmental safety knowledge on the autonomous safety of drones will be searched in the knowledge base. Based on the results of these matching and associations, multiple relational attribute knowledge systems for autonomous safety of drones are constructed.

[0063] The knowledge system stores safety knowledge at different levels in a structured manner according to their relationship attributes with drone autonomous safety, and stores them in categories such as environmental factors, equipment status factors, and operational behavior factors, so that this knowledge can be used more effectively when subsequently building an artificial intelligence safety assessment model.

[0064] S5. Construct a knowledge-driven and data-driven AI safety assessment model based on multiple relational attribute knowledge systems for autonomous safety of drones. Combining the knowledge in the knowledge base with the features and information learned from the data, the AI ​​safety assessment model is equipped with reasoning and decision-making capabilities.

[0065] In this embodiment, knowledge-driven means that the present invention constructs a drone safety knowledge base, which includes a hierarchical knowledge base system constructed by storing drone unlabeled information in a knowledge-based manner and based on symbolic reasoning. This knowledge covers different levels from basic battlefield descriptions to various safety-related concepts. When performing safety assessments and other operations, the knowledge in the knowledge base is fully utilized to understand and handle related issues.

[0066] Knowledge-driven emphasizes the use of existing knowledge systems to guide the decision-making process of the entire system. When building an AI security assessment model, based on the knowledge obtained from the knowledge base and the constructed relational attribute knowledge system, the AI ​​security assessment model can make inferences and decisions based on this knowledge, improve the accuracy and reliability of system decisions, and also increase the explainability of the decision-making process.

[0067] Data-driven means that the present invention focuses on the acquisition and utilization of multimodal data information (such as visual images, radar data, and infrared signals) in actual battlefield environments. By learning these multimodal data, such as multimodal representation learning for unmanned autonomous safety, features and information in the data are extracted. These features and information learned from the data will participate in the entire security assessment process.

[0068] S6. Obtain multimodal data information in an actual battlefield environment, input the multimodal data information in the actual battlefield environment into an artificial intelligence safety assessment model, output UAV safety status parameters and UAV safety threat parameters, and conduct a UAV flight safety assessment based on the UAV safety status parameters and UAV safety threat parameters.

[0069] In this embodiment, the drone safety status parameter reflects the comprehensive safety level of the drone in the current actual battlefield environment, and the drone safety threat parameter can identify the types of security threats that may be currently faced, such as signal interference, hardware destruction, network intrusion, data tampering, etc.

[0070] Specifically, in the process of using the variational autoencoder model for building a multimodal representation learning strategy, this embodiment processes the implicit variables in a more detailed manner, specifically by splitting the interpretable and uninterpretable parts of the implicit variables. By splitting the implicit variables into interpretable parts (non-entangled quantities) and uninterpretable parts (uninterpretable quantities), and adding corresponding constraints to different parts, the model can better capture the features and information in the data. In the model, it is assumed that there are two independent variables implied in the model. and ,but:

[0071] .

[0072] in, is the encoder in the variational autoencoder, x The safety status information of the drone; Given the safety status information of a drone x , non-entangled quantity v and unexplained z The posterior distribution of Given the safety status information of a drone x and the non-entangled quantity v The posterior distribution of Given the safety status information of a drone x and unexplained z The posterior distribution of 。 Define two variables with different functions, the non-entangled variable v is used to capture the representation of category information. The uninterpretable variable z is used to learn all representations other than category information, and the learning objective function is constructed based on this:

[0073] .

[0074] The above structure improves the interpretability of the model. This splitting and constraining method helps to improve the transparency and interpretability of the model. When learning and extracting features from multimodal information in battlefield environments, it is possible to have a clearer understanding of how the model processes and utilizes this information. For example, the non-entangled amount may be used to capture specific categories of information related to security, while the uninterpretable amount is used to learn feature representations in other aspects.

[0075] At the same time, the model structure provides a basis for subsequent semi-supervised learning based on the model and possible reinforcement learning modules. By clarifying the nature and constraints of the variables, the learning objective function can be better constructed, a more effective learning process can be achieved, and the extracting of interpretable features of multimodal information in the battlefield environment can be optimized.

[0076] Furthermore, the task-based UAV knowledge base construction technology of the present invention stores the massive unlabeled information in the UAV in a knowledge-based manner, combines the hierarchical knowledge base system construction model based on symbolic reasoning, and iterates from meta-feature reasoning to realize the reasoning construction of the knowledge base system. Figure 2 As shown in the figure, in the knowledge base system, multiple knowledge systems of relational attributes for autonomous safety of drones will be constructed. Figure 2 (a) is a schematic diagram of the drone knowledge base, which is divided into three layers: named entity, concept and meta-knowledge. The named entity layer includes vegetation, factory buildings, sandstorms, fuel, engines, etc. The concept layer is the surface, weather, temperature, drones, etc. obtained after classification by the named entity layer. The meta-knowledge layer is the three types of knowledge of environment, state and model that rise from the concept layer. Figure 2 (b) is a schematic diagram of safety knowledge relationships. The safety knowledge relationships in the figure are composed of environmental information, state variables and degree information. Environmental information includes temperature, humidity and altitude, state information includes equipment integrity, damage and availability, and degree information includes equipment integrity, damage and disability.

[0077] The present invention proposes an automated model for constructing a knowledge base for the combat field, which uses negative sampling to obtain basic battlefield descriptions from massive text information, fully utilizes the contextual information in the battlefield description, obtains the required battlefield environment knowledge pairs, and realizes efficient construction of a UAV knowledge base, providing a foundation and basis for interpretable research on UAVs.

[0078] According to the requirements for the robustness of drone safety, the present invention also verifies the proposed security assessment method through adversarial attack strategies. Deep learning attack models based on black-box attacks and white-box attacks are designed respectively to conduct adversarial attacks on the artificial intelligence security assessment model proposed above, and verify the robustness of the artificial intelligence security assessment model. For white-box attacks, an attack method targeting gradient information and other model information is designed. The attack model directly obtains the gradient information of the artificial intelligence security assessment model, and reconstructs the parameter information of the artificial intelligence security assessment model through the gradient information, thereby achieving the attack. For black-box attacks, a proxy model is used to verify the original model. Among them, The proxy model is mainly used to simulate the attack target, and ADV is the attack model, that is, the model that implements the attack. The proxy model and the attack model will not be able to obtain the intrinsic parameters and structure of the artificial intelligence security assessment model. Therefore, the present invention adopts the distillation method to indirectly transfer the information of the artificial intelligence security assessment model to the proxy model, thereby realizing the attack of the model. Through the above two methods of attack, the robustness of the artificial intelligence security assessment model is verified respectively, as shown in the following example. Figure 3 As shown, Figure 3(a) is a schematic diagram of a white-box attack model, where the attack results include a first attack result Y' and a second attack result Y. Figure 3 (b) is a schematic diagram of a black box attack model, where the attack results include a first attack result Y', a second attack result Y and a third attack result Y''.

[0079] In the black box scenario, we will use an attack method based on a proxy model. In order to obtain a robust proxy model, we will use knowledge distillation to learn the proxy model so that the proxy model can infinitely approach the AI ​​security assessment model, thereby ensuring that attacks on the proxy model can be well migrated to the AI ​​security assessment model. In order to improve the learning ability of the proxy model, we will build a distillation method based on the two norms on the basis of basic distillation. The basic process is as follows:

[0080] .

[0081] in, is the output of the proxy model, M is the target model, To learn the parameters, y is the sample label; is the objective function constructed, is the predicted output of the surrogate model, is the predicted output of the target model. Based on the above learning, the proxy model can achieve 90% of the performance of the artificial intelligence security assessment model. On this basis, the generative model is used to learn the attachment area to ensure that the attack point is attached to the image contour to achieve the concealment of the attack. Finally, points with attack characteristics are selected according to the point area. The generative model is used to learn the attachment area to ensure that the attack point is attached to the image contour to achieve the concealment of the attack. In this process, the generative model is used to reconstruct the data, and the obtained image is used as the basis for selecting the attack point.

[0082] When learning a general model, it is necessary to ensure that the anti-sample can make the model output the wrong classification result, and the higher the probability of the wrong category, the better. However, this method is prone to overfitting, misclassifying all categories into 1. In order to deal with such problems, the present invention proposes to construct a new adversarial sample learning method, which makes the probability of misclassification higher than all other categories and has stronger generalization ability.

[0083] In the white box attack test, the intelligent model carried by the drone will be attacked in the form of online learning. The training data set in online learning arrives in the form of data stream and can be expressed as , the initial weight is ,exist The weight of the moment is expressed as .exist The time is based on the current time data Perform iterative updates. The update formula using the online gradient descent algorithm can be expressed as:

[0084] .

[0085] in, It means taking partial derivative of a vector; is the loss function, represents the learning rate, which usually decreases over time, represents regularization processing, T represents the length of the data stream, middle , They are the initial safety status and the corresponding initial safety level: middle , They are the security state at time t and the corresponding security level at time t. The goal of efficient poisoning attack in online learning is to maximize the cumulative target of the entire online learning window by iteratively updating the training data, that is: .in , indicating that the iteratively updated training data causes the model to output incorrect results. The purpose of a poisoning attack is to mislead the model's learning process by updating the iterative training data, causing the model to produce incorrect output results. The attacker will carefully design and tamper with some training data to introduce deviations in the model's learning process.

[0086] in, represents the intelligent model carried by the drone; T represents the sequence length of the time series, T' represents the transposition operation of the matrix, represents the expectation, D represents the training data set, Represents the data of the training set; Represents the model parameters of the intelligent model carried by the drone.

[0087] First, in order to further optimize the objective function (the objective function is the intelligent model carried by the drone ), the present invention uses the logistic loss to smooth the objective function and performs a separate validation dataset Evaluation expectation. At the same time, in order not to maximize the logical loss of the validation set, the present invention constructs a reverse validation set , which contains each Counterexample , and then maximize the negative value of the logistic loss on this reverse validation set. Thus, the objective function is optimized as:

[0088] .

[0089] In the formula,S is the target subset to be solved by optimization, Represents the constructed loss function. Indicates that the loss of all data in the reverse validation set is calculated and summed for optimization. Represents the constructed loss function.

[0090] Secondly, in order to calculate the gradient in the objective function, the present invention introduces the chain rule, which can be specifically expressed as:

[0091] .

[0092] In the formula, x i represents the training data at the current moment, i Represents an index variable used to distinguish different input elements or different stages in the calculation process. t represents the tth moment, Express x The partial derivative of .

[0093] Finally, the present invention proposes an interval attack method to select points in the data stream for attack. The attack sequence length is limited to K, and the target function is calculated by serial connection. About Sequence The gradient of is used to select sequences greater than a given threshold for attack, thereby reducing the search space and ensuring the attack effect. The effectiveness of the attack is used to evaluate the degree of certainty of the intelligent software.

[0094] Through the above attack scheme, the artificial intelligence security assessment model can be:

[0095] 1. Verify the robustness of the AI ​​security assessment model. The AI ​​security assessment model is attacked through a deep learning attack model based on black-box attack and white-box attack. The main purpose is to verify the robustness of the model.

[0096] 2. Discover potential weaknesses. The attack method can help discover potential weaknesses that may exist in the design and implementation of the AI ​​security assessment model.

[0097] 3. Optimize the design of artificial intelligence security assessment model. According to the attack results, the artificial intelligence security assessment model can be optimized and improved to enhance the reliability and security of the entire drone flight safety assessment system.

[0098] The present invention has dynamic adaptability and can dynamically adjust safety assessment and control strategies according to real-time data and knowledge base information, thereby improving the adaptability and response speed of the UAV system in complex environments. By combining knowledge storage and symbolic reasoning with data-driven methods, the intelligence level of the UAV system is improved, enabling it to demonstrate higher autonomous decision-making and execution capabilities in diverse and complex tasks.

[0099] By comprehensively considering the security threats in the physical domain and information domain, the security of the UAV system under various potential attacks and interferences is evaluated. This multi-domain security assessment method can not only identify and respond to security threats in a timely manner, but also provide optimized security strategies to ensure the stable and safe operation of the UAV system in a complex battlefield environment.

[0100] The present invention realizes the accurate fusion of multiple sensor data (i.e., multimodal data information) through advanced multimodal representation learning strategies, and improves the perception ability and decision-making accuracy of drones; uses a variational autoencoder-based method to efficiently extract meaningful features from complex data, and enhances the autonomy and reliability of drones in complex environments; in addition, a hierarchical knowledge base system is constructed by using knowledge storage and symbolic reasoning methods, and an artificial intelligence security assessment model is constructed based on this, which improves the transparency and interpretability of the system decision-making process; at the same time, combining knowledge-driven and data-driven methods, the artificial intelligence security assessment model can evaluate the security status of drones in different fields in real time and accurately, identify and respond to security threats in a timely manner, and ensure the security of the system. Finally, through adversarial attack verification, black box and white box attack models are designed to verify the robustness of the artificial intelligence security assessment model, so that the assessment model can accurately integrate data, improve the perception and decision-making ability of drones, enhance autonomy and reliability, improve system transparency and interpretability, and ensure the safety of drones in different fields.

[0101] Based on the same inventive concept, the present invention also provides a UAV flight safety assessment system, such as Figure 4 As shown, it includes a feature extraction module 101, an evaluation model construction module 102 and a security evaluation module 103.

[0102] Specifically, the feature extraction module 101 is used to obtain multimodal data information in the historical battlefield environment, perform unmanned, autonomous and safe multimodal representation learning on the acquired multimodal data information, construct a multimodal representation learning strategy based on the variational autoencoder model through the learning results, and extract interpretable features of the multimodal information in the historical battlefield environment according to the multimodal representation learning strategy.

[0103] The evaluation model construction module 102 is used to interpret the interpretable features through the drone safety knowledge base, obtain the relational attribute information about the autonomous safety of the drone, construct multiple relational attribute knowledge systems for the autonomous safety of the drone based on the relational attribute information, and store the safety knowledge at different levels in a structured manner according to the relational attributes; and construct an artificial intelligence safety evaluation model based on multiple relational attribute knowledge systems for the autonomous safety of the drone.

[0104] The safety assessment module 103 is used to input the multimodal data information obtained in the actual battlefield environment into the artificial intelligence safety assessment model, output the drone safety status parameters and the drone safety threat parameters, and perform the drone flight safety assessment based on the drone safety status parameters and the drone safety threat parameters.

[0105] Each module in the above-mentioned UAV flight safety assessment system can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0106] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the embodiment of the method for evaluating the flight safety of a drone. The specific implementation method can be found in the method embodiment, which will not be described in detail here.

[0107] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the above instructions can be executed by a processor of a computer device to complete the above method. For example, a non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in the embodiment of the drone flight safety assessment method can be implemented. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0108] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] It should be pointed out that the specific implementation methods described above can enable those skilled in the art to understand the invention more comprehensively, but do not limit the invention in any way. Therefore, although the invention has been described in detail in this specification and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved. Any simple change or equivalent replacement of the technical solution that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belongs to the protection scope of the present invention.

Claims

1. A method for evaluating the flight safety of an unmanned aerial vehicle, characterized in that: The following steps are involved: Acquire multimodal data information in historical battlefield environments, and extract interpretable features of the multimodal information in historical battlefield environments; the multimodal information includes visual images, radar data, and infrared signals, and the interpretable features include environmental information, target identifiers, target movement trajectories, and related communication signals; Through the UAV safety knowledge base, the interpretable features are fed back and interpreted to obtain the relational attribute information about the autonomous safety of UAVs. Based on the relational attribute information, multiple relational attribute knowledge systems for autonomous safety of UAVs are constructed, and the safety knowledge at different levels is stored in a structured manner according to the relational attributes. Construct an artificial intelligence safety assessment model based on multiple relational attribute knowledge systems for autonomous safety of drones; The multimodal data information acquired in the actual battlefield environment is input into the artificial intelligence safety assessment model, and the drone safety status parameters and drone safety threat parameters are output, and the drone flight safety assessment is performed based on the drone safety status parameters and drone safety threat parameters; It also includes adversarial attacks on artificial intelligence security assessment models through deep learning attack models based on white-box attacks, including: The white-box attack uses an attack method targeting gradient information and other model information. The deep learning attack model directly obtains the gradient information of the artificial intelligence security assessment model and reconstructs the parameter information of the artificial intelligence security assessment model through the gradient information to achieve the attack. In the white-box attack test, the intelligent model carried by the drone is attacked in the form of online learning; the training data set in online learning arrives in the form of data stream, represented as {(x0,y0),...,(x t ,y t ),...}, the initial weight is ω0, and the weight at time t is ω t ; At time t, according to the current data (x t ,y t ) is iteratively updated; the update formula using the online gradient descent algorithm is expressed as: in, represents the partial derivative of the vector; l is the loss function, η t represents the learning rate; Ω represents the regularization process, and x0 and y0 in (x0, y0) are the initial safety state and the corresponding initial safety level respectively: (x t ,y t ) t ,y t They are the safety status at time t and the corresponding safety level at time t; By iteratively updating the training data, the cumulative objective of the entire online learning window is maximized, that is: where f(ω) = E (x,y)～D [sgn(ω Τ' x)≠y], indicating that the iteratively updated training data causes the model to output incorrect results; Among them, f(ω t ) represents the intelligent model carried by the drone; T represents the sequence length of the time series, T' represents the transposition operation of the matrix, E (x,y)～D represents the expectation, D represents the training data set, (x, y) represents the data of the training set; ω Τ 'Represents the model parameters of the intelligent model carried by the drone; In the white-box attack test, a reverse verification set S is also constructed validinv , contains each (x, y) counterexample (x, -y) in the validation set, and then maximizes the negative value of the logistic loss on this reverse validation set, thereby optimizing the objective function as: In the formula, S is the target subset obtained by optimization to be solved, and L represents the constructed loss function; Indicates that the loss calculation and sum optimization are performed on all the data in the reverse validation set, L(ω T (x,y)) represents the constructed loss function; Introduce the chain rule to calculate the gradient of the objective function, which is specifically expressed as: In the formula, x i represents the training data at the current moment, i represents an index variable used to distinguish different input elements or different stages in the calculation process, t represents the tth moment, represents the partial derivative with respect to x; The method for constructing the drone safety knowledge base is as follows: using negative sampling to obtain a basic battlefield description from the unlabeled information in the drone, using the context information in the battlefield description to obtain the required battlefield environment knowledge pairs, and storing the battlefield environment knowledge pairs in a knowledge-based manner; constructing a hierarchical knowledge base system based on symbolic reasoning based on the stored unlabeled information to establish a drone safety knowledge base.

2. The method for evaluating the flight safety of an unmanned aerial vehicle according to claim 1, characterized in that: The extracting of interpretable features of multimodal information in historical battlefield environments is specifically: The multimodal data information is subjected to unmanned, autonomous and safe multimodal representation learning, a multimodal representation learning strategy based on a variational autoencoder model is constructed through the learning results, and interpretable features of the multimodal information in the historical battlefield environment are extracted according to the multimodal representation learning strategy.

3. The method for evaluating the flight safety of an unmanned aerial vehicle according to claim 1, characterized in that: In the variational autoencoder model, the explainable and unexplainable parts of the implicit variables are split, and corresponding constraints are added for different variables. Assuming that the model implies two independent variables, namely, the non-entangled variable v and the unexplained variable z, then; Q φ (v,z|x)=Q φ (v|x)Q φ (z|x); Among them, Q φ is the encoder in the variational autoencoder, x is the safety status information of the drone; Q φ (v,z|x) is the posterior distribution of the given drone safety status information x, the non-entangled quantity v and the unexplained quantity z, Q φ (v|x) is the posterior distribution of the given drone safety state information x and the non-entanglement quantity v, Q φ (z|x) is the posterior distribution of the given drone safety status information x and the unexplainable quantity z.

4. The method for evaluating the flight safety of an unmanned aerial vehicle according to claim 1, characterized in that: It also includes adversarial attacks on artificial intelligence security assessment models through deep learning attack models based on black box attacks, including: The black box attack uses a proxy model to verify the original model, and the deep learning attack model uses distillation to indirectly transfer the information of the artificial intelligence security assessment model to the proxy model, thereby realizing the attack on the artificial intelligence security assessment model.

5. The method for evaluating the flight safety of an unmanned aerial vehicle according to claim 4, characterized in that: In the black box scenario, an attack method based on a proxy model is used. Specifically, the proxy model is learned by using knowledge distillation. On the basis of basic distillation, a distillation method based on the two norm is constructed. The specific form is: in, is the output of the proxy model, M is the target model, x' is the learning parameter, and y is the sample label; L mi is the objective function constructed, is the predicted output of the proxy model, and M(x') is the predicted output of the target model; on this basis, the generative model is used to learn the attachment area so that the attack point is attached to the image contour; finally, points with attack characteristics are selected according to the point area, and the generative model is used to learn the attachment area so that the attack point is attached to the image contour; wherein, the whole process uses the generative model to reconstruct the data, and the obtained image is used as the basis for selecting the attack point.

6. A system for implementing the UAV flight safety assessment method according to claim 1, characterized in that: include: The feature extraction module is used to obtain multimodal data information in the historical battlefield environment, conduct unmanned, autonomous and safe multimodal representation learning on the obtained multimodal data information, build a multimodal representation learning strategy based on the variational autoencoder model through the learning results, and extract interpretable features of multimodal information in the historical battlefield environment according to the multimodal representation learning strategy; The evaluation model building module is used to interpret the interpretable features through the drone safety knowledge base, obtain the relational attribute information about the autonomous safety of drones, build multiple relational attribute knowledge systems for autonomous safety of drones based on the relational attribute information, and store the safety knowledge at different levels in a structured manner according to the relational attributes; Construct an artificial intelligence safety assessment model based on multiple relational attribute knowledge systems for autonomous safety of drones; The safety assessment module is used to input the multimodal data information obtained in the actual battlefield environment into the artificial intelligence safety assessment model, output the UAV safety status parameters and the UAV safety threat parameters, and perform the UAV flight safety assessment based on the UAV safety status parameters and the UAV safety threat parameters.

7. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the drone flight safety assessment method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded by a processor, it can execute the steps of the UAV flight safety assessment method described in any one of claims 1 to 5.

Images