Airborne machine learning system security assessment method, device, equipment, and storage medium

Abstract

The present invention relates to the field of aircraft design technology and discloses a safety assessment method, apparatus, device, and storage medium for an airborne machine learning system. The method comprises: obtaining an airborne predictive neural network model, which is used to predict airborne equipment characteristics or flight scenario characteristics; and calculating the failure rate of the neural network model using the PAC-Bayes generalization boundary theory and an improved method based on conditional Gaussian distribution. The present invention proposes using the generalization boundary theory to calculate the generalization error and uncertainty introduced by the machine learning model during its operation phase, and using a fault tree to calculate the top-level failure rate of the equipment to complete the airworthiness safety assessment of the airborne machine learning system.

Description

Technical Field

[0001] The present application relates to the field of aircraft design technology, and more particularly to a method, apparatus, device, and storage medium for safety assessment of an airborne machine learning system. Background Art

[0002] As a hub for cutting-edge technologies, the aviation sector has also begun exploring and applying artificial intelligence technologies. The European Union Aviation Safety Agency (EASA) released its AI roadmap in February 2020. Figure 1 .0, launching artificial intelligence route in April 2023 Figure 3 As a supplement to 1.0, ML (machine learning) applications in airborne systems are categorized into three phases based on their safety impact: human-assisted (LEVEL 1), human-collaborative (LEVEL 2), and fully autonomous (LEVEL 3). Airworthiness certification for pilot-assistance functions is planned to be completed by 2025. Simultaneously, several related projects have been launched to explore airworthiness requirements and compliance standards. These include collaborations with Airbus, Daedalean, and other companies to research the challenges and requirements of LEVEL 1 machine learning applications for airworthiness certification, based on pilot-developed vision-based runway recognition and detection, and drone perception and avoidance. Furthermore, the A350 aircraft has been retrofitted with visual cameras and linked to autonomous driving, demonstrating automated takeoff, landing, and taxiing capabilities based on vision recognition technology in a Toulouse demonstration. With the impending emergence of machine learning-based airborne systems and equipment both domestically and internationally, there is an urgent need to clarify the safety assurance technologies for these products and promote the development of AI applications.

[0003] However, machine learning, one of the most intelligent and cutting-edge research fields in artificial intelligence, differs from traditional airborne system / equipment development in that it treats the system / equipment as a "black box." Training, validation, and testing are performed using massive amounts of data, fitting input-output relationships and understanding and building models of the system's expected behavior. This approach is characterized by being "data-driven," having hidden mechanisms, and having complex models. On the one hand, it supports autonomous decision-making and other functions unattainable by traditional systems, bringing tremendous development opportunities to the aviation sector. On the other hand, assessing the credibility of machine learning data and models poses unprecedented challenges to aviation safety and airworthiness certification. The current airborne equipment development safety assurance system, comprised of standards such as SAE ARP4754A, ARP4761, RTCADO-178C, and RTCA DO-254, widely recognized by international airworthiness authorities, is no longer fully applicable. There is an urgent need to research airworthiness certification techniques for the application of machine learning technologies in airborne systems and develop relevant standards to ensure flight safety.

[0004] Especially in the quantitative assessment of system safety, quantitative assessments based on the ARP4761 standard only consider reliability issues caused by random hardware failures. While machine learning models are deterministic from a mathematical perspective, such as fixed weights in a neural network (NN), for any new input during the inference run, the output will depend on the correlation between that input and the data set used for the design process. This, along with the inherent complexity of AI models, can lead to unpredictable outputs that are difficult to interpret and produce generalization errors, posing challenges to traditional safety assessments. Table 1 compares the characteristics of classic airborne systems and airborne systems that incorporate machine learning, as well as existing airworthiness compliance methods. For high-safety airborne systems, this uncertainty needs to be quantitatively assessed to provide appropriate generalization guarantees for machine learning models. Summary of the Invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The embodiments of the present disclosure provide a method, apparatus, device, and storage medium for safety assessment of an airborne machine learning system, which are used to address the problem in the prior art that airborne predictive neural network models may result in unpredictable outputs, thereby generating generalization errors.

[0007] In some embodiments, a method for safety assessment of an airborne machine learning system is provided, the method comprising:

[0008] Acquiring an airborne prediction neural network model, wherein the airborne prediction neural network model is used to predict airborne equipment characteristics or flight scenario characteristics;

[0009] The failure rate of the on-board prediction neural network model is calculated using the PAC-Bayes generalization bound theory based on conditional Gaussian improvement.

[0010] Preferably, obtaining an onboard prediction neural network model includes:

[0011] Get the signal time-frequency diagram dataset;

[0012] The convolutional neural network is trained using the signal time-frequency graph dataset to generate a neural network model for signal time-frequency graph recognition.

[0013] Preferably, the convolutional neural network is trained using a signal time-frequency graph dataset to generate a neural network model for signal time-frequency graph recognition, including:

[0014] The signal time-frequency graph dataset is divided into a training set and a test set;

[0015] Build the architecture of a convolutional neural network;

[0016] Initialize the parameter distribution of the convolutional neural network recognition model;

[0017] The convolutional neural network is trained and tested using the training set and test set to complete the model's recognition of the signal time-frequency diagram and generate a neural network model for signal time-frequency diagram recognition.

[0018] Preferably, the architecture of the convolutional neural network is constructed, including:

[0019] The network architecture consists of 4 convolutional layers, 4 pooling layers, 4 ReLU activation functions φ, 2 linear layers and 1 fully connected layer.

[0020] Preferably, initializing the parameter distribution of the convolutional neural network recognition model includes:

[0021] Get the prior distribution P of weight w and bias b so that it obeys N~(0,1) normal distribution. The prior mean and variance of w and b are used To express.

[0022] Preferably, the failure rate of the on-board prediction neural network model is calculated using the PAC-Bayes generalization boundary theory based on conditional Gaussian improvement, including:

[0023] S31, adjust the mean M and variance V of the hidden layer. Assume that the posterior distribution of parameters w and b is Q, and its mean and variance are m and s. After iterative training, the hidden layer M and V can be expressed as follows:

[0024]

[0025]

[0026] Where W is the vector of parameter bias w, and B is the vector of bias b;

[0027] S32, using Q S =l(M,V) calculates the empirical error of the training set;

[0028] S33, based on empirical error, uses the following formula

[0029] Calculate the true error of machine learning;

[0030] S34, through θ=θ-η▽Q D Update the network parameters w and b, and repeat the calculations from S31 to S33 until the network iteration is completed;

[0031] S35, obtaining a failure rate δ of the signal time-frequency graph recognition neural network model according to the true error.

[0032] In some embodiments, an airborne machine learning system safety assessment apparatus is disclosed, comprising: a training neural network model module configured to obtain an airborne prediction neural network model, the airborne prediction neural network model being used to predict airborne equipment characteristics or flight scenario characteristics;

[0033] The failure rate calculation module is configured to calculate the failure rate of the on-board prediction neural network model by using the PAC-Bayes generalization boundary theory based on conditional Gaussian improvement.

[0034] In some embodiments, a device is disclosed, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the airworthiness safety assessment method in an airborne system as described above is performed.

[0035] In some embodiments, a computer-readable storage medium is disclosed, on which a computer program is stored. The program is executed by a processor to perform the above-mentioned airworthiness safety assessment method in an airborne system.

[0036] The embodiments of the present disclosure provide a method, apparatus, device, and storage medium for airborne machine learning system security assessment, which can achieve the following technical effects:

[0037] In existing technologies, airborne predictive neural network models can produce unpredictable outputs, leading to generalization errors and posing challenges to traditional safety assessments. This disclosed embodiment utilizes generalization boundary theory to calculate the failure rate of machine learning classification models during operation, and uses fault trees to calculate the top-level failure rate of navigation equipment, completing a safety assessment of machine learning-based airborne navigation equipment.

[0038] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] One or more embodiments are exemplarily described by corresponding drawings. These exemplary descriptions and drawings do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a scale limitation. In addition,

[0040] Figure 1 This is a flow chart of an airworthiness safety assessment method for an airborne system provided by an embodiment of the present disclosure;

[0041] Figure 2 This is a flow chart of obtaining an airborne prediction neural network model provided by an embodiment of the present disclosure;

[0042] Figure 3 This is a flowchart of a neural network model for generating signal time-frequency graph recognition provided by an embodiment of the present disclosure;

[0043] Figure 4 This is a schematic diagram of the internal structure of a positioning receiver based on machine learning interference identification provided by an embodiment of the present disclosure;

[0044] Figure 5 This is a modified receiver fault tree based on machine learning provided by the embodiment of the present disclosure.

[0045] Figure 6 1 is a schematic diagram of an airworthiness safety assessment device in an airborne system provided by an embodiment of the present disclosure;

[0046] Figure 7 This is a schematic diagram of an airworthiness safety assessment device in an airborne system provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0048] The following description and accompanying drawings sufficiently illustrate specific embodiments of the present invention to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process and other changes. The examples represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or replace portions and features of other embodiments. The scope of the embodiments of the present invention includes the entire scope of the claims, and all available equivalents of the claims. In this article, each embodiment may be referred to individually or collectively by the term "invention", which is merely for convenience and is not intended to automatically limit the scope of the application to any single invention or inventive concept if more than one invention is in fact disclosed. In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, without requiring or implying any actual relationship or order between these entities or operations. Moreover, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, an element defined by the statement "comprising a..." does not exclude the presence of other identical elements in the process, method or device comprising the element. The various embodiments herein are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the methods, products, etc. disclosed in the embodiments, since they correspond to the method part disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0049] As we all know, machine learning consists of two phases: design and operation. The design phase includes model training, validation, and testing, while the operation phase utilizes the frozen model from the design phase to perform reasoning on unknown data. However, machine learning is characterized by being "data-driven" and having hidden mechanisms. Even if the model achieves a good state during training, such as an infinitesimal loss function, during inference, the output will depend on the correlation between the new input and the dataset used in the design process. This, along with the inherent complexity of machine learning models, can lead to unpredictable outputs, resulting in generalization errors and posing challenges to traditional safety assessments.

[0050] Existing research on interference signals affecting navigation receiver performance relies solely on machine learning for signal classification and identification. It fails to consider the generalization error bounds and resulting software failure rates generated when positioning and navigation equipment based on machine learning identification modules is installed and operated. This means there's no quantitative method for assessing the failure rate of aircraft using machine learning-based retrofitted airborne navigation equipment in the aviation industry, or whether the resulting top-level positioning system failure rate meets safety targets.

[0051] In order to solve the problems existing in the related technology, the embodiment of the present disclosure uses the generalization boundary theory to calculate the failure rate caused by the machine learning classification model during the operation stage, and uses the fault tree to calculate the top-level failure rate of the navigation equipment to complete the safety assessment of the machine learning-based airborne navigation equipment.

[0052] Combine Figure 1 As shown, an embodiment of the present disclosure provides a flow chart of an airworthiness safety assessment method in an airborne system, including:

[0053] S100, obtaining an airborne prediction neural network model, where the airborne prediction neural network model is used to predict airborne equipment characteristics or flight scene characteristics.

[0054] For example, the airborne prediction neural network model can be used to predict potential interference signals encountered by the aircraft in the flight environment, or it can use the landing pictures taken by the aircraft to determine whether it is a reasonable landing point and provide the docking coordinates of the landing runway.

[0055] See also Figure 2 , is a flow chart of obtaining an onboard prediction neural network model in an embodiment of the present disclosure. Specifically, obtaining an onboard prediction neural network model includes:

[0056] S110, obtaining a signal time-frequency diagram dataset.

[0057] A data set of 50,000 signal time-frequency diagrams is obtained. Further, the signal time-frequency diagram data set is preprocessed, including grayscale value processing.

[0058] S120, using the signal time-frequency graph dataset to train a convolutional neural network to generate a neural network model for signal time-frequency graph recognition.

[0059] It should be noted that, in this embodiment, the airborne prediction neural network model is a neural network model for signal time-frequency diagram recognition. In order to obtain a convolutional neural network interference signal classification and recognition model with good generalization performance, the original input data should contain as many interference signal feature vectors as possible with significant characteristics, ensuring that the trained network model has a high correct recognition rate during the operation stage. The time-frequency diagram obtained by short-time Fourier transform can take into account the characteristics of the interference signal in the time domain and frequency domain at the same time, and meets the conditions for providing significant interference signal characteristics. Therefore, the interference signal time-frequency diagram is used as the input of the convolutional neural network.

[0060] Specifically, see Figure 3 , is a flowchart of a neural network model for generating a signal time-frequency graph for recognition in an embodiment of the present disclosure. The neural network model for generating a signal time-frequency graph for recognition includes:

[0061] S121, divide the signal time-frequency diagram dataset into a training set and a test set.

[0062] Specifically, depending on the type of dataset required for neural network training, the training set is used to fit the model. By setting classifier parameters and network weights, gradient descent is performed on the training error during training. The test set is used to evaluate the generalization ability of the final model. However, it cannot be used as a basis for algorithmic choices such as parameter tuning and feature selection.

[0063] The 50,000 datasets are divided into training and test sets in a ratio of 3:2.

[0064] S122, Constructing the architecture of convolutional neural network.

[0065] Constructing a convolutional neural network. Excluding the input and output layers, a convolutional neural network generally consists of convolutional layers, pooling layers, activation functions, and linear layers. This network architecture consists of four convolutional layers, four max-pooling layers, four ReLU activation functions (φ), two linear layers, and one fully connected layer. The convolution kernel sizes are 13×13, 9×9, 3×3, and 2×2, respectively. The window size of the first two pooling layers is 4×4, and the window size of the last two pooling layers is 2×2. The image shape changes after the original input data passes through each layer are shown in Table 1 below.

[0066] Table 1

[0067] Network layer Convolution kernel Activation Function Output shape Input layer Conv1 (13，13) ReLU (216，216，10) Conv2 (9，9) ReLU (208，208，15) Conv3 (3，3) ReLU (50，50，20) Conv4 (2，2) ReLU (49，49，30) Full_con 17280 Linear1 (100，) Linear2 (5,)

[0068] S123, initializing the parameter distribution of the convolutional neural network recognition model.

[0069] The failure rate of the neural network recognition model based on the signal time-frequency diagram is calculated using the conditional Gaussian PAC-Bayes generalization boundary theory. It is necessary to ensure that the parameter distribution obeys the Gaussian distribution. Therefore, the parameter distribution of the convolutional neural network recognition model is initialized, and the prior distribution P of the weight w and the bias b is obtained to make it obey the N~(0,1) normal distribution. The prior mean and variance of w and b are used. It is represented by . Given that the number of training sets is 30,000, the network learning rate η is set to 0.05 and the number of iterations T is set to 4.

[0070] S124, training and testing the convolutional neural network to complete the model's recognition of the interference signal's time-frequency diagram type, output the interference signal type, and generate a neural network model for signal time-frequency diagram recognition.

[0071] The convolutional neural network is trained and tested according to the parameter settings and the division of the type and number of the time-frequency graph data sets, and the interference signal type is output to generate a neural network model for signal time-frequency graph recognition. It should be noted that in this embodiment, the neural network model for signal time-frequency graph recognition is used to identify the type of signal time-frequency graph, but is not limited to identifying the type of signal time-frequency graph.

[0072] S200 uses the PAC-Bayes generalization boundary theory based on conditional Gaussian improvement to calculate the failure rate of the on-board prediction neural network model.

[0073] It should be understood that the onboard prediction neural network model can achieve good prediction results, but the onboard prediction neural network model itself has a failure rate, which may pose a threat to the safe flight of the aircraft. Therefore, the onboard prediction neural network model needs to be further optimized. The process of calculating the failure rate based on the conditional Gaussian improved PAC-Bayes theory can be that first, an optimization boundary needs to be determined as the optimization target; then the hyperparameters of the network, such as weights, are determined, and the hyperparameters are used as priors P, the network is trained through the data set, the hyperparameter distribution is modified, and the posterior distribution Q is obtained; finally, the network output is adjusted to a Gaussian distribution by adjusting the hidden layer, so that the optimal boundary and the corresponding failure rate are obtained using the conditional Gaussian PAC-Bayes generalization boundary theory. The failure rate calculated based on the conditional Gaussian improved PAC-Bayes theory can be used to determine whether the onboard prediction network model poses a safety threat.

[0074] Specifically, the failure rate of the on-board prediction neural network model is calculated using the PAC-Bayes generalization boundary theory based on conditional Gaussian improvement, including:

[0075] S201, adjust the mean M and variance V of the hidden layer. Assume that the posterior distribution of parameters w and b is Q, and its mean and variance are m and s. After iterative training, M and V of the hidden layer can be expressed as follows:

[0076]

[0077]

[0078] Where W is the vector of parameter bias w, and B is the vector of bias b;

[0079] S202, use Q S =l(M,V) calculates the empirical error of the training set;

[0080] S203, based on empirical error, uses the following formula

[0081] Calculate the true error of machine learning;

[0082] S204, passed Update the network parameters w and b, and repeat the calculations from S31 to S33 until the network iteration is completed;

[0083] S205, obtaining the failure rate δ of the signal time-frequency diagram recognition neural network model based on the true error. In a preferred embodiment, the airborne machine learning system safety assessment method further includes:

[0084] S300 uses failure rates to conduct safety assessments on the loss of navigation and positioning functions during flight.

[0085] Specifically, the failure rate of the signal time-frequency graph recognition neural network model is integrated with the failure rate of navigation hardware. The top-level failure rate is calculated with the help of the fault tree analysis method. It is then compared with the safe flight targets for transport aircraft specified by the Airworthiness Administration to complete the safety assessment of the aircraft losing its navigation and positioning function during flight.

[0086] The failure rate is used to conduct a safety assessment of the loss of navigation and positioning functions during flight. The failure rate of the signal time-frequency diagram identification neural network model obtained by the generalized boundary theory is brought into the fault tree of the loss of navigation and positioning functions during flight of the aircraft to calculate the top-level function failure rate from the bottom up.

[0087] Modify the GNSS positioning receiver to replace the traditional interference identification technology with machine learning to classify the suppression interference type, and apply the positioning receiver based on machine learning to the airborne positioning and navigation system. Figure 4, which is a diagram of the internal structure of a positioning receiver based on machine learning interference recognition in an embodiment of the present disclosure. The CNN interference recognition portion is a machine learning recognition algorithm that replaces traditional algorithms in this application, effectively improving the correct recognition rate of interference signals. However, the inherent uncertainty of this algorithm leads to a failure rate that could cause typical aircraft models to fail during flight, such as "losing navigation and positioning functions when flying in remote areas," meaning a dangerous failure probability exceeding the quantitative safety requirement (10E-5 / FH). This is the background to this application.

[0088] Among them, the failure probability of the machine learning module obtained by using the PAC-Bayes boundary theory is δ = 5×10 -5 , participate in the quantitative calculation of the fault tree. This application is based on machine learning to modify the receiver fault tree diagram as shown Figure 5 As shown. Figure 5 It can be seen that any of the following events will cause the failure:

[0089] The hardware is intact, but the ML model uncertainty exceeds the limit;

[0090] The signal processing module hardware fails.

[0091] The hardware failure of the signal processing module can be obtained by referring to the relevant regulations. When the hardware is intact, the failure rate of the machine learning recognition algorithm obtained by the generalization boundary theory is δ = 5×10 -5 .

[0092] Substituting formula (1) into the fault tree, the top-level functional failure rate is calculated from bottom to top to be Q = 7.9 × 10 -8 ,

[0093] P S =P h +(1-P h )P m (1)

[0094] P S is the comprehensive failure probability; P h is the probability of hardware failure; P m is the probability of harm occurring due to uncertainty in the machine learning model.

[0095] By comparison, it can be seen that the failure rate of the ML-based navigation and positioning function is within the Category III safety target range and will not cause unacceptable risks to the flight safety of the aircraft.

[0096] Figure 6 An airworthiness safety assessment device in an airborne system provided in an embodiment of the present invention is shown, the device comprising:

[0097] a training neural network model module configured to obtain an onboard prediction neural network model, wherein the onboard prediction neural network model is used to predict onboard equipment characteristics or flight scenario characteristics;

[0098] The failure rate calculation module is configured to calculate the failure rate of the on-board prediction neural network model by using the PAC-Bayes generalization boundary theory based on conditional Gaussian improvement.

[0099] Combine Figure 7 As shown, an embodiment of the present disclosure provides an airworthiness safety assessment device in an airborne system, comprising a processor and memory. Optionally, the device may further comprise a communication interface and a bus. The processor, communication interface, and memory may communicate with each other via the bus. The communication interface may be used for information transmission. The processor may invoke logic instructions in the memory to execute the airworthiness safety assessment method in the airborne system of the above embodiment.

[0100] An embodiment of the present disclosure provides a storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned airworthiness safety assessment method for an airborne system.

[0101] The aforementioned storage medium may be a transient computer-readable storage medium or a non-transient computer-readable storage medium. Non-transient storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks. These media may also be transient storage media.

[0102] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to also include plural forms. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, an element defined by the sentence "comprising a..." does not exclude the presence of other identical elements in the process, method or device that includes the element. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments can be referenced to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be found in the description of the method part.

[0103] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0104] The flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by dedicated hardware-based devices that perform the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for safety assessment of an airborne machine learning system, characterized in that: The method comprises: Acquiring an airborne prediction neural network model, wherein the airborne prediction neural network model is used to predict airborne equipment characteristics or flight scenario characteristics; The failure rate of the on-board prediction neural network model is calculated using the PAC-Bayes generalization bound theory based on conditional Gaussian improvement. The safety of the airborne positioning and navigation system is evaluated by integrating the failure rate of the airborne prediction neural network model with the failure rate of the airborne equipment hardware. An airborne machine learning system safety assessment device applying the method comprises: a training neural network model module configured to obtain an onboard prediction neural network model, wherein the onboard prediction neural network model is used to predict onboard equipment characteristics or flight scenario characteristics; A failure rate calculation module is configured to calculate the failure rate of the onboard prediction neural network model using the PAC-Bayes generalization boundary theory based on the conditional Gaussian improvement, and to perform a safety assessment of the airborne positioning and navigation system by fusing the failure rate of the onboard prediction neural network model with the failure rate of the airborne equipment hardware; The safety assessment of the loss of navigation and positioning functions during flight is conducted using the failure rate, specifically: By integrating the failure rate of the signal time-frequency graph recognition neural network model with the navigation hardware failure rate, the top-level failure rate was calculated using the fault tree analysis method. This was then compared with the safe flight targets for transport aircraft specified by the Airworthiness Administration to complete a safety assessment of the aircraft's loss of navigation and positioning capabilities during flight. The failure rate is used to conduct a safety assessment of the loss of navigation and positioning functions during flight. The failure rate of the signal time-frequency diagram identification neural network model obtained by the generalized boundary theory is brought into the fault tree of the loss of navigation and positioning functions during flight of the aircraft to calculate the top-level function failure rate from the bottom up.

2. The airborne machine learning system security assessment method according to claim 1, characterized in that: Get onboard prediction neural network models, including: Get the signal time-frequency diagram dataset; The convolutional neural network is trained using the signal time-frequency graph dataset to generate a neural network model for signal time-frequency graph recognition.

3. The airborne machine learning system security assessment method according to claim 2, characterized in that: The convolutional neural network is trained using the signal time-frequency graph dataset to generate a neural network model for signal time-frequency graph recognition, including: The signal time-frequency graph dataset is divided into a training set and a test set; Build the architecture of a convolutional neural network; Initialize the parameter distribution of the convolutional neural network recognition model; The convolutional neural network is trained and tested using the training set and test set to complete the model's recognition of the signal time-frequency diagram and generate a neural network model for signal time-frequency diagram recognition.

4. The airborne machine learning safety assessment method according to claim 3, characterized in that: Constructing the architecture of a convolutional neural network, including: The network architecture consists of 4 convolutional layers, 4 pooling layers, 4 ReLU activation functions φ, 2 linear layers and 1 fully connected layer.

5. The airborne machine learning system security assessment method according to claim 4, characterized in that: Initialize the parameter distribution of the convolutional neural network recognition model, including: Get the prior distribution P of weight w and bias b so that it obeys N~(0,1) normal distribution. The prior mean and variance of w and b are used To express.

6. The airborne machine learning system security assessment method according to claim 5, characterized in that: The failure rate of the on-board prediction neural network model is calculated using the PAC-Bayes generalization bound theory based on conditional Gaussian improvement, including: S201, adjust the mean M and variance V of the hidden layer. Assume that the posterior distribution of parameters w and b is Q, and its mean and variance are m and s. After iterative training, M and V of the hidden layer can be expressed as follows: Where W is the vector of parameter bias w, and B is the vector of bias b; S202, use Q S =l(M,V) calculates the empirical error of the training set; S203, based on the empirical error, calculate the true error of the machine learning using the following formula: S204, passed Update the network parameters w and b, and repeat the calculations from S31 to S33 until the network iteration is completed; S205, obtaining the failure rate of the signal time-frequency diagram identification neural network model according to the true error.

7. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images