Model training and flight operation control core risk detection method, device and equipment

Abstract

The invention relates to a model training and flight operation control core risk detection method and device, computer equipment, a computer readable storage medium and a computer program product. The model training method comprises the following steps: acquiring an index sample set and a risk value sample set of flight operation control core risks; the index sample set comprises index feature samples and index risk value samples corresponding to the index feature samples, and the risk value sample set comprises the index risk value samples and core risk value samples corresponding to the index risk value samples; based on the index sample set, training the first initial model to obtain an index risk prediction model; the index risk prediction model is used for predicting an index risk value; based on the risk value sample set, training the second initial model to obtain a core risk prediction model; and the core risk prediction model is used for predicting a flight operation control core risk value based on the index risk value. By adopting the method, the accuracy of risk prediction can be improved.

Description

Technical Field

[0001] The present application relates to the aviation field, and in particular to a model training and flight operation control core risk detection method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] In the aviation industry, the core risk indicator system has many indicators. In addition, when doing safety performance management in the operation control business, the core risk indicator system needs to be sufficiently interpretable to help dispatchers quickly locate the shortcomings of safety management, take risk control measures in a timely manner, and explore potential safety hazards and conduct investigations and treatments.

[0003] Traditional technologies rely heavily on experience-based safety management and risk warning models, which are crude, imprecise, and take a long time to detect unsafe events. Over-reliance on experience can easily lead to a lack of scientificity and consistency in analysis and decision-making, making it difficult to discover safety hazards and operational risks hidden behind the data. Summary of the invention

[0004] Based on this, it is necessary to provide a model training and flight operation control core risk detection method, device, computer equipment, computer readable storage medium and computer program product that can improve accuracy in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a model training method, the method comprising:

[0006] Obtaining an indicator sample set and a risk value sample set of the core risks of flight operation control; the indicator sample set includes an indicator feature sample and an indicator risk value sample corresponding to the indicator feature sample, and the risk value sample set includes the indicator risk value sample and a core risk value sample corresponding to the indicator risk value sample;

[0007] Based on the indicator sample set, the first initial model is trained to obtain an indicator risk prediction model; the indicator risk prediction model is used to predict the indicator risk value;

[0008] Based on the risk value sample set, a second initial model is trained to obtain a core risk prediction model; the core risk prediction model is used to predict the flight operation control core risk value based on the indicator risk value.

[0009] In one embodiment, the step of obtaining a sample set of indicators and a sample set of risk values ​​of the core risks of flight operation control includes:

[0010] Obtaining a first indicator sample of the flight operation control core risk and a second indicator sample contained in the first indicator sample;

[0011] Obtaining an indicator risk value sample and a core risk value sample corresponding to the first indicator sample, and determining a risk value sample set according to the indicator risk value sample and the core risk value sample;

[0012] An indicator feature sample is determined based on the second indicator sample, and an indicator sample set is determined according to the indicator feature sample and the indicator risk value sample.

[0013] In one embodiment, the generation process of the first indicator sample has multiple sub-processes, the multiple sub-processes are associated with the core risk source of flight operation control, the core risk source of flight operation control has corresponding control measures, and the implementation data of the control measures and the existence of the implementation data belong to the second indicator sample.

[0014] In one embodiment, the first initial model is trained based on the indicator sample set to obtain the indicator risk prediction model, including:

[0015] Extracting samples from the indicator sample set multiple times to obtain multiple indicator sample subsets;

[0016] According to each of the indicator sample subsets, the first decision tree model is trained to obtain the decision tree model of each of the indicator sample subsets; the random forest model composed of the decision tree models of each of the indicator sample subsets is the indicator risk prediction model;

[0017] The second initial model is trained based on the risk value sample set to obtain a core risk prediction model, including:

[0018] Extracting samples from the risk value sample set multiple times to obtain multiple risk value sample subsets;

[0019] According to each of the risk value sample subsets, the second decision tree model is trained to obtain the decision tree model of each of the risk value sample subsets; the random forest model composed of the decision tree models of each of the risk value sample subsets is the core risk prediction model.

[0020] In one embodiment, the first decision tree model is trained according to each of the indicator sample subsets to obtain the decision tree model of each of the indicator sample subsets, including:

[0021] In the case where the first decision tree model is a regression tree model, the first decision tree model is trained according to each of the indicator sample subsets and the square error loss function to obtain a regression tree model of each of the indicator sample subsets;

[0022] In the case where the first decision tree belongs to a classification tree model, the Gini index of the first decision tree model in the classification process is determined based on the indicator sample subset, and the first decision tree model is trained according to the Gini coefficient of the first decision tree model in the classification process to obtain the classification tree model of each of the indicator sample subsets;

[0023] The step of training the second decision tree model according to each risk value sample subset to obtain the decision tree model of each risk value sample subset includes:

[0024] In the case where the second decision tree model is a regression tree model, the second decision tree model is trained according to each risk value sample subset and the square error loss function to obtain a regression tree model of each indicator risk value subset;

[0025] In the case where the second decision tree belongs to a classification tree model, the Gini index of the second decision tree model in the classification process is determined based on the indicator sample subset, and the second decision tree is trained according to the Gini coefficient of the second decision tree in the classification process to obtain the classification tree model of each risk value sample subset.

[0026] The present application also provides a flight operation control core risk detection method, including:

[0027] Obtain the characteristics of indicators to be tested for the core risks of flight operation control;

[0028] Determine the indicator risk value corresponding to the indicator feature to be detected through the indicator risk prediction model;

[0029] The indicator risk value is converted into a flight operation control core risk value through a core risk prediction model;

[0030] Among them, the indicator risk prediction model and the core risk prediction model are trained based on any one of claims 1 to 5.

[0031] In a second aspect, the present application also provides a model training device, the device comprising:

[0032] A sample acquisition module, used to acquire an indicator sample set and a risk value sample set of the core risk of flight operation control; the indicator sample set includes an indicator feature sample and an indicator risk value sample corresponding to the indicator feature sample, and the risk value sample set includes the indicator risk value sample and a core risk value sample corresponding to the indicator risk value sample;

[0033] A training module, used to train the first initial model based on the indicator sample set to obtain an indicator risk prediction model; the indicator risk prediction model is used to predict the indicator risk value;

[0034] The training module is used to train the second initial model based on the risk value sample set to obtain a core risk prediction model; the core risk prediction model is used to predict the flight operation control core risk value based on the indicator risk value.

[0035] The present application also provides a flight operation control core risk detection device, the device comprising:

[0036] An acquisition module is used to obtain the characteristics of indicators to be detected for the core risks of flight operation control;

[0037] A first detection module, used to determine the indicator risk value corresponding to the to-be-detected data set through an indicator risk prediction model;

[0038] A second detection module is used to convert the indicator risk value into a flight operation control core risk value through a core risk prediction model;

[0039] Among them, the indicator risk prediction model and the core risk prediction model are trained based on the method of the first aspect.

[0040] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the model training or flight operation control core risk detection method in any of the above embodiments are implemented.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the model training or flight operation control core risk detection method in any of the above embodiments.

[0042] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the model training or flight operation control core risk detection method in any of the above embodiments are implemented.

[0043] The above-mentioned model training method, device, computer equipment, computer-readable storage medium and computer program product, since the indicator risk prediction model is used to predict the indicator risk value, and the core risk prediction model is used to predict the flight operation control core risk value based on the indicator risk value, the indicator risk prediction model is the first layer model in actual use, and the core risk prediction model is the second layer model in actual use, and the two layers of models are trained based on different data sets, respectively, so the training and centralized execution can ensure the stability of the training, and the training of the second layer model will not be affected by the problems of the first layer model training; at the same time, the prediction results of each indicator feature sample can be obtained through the first layer model, which is convenient for accurately locating the safety shortcomings; and it helps to reduce the feature items of each input of the two layers of models, avoiding the degradation of algorithm performance caused by too many feature items when the number of samples is limited, so that the accuracy can be guaranteed. Therefore, the decoupling training of the two models can make the training stability relatively high, and can accurately determine the safety shortcomings corresponding to the indicator features, and the accuracy of the prediction results is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 An application environment diagram of a model training method in an embodiment;

[0046] Figure 2 A schematic diagram of a flow chart of a model training method in one embodiment;

[0047] Figure 3 A schematic diagram of a process of obtaining an indicator in an embodiment;

[0048] Figure 4 A schematic diagram of the overall process of obtaining indicators in another embodiment;

[0049] Figure 5 A schematic diagram of a process of model training in one embodiment;

[0050] Figure 6 A schematic diagram of a process of model training in one embodiment;

[0051] Figure 7 A schematic diagram of a process of multi-layer random forest model prediction in one embodiment;

[0052] Figure 8A schematic diagram of the prediction results of a multi-layer random forest model in one embodiment;

[0053] Fig. 9 A schematic diagram of a flow chart of a flight operation control core risk detection method in one embodiment;

[0054] Fig.10 is a structural block diagram of a model training device in one embodiment;

[0055] Fig.11 It is a structural block diagram of a flight operation control core risk detection device in one embodiment;

[0056] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] With the rapid expansion of fleet size and operating areas, the aircraft composition and network layout are becoming increasingly complex, and the characteristics of multiple points and wide coverage are becoming increasingly prominent. The risks of air transport are becoming unpredictable, so it is necessary to enhance risk prevention and control and responsibility awareness and build a risk control system.

[0059] Operation and control safety performance management refers to monitoring the key operation chains and core links of airline flight operation control, aiming to evaluate the safety status of the overall flight operation control business, warn of safety issues, achieve full coverage of safety management through safety performance management, shift from qualitative management to qualitative and quantitative management, increase safety management flexibility, and improve safety levels and safety culture. Through operation and control safety performance management, it can play a risk warning effect and improve the risk prevention and control capabilities of flight operation control.

[0060] As the starting point and focus of safety performance management and risk warning, core risk control combines safety performance indicators, process architecture governance and double prevention mechanism to sort out the sources of danger and hidden dangers, and identify the important factors that affect flight operation safety during flight operation control. Existing research methods mainly take the overall safety performance indicator system of airlines as the research object, and use traditional ratio analysis and other means to monitor and evaluate safety performance indicators. The core risk of flight operation control is to extract the core links directly related to operational safety in the safety performance indicator system, and evaluate and predict operational risks from the perspective of safety management at the overall level of flight operation control. At present, there is no research on the assessment and prediction of core risks in flight operation control, and the industry mostly relies on experience-based safety management and risk warning models, which are rough, imprecise, and have a long cycle for discovering unsafe events. Over-reliance on experience can easily lead to a lack of scientificity and consistency in analysis and decision-making, and it is difficult to discover safety hazards and operational risks hidden behind the data.

[0061] Using machine learning methods to study the evaluation and prediction of core risk indicators, accurately evaluate and predict the core risk value of operation control, and improve the efficiency of operation control safety performance management and risk warning and prevention. It is of great practical significance to better guide operation risk control work in business scenarios such as risk warning release, safety performance indicator monitoring, and operation policy formulation, and to achieve the transformation of safety management and risk prevention and control work from the usual post-event disposal to pre-event warning.

[0062] The core risk indicator system has many indicators and levels. Since it needs to be applied to the operation control business for safety performance management, it needs to be sufficiently explainable to help dispatchers quickly locate the shortcomings of safety management, take risk control measures in a timely manner, and explore potential safety hazards and conduct investigations and treatments. If the classic random forest technology is used, only the overall core risk assessment can be obtained, and too many indicators can easily lead to a decline in algorithm performance.

[0063] Based on this, the present application embodiment provides a model training method, which can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0064] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system determined by multiple physical servers, or a cloud server that provides cloud computing services.

[0065] In an exemplary embodiment, Figure 2 As shown, a model training method is provided, which is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 202 to 206. Among them:

[0066] Step 202, obtaining an indicator sample set and a risk value sample set of the core risks of flight operation control; the indicator sample set includes an indicator feature sample and an indicator risk value sample corresponding to the indicator feature sample, and the risk value sample set includes an indicator risk value sample and a core risk value sample corresponding to the indicator risk value sample.

[0067] Flight Operations Control refers to the process of airlines organizing, directing, monitoring and managing flight activities. It covers the entire process from flight planning to actual execution, ensuring that flights are safe, on time and efficient.

[0068] Core risks are those basic risks that may pose a direct threat to aviation safety. These risks may include, but are not limited to, technical failures, human errors, natural disasters, safety management systems, and other dimensions; among them, technical failures are such as mechanical failures of aircraft, electronic system failures, etc., which may lead to flight accidents or other safety issues. Among them, human errors are human factors such as pilot operating errors and improper maintenance by ground staff, which may directly affect the safe operation of aircraft; among them, deficiencies in the safety management system: imperfect safety supervision system or lack of safety management measures may also lead to safety risks.

[0069] In this case, the core risk of flight operation control is to extract the core links in the safety performance indicator system that are directly related to operational safety, and to manage operational risks from the dimension of safety management at the overall level of airline flight operation control.

[0070] The core risks of flight operation control include but are not limited to: illegal dispatch release, inadequate operation monitoring, key operation data errors and improper emergency response; among them, illegal dispatch release refers to situations such as high-risk flight landing, unqualified personnel dispatching flights, wrong or missed selection of alternate airports, etc.; inadequate operation monitoring refers to situations such as the IAF assessment recommendation of the actual alternate flight to continue the approach, failure to perform the IAF assessment, and failure to land for more than 15 minutes; key operation data errors refer to situations such as the failure to revise and update the program produced by the supplier in a timely manner, the mismatch between the FPL and the computer flight plan, the failure to update the EFB information in a timely manner, or the data release errors, etc. Improper emergency response refers to situations such as the failure to handle the aborted takeoff warning and the abnormal descent of the flight.

[0071] The indicator sample set includes corresponding indicator feature samples and indicator risk value samples, which are used to train the model to identify the corresponding relationship between the indicator feature samples and the indicator risk value samples, so as to determine the corresponding indicator risk value through the indicator features.

[0072] The indicator feature sample is a sample feature used for model training. The sample feature is a feature item used to describe the indicator of the core risk of flight operation control, which can describe the indicator from multiple dimensions, thereby more accurately reflecting the indicator attributes. Optionally, the indicator feature is set according to each indicator separately, so as to form the characteristics of each indicator item.

[0073] The indicator risk value sample is the indicator risk value used for model training. The indicator risk value sample corresponds to the risk assessment result of the indicator feature sample. The indicator risk value corresponds to each indicator. There is an inherent correspondence between the indicator risk value sample and the indicator feature sample included in each indicator, so the indicator risk value sample can be determined based on the indicator feature sample.

[0074] The risk value sample set includes the corresponding indicator risk value samples and core risk value samples, which are used to train the model to identify the corresponding relationship between the indicator risk value and the core risk value, so as to determine the corresponding core risk value through the indicator risk value. The core risk value is the risk assessment result based on each indicator risk value, which can reflect the core risk level of flight operation control from an overall perspective.

[0075] Optionally, the indicator sample set is determined based on the historical operation and maintenance data of the flight, and the historical operation and maintenance data can be multi-source heterogeneous data. Optionally, on the basis of determining various core risk indicators, by setting up embedded points in the flight operation control system and other systems, using scales to collect and record dispatch release logs, FPL messages, flight warnings, performance records, flight intelligence and other data, screening valid data, performing default value and abnormal value processing on the data, obtaining multi-source heterogeneous data, and then constructing indicator sample sets for each indicator. Therefore, multiple analysis angles can be formed through multi-source heterogeneous data, which can provide a more comprehensive information perspective, reduce the deviation of a single data source, and improve the indicator sample set to be more in line with the characteristics in actual application scenarios.

[0076] Optionally, the obtained core risk data set is used to organize dispatch experts to evaluate and score the risk value of each sample in each indicator data set and indicator risk value sample to obtain the indicator risk value sample. Optionally, dispatch experts are organized to conduct comprehensive scoring of the indicator risk value samples of each indicator data set to obtain the core risk value of flight operation control.

[0077] Step 204: Based on the indicator sample set, the first initial model is trained to obtain an indicator risk prediction model; the indicator risk prediction model is used to predict the indicator risk value.

[0078] The first initial model is a machine learning model trained based on the indicator sample set with the goal of predicting the indicator risk value, and the first initial model has at least one function of classification and regression. Optionally, the first initial model can adopt at least one model of the classic random forest (RF), K-Nearest Neighbors (KNN), decision tree (DT), gradient boosting (GB), and elastic net (EN).

[0079] The indicator risk prediction model can perform classification regression based on the input indicator characteristics to obtain the indicator risk value corresponding to the indicator characteristics. The indicator risk value is the result obtained by the indicator risk prediction model through classification regression based on the indicator characteristics contained in the indicator.

[0080] In an optional embodiment, a first initial model is trained based on an indicator sample set to obtain an indicator risk prediction model, including: determining an indicator sample training set and an indicator sample test set into which each indicator sample set is divided; taking the predicted indicator risk value as a goal, training the first initial model according to the indicator sample training set and the loss function until the loss function converges to obtain a first model to be tested corresponding to the first initial model; judging whether the first model to be tested meets the preset adjustment according to the indicator sample test set; if not, retraining the first model to be tested as the first initial model; if satisfied, using the first model to be tested as the indicator risk prediction model.

[0081] Step 206, based on the risk value sample set, the second initial model is trained to obtain a core risk prediction model; the core risk prediction model is used to predict the flight operation control core risk value based on the indicator risk value.

[0082] The second initial model is a machine learning model trained based on a risk value sample set with the goal of predicting the core risk value of flight operation control. The second initial model has at least one function of classification and regression. Optionally, the second initial model can adopt at least one model of the classic random forest (RF), K-Nearest Neighbors (KNN), decision tree (DT), gradient boosting (GB), and elastic net (EN).

[0083] The difference between the first initial model and the second initial model is the training objectives of the two. The first initial model is trained with the goal of predicting the indicator risk value, and the second initial model is trained with the goal of predicting the flight operation control core risk value; and the first initial model and the second initial model are trained based on different sample sets. Optionally, the results predicted by the first initial model include the indicator risk value, and the second initial model is used to generate the flight operation control core risk value based on the indicator risk value, but the first initial model and the second initial model are trained separately, and the results predicted by the first initial model are not used to train the second initial model.

[0084] The core risk prediction model is used to perform classification regression based on the indicator risk value output by the indicator risk prediction model to obtain the core risk value of flight operation control.

[0085] Optionally, when both the first initial model and the second initial model are multiple decision tree models, during the training process, the indicator risk prediction model and the core risk prediction model constitute two independent random forest models; and after the training is completed, the indicator risk prediction model is the first-layer random forest model, the core risk prediction model is the second-layer random forest model, and the first-layer random forest model and the second-layer random forest model constitute a hierarchical random forest (HRF). As a result, the accuracy of the core risk value of flight operation control is improved.

[0086] In an optional embodiment, a second initial model is trained based on a risk value sample set to obtain a core risk prediction model, including: determining a risk value sample training set and a risk value sample test set divided by each risk value sample set; taking the predicted indicator risk value as a target, training the second initial model according to the risk value sample training set and a loss function until the loss function converges to obtain a second model to be tested corresponding to the second initial model; judging whether the second model to be tested meets the preset adjustment according to the risk value sample test set; if not, retraining the second model to be tested as the second initial model; if satisfied, using the second model to be tested as the core risk prediction model.

[0087] In the above model training method, since the indicator sample set includes indicator feature samples and corresponding indicator risk value samples, by training the first initial model with the indicator sample set, the first initial model can learn the intrinsic relationship between the indicator feature and the indicator risk value samples, and obtain the indicator risk prediction model; similarly, since the risk value sample set includes the indicator risk value samples and the corresponding core risk value samples, the first initial model can learn the intrinsic relationship between the indicator risk value and the core risk value, and obtain the core risk prediction model. Since the indicator risk prediction model is used to predict the indicator risk value, and the core risk prediction model is used to predict the core risk value based on the indicator risk value, the indicator risk prediction model is the first-layer model, and the core risk prediction model is the second-layer model, and the two-layer models are trained separately, so the separate training and centralized execution can ensure the stability of the training, and the training of the second-layer model will not be affected by the problems of the first-layer model training; at the same time, the prediction results of each indicator feature sample can be obtained through the first-layer model, which helps business personnel to more clearly obtain the composition of the core risk value and accurately locate the safety shortcomings; and it helps to reduce the feature items input by each of the two-layer models, avoiding the degradation of algorithm performance caused by too many feature items when the number of samples is limited, so as to ensure accuracy. Therefore, the decoupled training of these two models can make the training relatively stable, and can accurately determine the safety shortcomings corresponding to the indicator characteristics, and the accuracy of the prediction results is high.

[0088] In an exemplary embodiment, obtaining an indicator sample set and a risk value sample set of core risks of flight operation control includes: obtaining a first indicator sample of core risks of flight operation control and a second indicator sample contained in the first indicator sample; obtaining an indicator risk value sample and a core risk value sample corresponding to the first indicator sample, and determining a risk value sample set based on the indicator risk value sample and the core risk value sample; determining an indicator feature sample based on the second indicator sample, and determining an indicator sample set based on the indicator feature sample and the indicator risk value sample.

[0089] The first indicator sample is the first indicator used for model training; the first indicator is a core risk indicator of flight operation control that is higher in level than the second indicator, and thus the first indicator can be refined into a second indicator, that is, the first indicator has multiple measurable dimensions, and each dimension has a corresponding second indicator sample.

[0090] The second indicator sample is the second indicator used for model training, and the second indicator is a measurable data indicator. The second indicator includes but is not limited to quantitative indicators and data indicators; therefore, in the random forest, classification tree models and regression tree models can exist at the same time to more accurately determine risks.

[0091] Optionally, the first indicator is a primary indicator, and the second indicator is a secondary indicator; correspondingly, obtaining the first indicator sample of the core risk of flight operation control and the second indicator sample contained in the first indicator sample includes: multi-source heterogeneous data collection. Specifically, on the basis of the determined core risk indicators, by setting up embedded points in the flight operation control system and other systems, using a meter to collect and record dispatch release logs, FPL messages, flight warnings, performance records, flight information and other data, screening valid data, and performing default value and abnormal value processing on the data, a quantitative data set D of each primary indicator is obtained. 1 ,D 2 ,...,D m , where m is the number of first-level indicators. Then the data set of first-level indicator i in For the dataset D i A feature vector of D, n(i) is the number of features, j = 1, 2, ..., N, N is the data set D i The capacity of each first-level indicator data set D i The overall data set D is formed by horizontal splicing:

[0092]

[0093] Then, the model sample set is constructed. The core risk data set is obtained, and experts are organized to perform a survey on each first-level indicator data set D i Evaluate and score the risk value of each sample in the overall data set D to obtain a typical core risk level 1 indicator sample set S including expert scoresi , and then horizontally splice the scores of each first-level indicator data set with the scores of the overall data set to form the overall sample set S.

[0094] First-level indicator sample set S i =[D i Y i ] as the training sample set of the first layer model, where is the first-level indicator dataset D i The corresponding score. The overall sample set S = [Y 1 Y 2 …Y m Y] as the training sample set of the second layer model, where Y = [y1, y2, ..., y N ] is the score corresponding to the entire dataset D.

[0095] Optionally, there are multiple sub-processes in the generation process of the first indicator sample, the multiple sub-processes are associated with the core risk source of flight operation control, there are corresponding control measures for the core risk source of flight operation control, and the implementation data of the control measures and the existence of the implementation data belong to the second indicator sample.

[0096] A sub-process is a process that may cause the first indicator to occur, and there are multiple sub-processes for the first indicator. The core risk source of flight operation control is the event that causes flight operation risk. The core risk source of flight operation control corresponds to multiple control measures, and control measures are operational events. Each control measure is used to reduce the risk brought by the core risk source of flight operation control. In the process of using the control measure operational event, implementation data is required, so quantitative analysis can be achieved through implementation data, and qualitative analysis can be achieved through the existence or non-existence of implementation data.

[0097] like Figure 3 As shown in the figure, the five sub-processes of route selection, flight plan preparation, release message processing, data submission and pre-flight briefing are all related to the "multi-route route" hazard source. At the same time, from the perspective of the "double prevention mechanism", it is found that this hazard source already has control measures such as "flight plan route warning", "FPL route warning", "REJ message", "RLS route warning", and "crew report route error". Among the above measures, the "REJ message" already has stable and reliable basic data. After transformation, it can be used as a data burial point to provide pre-indicator data support for the core risk of "illegal dispatch release", and it can also effectively verify the reliability of the model. Therefore, "receiving REJ FPL" is used as a secondary indicator of "illegal dispatch release" and is included in the core risk indicator system.

[0098] Optionally, the first indicator is a first-level indicator and the second indicator is a second-level indicator; correspondingly, if there are 6 first-level indicators, each first-level indicator is divided into several second-level indicators, and there are a total of 70 second-level indicators, then the risk value sample set can be expressed as {first-level indicator A expert score, first-level indicator B expert score, ..., first-level indicator F expert score, overall expert score}, the indicator sample set of first-level indicator A can be expressed as {secondary indicator 1, secondary indicator 2, ..., secondary indicator 10, indicator risk value of first-level indicator A}, the indicator sample set of first-level indicator B can be expressed as {secondary indicator 11, secondary indicator 12, ..., secondary indicator 16, indicator risk value of first-level indicator B}, and so on.

[0099] Optionally, as shown in Table 1, the primary indicator is decomposed into several secondary indicators, and corresponding data points are set to clarify the type and source of feature data required for the indicator monitoring quantification algorithm.

[0100] Table 1

[0101]

[0102]

[0103] Optionally, the corresponding data points of the second indicator are set to clarify the characteristic data types and sources required by the indicator monitoring quantification algorithm. Optionally, in the constructed core risk indicator system, 6 first-level indicators and 77 second-level indicators are included. The 6 first-level indicators are: illegal dispatch release, inadequate operation monitoring, flight application errors, improper emergency handling, key operation data errors, and low fuel events, as shown in Table 2.

[0104] Table 2

[0105]

[0106] Optionally, after the core risk indicator system is established, it can be combined with the flight operation control core risk assessment prediction model to form a digital safety performance monitoring method to assist dispatchers in conducting accurate operation risk management and hidden danger investigation and treatment. According to the management and treatment results, the core risk indicator system can be traced back and improved. The overall process is as follows: Figure 4 shown.

[0107] In an optional embodiment, obtaining a first indicator sample of a flight operation control core risk and a second indicator sample contained in the first indicator sample includes: determining the first indicator sample of a flight operation control core risk; determining a flight operation control core risk source associated with the sub-process according to a sub-process used to generate the first indicator sample; and determining the second indicator sample contained in the first indicator sample according to implementation data of control measures corresponding to the flight operation control core risk source and an existence event of the implementation data.

[0108] Therefore, the introduction of business process management can more comprehensively determine the sub-processes, and then through the "double prevention mechanism" methodology formed by the core risk sources and control measures of flight operation control, determine the indicator classification of the core risks of flight operation control, and construct corresponding regression data with the implementation data potentially corresponding to the risk source, and use the existence of the implementation data as the data for classification, so that the classification algorithm and the regression algorithm can be used at the same time. Therefore, the event tree analysis method (ETA) is used to decompose the first indicator into the second indicator for training through qualitative and quantitative analysis, so that the accuracy of the indicator risk prediction model and the core risk prediction model is increased.

[0109] Optionally, the indicator risk value sample and the core risk value sample corresponding to the first indicator sample may be in a preset data set, or may be experimental data or theoretical data provided by experts. Optionally, the indicator risk value sample and the core risk value sample may be used as a risk value sample set, or each indicator risk value sample and the core risk value sample calculated from each indicator risk value sample may be combined into an array, and each array may be used as a risk value sample set to improve processing efficiency.

[0110] Optionally, the second indicator sample can be used as an indicator feature sample, and the indicator feature sample and the indicator risk value sample are combined into an indicator sample set. Optionally, multiple features obtained by sampling the second indicator sample can be used as an indicator feature sample, and the indicator feature sample and the indicator risk value sample are combined into an indicator sample set.

[0111] In this embodiment, the indicator feature samples of the first indicator samples are constructed through the second indicator samples, thereby forming two indicator levels in the training process, which can enable different indicator levels to assist each other to improve accuracy. At the same time, using the risk value samples and core risk value samples of the first indicator alone to determine the risk value sample set can make predictions more accurately from an overall perspective to improve accuracy.

[0112] In an alternative embodiment, Figure 5 As shown, based on the indicator sample set, the first initial model is trained to obtain the indicator risk prediction model, including steps 502-504; correspondingly, based on the risk value sample set, the second initial model is trained to obtain the core risk prediction model, including steps 506-508. Among them:

[0113] Step 502: extract multiple samples from the indicator sample set to obtain multiple indicator sample subsets.

[0114] The indicator sample subset is a sample set obtained by multiple sampling with replacement from the indicator sample set. Optionally, the indicator sample subset contains sample data in the indicator sample set, and the samples of some indicator sample subsets are repeated. Optionally, multiple training subsets are obtained by Bootstrap sampling. Bootstrap sampling technology is used to extract the original indicator sample set S i Draw samples with replacement to generate b different training subsets Where b is the number of training subsets. Each training subset contains N samples.

[0115] Step 504, training the first decision tree model according to each indicator sample subset to obtain the decision tree model of each indicator sample subset; the random forest model composed of the decision tree models of each indicator sample subset is the indicator risk prediction model.

[0116] In one embodiment, a first decision tree model is trained according to each indicator sample subset to obtain a decision tree model for each indicator sample subset, including: in the i-th indicator sample subset to be regressed, based on the decision tree model to be trained for predicting the indicator risk value, a regression prediction is performed on the q-th indicator feature sample to obtain the q-th regression value; according to the difference value between the indicator risk value sample of the q-th indicator sample and the q-th regression value, the first decision tree model is trained to obtain a decision tree model for the i-th indicator sample subset; wherein i and q are positive integers.

[0117] In one embodiment, a first decision tree model is trained according to each indicator sample subset to obtain a decision tree model for each indicator sample subset, including: in the indicator sample subset to be classified, based on the decision tree model to be trained for predicting the indicator risk value, the kth indicator feature sample is classified and predicted to obtain the kth probability; the kth probability represents the probability of the category to which the kth indicator feature sample belongs; the difference between the kth probability and the preset probability value is determined, and the kth probability is combined with the difference value to obtain a sample subset classification index; according to the sample subset classification index, the decision tree model to be trained for predicting the indicator risk value is adjusted to obtain the decision tree model of the indicator sample subset; k is a positive integer.

[0118] Step 506: perform multiple sample extractions on the risk value sample set to obtain multiple risk value sample subsets.

[0119] The risk value sample subset is a sample set obtained by multiple sampling with replacement from the risk value sample set. Optionally, the risk value sample subset contains sample data in the risk value sample set, and some samples of the risk value sample subset are repeated. Optionally, multiple training subsets in the risk value sample set are obtained by Bootstrap sampling. iDraw samples with replacement to generate b different training subsets Where b is the number of training subsets. Each training subset contains N samples.

[0120] Step 508, training the second decision tree model according to each risk value sample subset to obtain the decision tree model of each risk value sample subset; the random forest model composed of the decision tree models of each risk value sample subset is the core risk prediction model.

[0121] Optionally, the decision tree model of each indicator sample subset is the averaging result of the indicator risk value predicted by the indicator risk prediction model based on the second indicator contained in the same first indicator, which is the indicator risk value predicted by the indicator risk prediction model; the decision tree model of each risk value sample subset is the averaging result of the core risk value predicted by the first indicator of the same type, which is the core risk value of flight operation control predicted by the core risk prediction model.

[0122] In one embodiment, the second decision tree model is trained according to each risk value sample subset to obtain the decision tree model of each risk value sample subset, including: in the i-th risk value sample subset to be regressed, based on the decision tree model to be trained for predicting the core risk value of flight operation control, the q-th indicator risk value sample is regressed and predicted to obtain the q-th regression value; according to the difference value between the core risk value sample of the q-th indicator risk value sample and the q-th regression value, the decision tree model for predicting the core risk value of flight operation control is trained to obtain the decision tree model of the i-th risk value sample subset; wherein i and q are positive integers.

[0123] In one embodiment, Figure 6 As shown, according to each risk value sample subset, the second decision tree model is trained to obtain the decision tree model of each risk value sample subset, including: in the risk value sample subset to be classified, based on the decision tree model to be trained for the core risk value of flight operation control, the indicator risk value sample of the kth indicator sample is classified and predicted to obtain the kth probability; the kth probability represents the probability of the category to which the indicator risk value sample of the kth indicator sample belongs; the difference value between the kth probability and the preset probability value is determined, and the kth probability is combined with the difference value to obtain a risk classification index; according to the risk classification index, the decision tree model to be trained for predicting the core risk value of flight operation control is adjusted to obtain the decision tree model of the risk value sample subset.

[0124] In an optional embodiment, the first decision tree model is used to obtain the decision tree in the first layer random forest model, the second decision tree model is used to obtain the decision tree in the second layer random forest model, and the training process of the first decision tree model and the second decision tree model is the same.

[0125] Random forest (RF) is based on the bagging ensemble theory. It uses bootstrap sampling technology to extract multiple different training data sets from the original data set with replacement, and then combines the random subspace method to model the decision tree for each bootstrap data set to form a random forest. When predicting the model, the test data is input into the model, and the output categories of multiple decision trees are voted to obtain the final prediction result. Compared with the classic decision tree classifier, random forest has excellent performance in generalization ability, classification effect, noise tolerance, etc.

[0126] (1) Random forest construction process

[0127] Assume that the original dataset D has N samples, consisting of M input features and a classification label Y. Random forest combines multiple independently trained decision trees to form a forest. The construction process of each tree can be regarded as a partition of the data space. In other words, a leaf represents a partition of the complete data space, and each node corresponds to a hyperrectangular unit of the data space.

[0128] Before building the decision tree, use the bootstrap method sampling technique to extract K training data sets from the original data set D with replacement, and the number of samples in each training subset is also N. Use these bootstrap samples to train the decision tree;

[0129] The classification and regression tree (CART) method is used to construct the base classifier model. At the node of the tree, m features (m≤M) are randomly selected from the M input features as the split feature set of the current node of the decision tree, and the optimal split feature and split point are selected to divide the training data set into two child nodes. The criteria for selecting split features and split points are the Gini index (Giniindex) minimization criterion (for classification) or the mean square error (MSE) minimization criterion (for regression). Repeat the above division process until the stopping condition is met;

[0130] Train the decision tree model with K bootstrap sample sets according to step 2, and combine all generated decision trees into a random forest model {t i ,i=1,2,...,K}. Input the test sample x into the model and get the corresponding classification result {t1(x),t2(x),...,t K (x)};

[0131] The RF algorithm makes a final prediction by counting the results of each tree. Generally, a simple majority voting method is used to take the mode of the predicted classification results as the final classification result, that is, Or use the simple averaging method to take the average of the predicted result values ​​as the final regression result, that is,

[0132] In this embodiment, multiple sample extractions are performed using these two sample sets, so that each feature of the sample set can exist in different indicator sample subsets at the same time, so that data can be reused repeatedly, improving sample efficiency. At the same time, the two decision tree models are trained based on their respective sample subsets to form a two-layer random forest model. The training data of the two layers of random models do not interfere with each other, and the subsets used by their respective decision trees do not interfere with each other, which helps to further increase the stability of the training and make the prediction results more accurate.

[0133] In an optional embodiment, the first decision tree model is trained according to each indicator sample subset to obtain a decision tree model for each indicator sample subset, including:

[0134] In the case where the first decision tree model is a regression tree model, the first decision tree model is trained according to each indicator sample subset and the square error loss function to obtain a regression tree model of each indicator sample subset; in the case where the first decision tree is a classification tree model, the Gini index of the first decision tree model in the classification process is determined based on the indicator sample subset, and the first decision tree model is trained according to the Gini coefficient of the first decision tree model in the classification process to obtain a classification tree model of each indicator sample subset;

[0135] Correspondingly, according to each risk value sample subset, the second decision tree model is trained to obtain the decision tree model of each risk value sample subset, including:

[0136] When the second decision tree model is a regression tree model, the second decision tree model is trained according to each risk value sample subset and the square error loss function to obtain a regression tree model for each indicator risk value subset; when the second decision tree is a classification tree model, the Gini index of the second decision tree model in the classification process is determined based on the indicator sample subset, and according to the Gini coefficient of the second decision tree in the classification process, the second decision tree is trained to obtain a classification tree model for each risk value sample subset.

[0137] The square error loss function is the mean square error loss function (mean-square error, MSE)

[0138] The Gini index is used to select the best feature partitioning node during the decision tree construction process.

[0139] In an optional embodiment, the Gini index of the first decision tree model in the classification process is determined based on the indicator sample subset, including: in the indicator sample subset to be classified, based on the first decision tree model for predicting the indicator risk value, classifying and predicting the kth indicator feature sample to obtain the kth probability; the kth probability represents the probability of the category to which the kth indicator feature sample belongs; determining the difference between the kth probability and the value 1, and multiplying the kth probability by the difference to obtain the Gini index.

[0140] In an optional embodiment, the Gini index of the second decision tree model in the classification process is determined based on the indicator sample subset, including: in the indicator sample subset to be classified, based on the second decision tree model for predicting the indicator risk value, classifying and predicting the kth indicator feature sample to obtain the kth probability; the kth probability represents the probability of the category to which the kth indicator feature sample belongs; determining the difference between the kth probability and the value 1, and multiplying the kth probability by the difference to obtain the Gini index.

[0141] In this embodiment, the square error loss function is used as the regression loss function, and the Gini index is used as the classification loss function; since both the indicator sample subset and the risk value sample subset may have classification problems and regression problems, these two types of data are used at the same time to improve accuracy.

[0142] In an exemplary embodiment, a flight operation control core risk assessment prediction model based on a hierarchical random forest algorithm (HRF). The model of this embodiment is based on the classic random forest algorithm, combined with the characteristics of the flight operation control core risk indicator system with many indicators and hierarchical existence, to construct a hierarchical multi-cluster independent random forest. The first layer of random forest independently predicts each first-level indicator, and then the prediction results of the first-level indicators are quantized and standardized, and the second layer of random forest is used for prediction to obtain the final core risk prediction results. The core risk prediction model process based on the HRF algorithm is as follows: Figure 7 shown.

[0143] Optionally, the first indicator is a first-level indicator, the second indicator is a second-level indicator; the indicator sample set is a first-level indicator sample set; the risk value sample set is an overall sample set; the indicator risk value is the score of the first-level indicator; and the core risk value is the overall score of the expert.

[0144] Correspondingly, step 202 specifically includes:

[0145] (1) Multi-source heterogeneous data collection. Based on the core risk indicators, we set up tracking points in the flight operation control system and other systems, and use scales to collect and record dispatch logs, FPL messages, flight warnings, performance records, flight information and other data. We then filter out valid data, process the data with default values ​​and abnormal values, and obtain the quantitative data sets D for each first-level indicator. 1 ,D 2 ,...,D m , where m is the number of first-level indicators. Then the data set of first-level indicator i in For the dataset D i A feature vector of D, n(i) is the number of features, j = 1, 2, ..., N, N is the data set D i The capacity of each first-level indicator data set D i The overall data set D is formed by horizontal splicing:

[0146]

[0147] (2) Model sample set construction. The core risk data set is obtained, and experts are organized to conduct a survey on each first-level indicator data set D i Evaluate and score the risk value of each sample in the overall data set D to obtain a typical core risk level 1 indicator sample set S including expert scores i , and then horizontally splice the scores of each first-level indicator data set with the scores of the overall data set to form the overall sample set S.

[0148] Step 204 specifically includes:

[0149] (3) The first-level Bootstrap sampling technique is used to obtain multiple training subsets. i Draw samples with replacement to generate b different training subsets Where b is the number of training subsets. Each training subset contains N samples.

[0150] (4) First-level random forest training. As input, use the CART algorithm to generate b decision trees, and use the square error minimization criterion for the regression tree. Perform feature selection, where Q is the number of units divided into feature space, R q is a partition of the feature space, c q For R q A fixed output value; Gini index minimization criterion for classification trees Perform feature selection, where K is the number of sample classes, p kis the probability that the sample point belongs to the kth class. The decision trees generated by the sample subsets of all first-level indicator sample sets are combined into the first-layer random forest model {tr 1 ,tr 2 ,...,tr m},in is the first-level indicator sample set S i The corresponding random forest model.

[0151] (5) The first-layer random forest regression results. 1 ,tr 2 ,...,tr m The output result uses the simple average method to take the average value of the predicted result as the risk prediction value of each first-level indicator in i=1,2,...,m, b is the number of decision trees. The prediction results of the first-level indicators are combined into the input vector of the second-layer random forest. Where m is the number of first-level indicators.

[0152] Step 206 specifically includes:

[0153] (6) The second layer of Bootstrap sampling obtains multiple training subsets. Similar to step 3, Bootstrap sampling technology is used to extract samples from the original overall sample set S with replacement to generate b different training subsets Where b is the number of training subsets. Each training subset contains N samples.

[0154] (7) Second-level random forest training. Similar to step 4, the training subset S a As input, use the CART algorithm to generate b decision trees, and use the square error minimization criterion for the regression tree. Perform feature selection, where Q is the number of units divided into feature space, R q is a partition of the feature space, c q For R q A fixed output value; Gini index minimization criterion for classification trees Perform feature selection, where K is the number of sample classes, p k is the probability that the sample point belongs to the kth class. The decision trees generated by the sample subsets of the overall sample set S are combined into the second-layer random forest model tr = {tr j ,j=1,2,...,b}

[0155] (8) Second-level random forest regression results. The results of the second-level random forest model tr are output using the simple average method to take the average of the predicted results as the overall prediction value of the core risk. b is the number of decision trees.

[0156] Thus, the first and second layer random forest models are combined into a hierarchical random forest model HRF = {tr 1 ,tr 2 ,...,tr m ,tr}, the input is the feature vector composed of each indicator data, and the output is the predicted value of the core risk first-level indicator and the overall predicted value. The output example is as follows Figure 8 shown.

[0157] Therefore, on the one hand, the established flight operation control core risk indicator system can serve as an important starting point for the airline flight operation control safety performance management, transforming the traditional safety management model from experience-oriented to data-oriented, and meeting the requirements of timely risk monitoring and early warning. On the other hand, through the flight operation control core risk assessment and prediction model based on hierarchical random forest (HRF), the current and future operational risk changes and trends can be accurately predicted, and the first-level indicators of each core risk and the overall risk situation can be quickly and accurately obtained, helping dispatchers to quickly locate weak links, take control measures in a timely manner, and improve the safety performance management of operation control and the effectiveness of risk early warning and prevention.

[0158] In one embodiment, Fig. 9 As shown, the present application also provides a flight operation control core risk detection method, including:

[0159] Step 902, obtaining the characteristics of the indicators to be detected for the core risks of flight operation control.

[0160] The indicator feature to be detected is a sample feature used for prediction; the sample feature of the indicator feature to be detected is a feature item used to describe the indicator of the core risk of flight operation control, which can describe the indicator from multiple dimensions, thereby more accurately reflecting the indicator attribute. Optionally, the indicator feature is set according to each indicator separately, so as to form the feature of each indicator item. Optionally, the indicator feature to be detected includes the feature of the first indicator and the feature of the second indicator.

[0161] Optionally, the indicator feature to be detected may be input data.

[0162] Step 904: Determine the indicator risk value corresponding to the indicator feature to be detected through the indicator risk prediction model.

[0163] In an optional embodiment, the indicator risk value corresponding to the indicator feature to be detected is determined through the indicator risk prediction model, including: inputting the indicator feature to be detected into each classification tree model in the indicator risk prediction model, performing classification prediction through each classification tree model, and outputting each category label; according to the number of the category labels, determining the indicator risk value corresponding to the indicator feature to be detected.

[0164] In an optional embodiment, the indicator risk value corresponding to the indicator feature to be detected is determined through the indicator risk prediction model, including: inputting the indicator feature to be detected into each regression tree model in the indicator risk prediction model, performing regression prediction through each regression tree model, and outputting each regression value; averaging the regression value, and determining the indicator risk value corresponding to the indicator feature to be detected based on the averaging result.

[0165] Step 906, converting the indicator risk value into a flight operation control core risk value through a core risk prediction model; wherein the indicator risk prediction model and the core risk prediction model are obtained by training based on any embodiment of the above-mentioned model training method.

[0166] In an optional embodiment, the indicator risk value is converted into a flight operation control core risk value through a core risk prediction model, including: inputting the indicator risk value into each classification tree model in the core risk prediction model, performing classification prediction through each classification tree model, and outputting each category label; and obtaining the flight operation control core risk value based on the number of category labels.

[0167] In an optional embodiment, the indicator risk value is converted into a core risk value for flight operation control through a core risk prediction model, including: inputting the indicator risk value into each regression tree model in the core risk prediction model, performing regression prediction through each regression tree model, and outputting each regression value; averaging the regression value, and obtaining the core risk value for flight operation control based on the averaging result.

[0168] In this embodiment, since the two-layer model of the indicator risk prediction model and the core risk prediction model are trained based on different data sets respectively, the prediction results are more stable; at the same time, the prediction results of each indicator feature sample can be obtained through the first-layer model, which is convenient for accurately locating the specific safety shortcomings in the indicator features to be detected; and the results output by the indicator risk prediction model are input into the core risk prediction model for prediction, which can ensure the high accuracy of the prediction results.

[0169] Optionally, the expert-scored dataset is used as a sample to conduct a validation experiment on the HRF algorithm, and experiments are conducted with the other six algorithms in the same environment, and finally the performance of the seven algorithms is compared.

[0170] In this experimental environment, the hardware platform used is a Windows 11 64-bit laptop with a CPU of 11th Gen Intel(R) Core(TM) i7-1165G72.80GHz and 16GB of memory. The language used is Python 3.11 and the programming environment is PyCharm 2023.1.4.

[0171] In this embodiment, the data obtained by Airline X based on the operation situation is selected as samples, and the dispatch experts score to construct the sample set Core Risk. The sample set Core Risk contains 77 features and 200 samples. After being subdivided according to the first-level indicators, the sample set Core Risk can be divided into 6 sub-sample sets Core Risk-Sub1-6 and 1 first-level indicator prediction value sample set Core Risk-Sub-Pred, and the sample order of each sample set corresponds one to one. The relevant information of the experimental sample set is shown in Table 3.

[0172] Table 3 Experimental data information

[0173]

[0174]

[0175] It can be seen that the evaluation of the regression model can be completed by calculating the difference between the predicted value and the actual value. This paper uses the root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE).

[0176] Let N be the number of samples, y i is the actual value, y i ′ is the predicted value, then the formulas for mean absolute error MAE, root mean square error RMSE and mean absolute percentage error MAPE are as follows.

[0177] (1)RMSE

[0178] The root mean square error refers to the square root of the expected value of the square of the difference between the actual value and the predicted value. It is a standard for measuring the average error. It can better reflect the degree of change in the data. The smaller the result value, the higher the accuracy of the prediction algorithm. The formula is as follows:

[0179]

[0180] (2)MAE

[0181] The mean absolute error refers to the average of the absolute values ​​of the difference between the actual value and the predicted value. It can accurately reflect the size of the error in the algorithm's prediction results and better reflect the actual situation of the prediction value error. The formula is as follows:

[0182]

[0183] (3)MAPE

[0184] The mean absolute percentage error refers to the ratio of the difference between the actual value and the predicted value to the absolute value of the actual value to measure the quality of the prediction results of an algorithm model. The formula is as follows:

[0185]

[0186] Finally, the experiment used 5-fold stratified cross-validation to obtain the evaluation results by taking the average. Each time, one-fifth of the data was extracted from each sample set as the test set, and the remaining data was used as the training set. The algorithm models were trained and tested separately, and the regression evaluation indicators were used to analyze the performance of the model. All data results are the average of 5 experiments.

[0187] We designed 7 algorithms to build models for comparison and verification, including the Hierarchical Random Forest (HRF) algorithm proposed in the implementation example, the classic Random Forest (RF), the K-Nearest Neighbors (KNN), the Decision Tree (DT), the Gradient Boosting (GB), and the Elastic Net (EN). Among them, HRF is trained using the Core Risk-Sub1-6 and Core Risk-Sub-Pred sample sets due to the characteristics of the hierarchical structure of the algorithm, and the other algorithms are trained using the Core Risk sample set. Finally, the test set divided from the Core Risk sample set is used for testing and verification.

[0188] The algorithm comparison results are shown in Table 4. The HRF algorithm designed in this embodiment achieves the best results in all three indicators, and the performance of the classic random forest algorithm is second only to the HRF algorithm. It can be seen that the random forest algorithm has a good performance in the core risk assessment and prediction problem of flight operation control, and after the algorithm model is hierarchically improved, the HRF algorithm model obtained can further improve the prediction accuracy.

[0189] Table 4 Comparison of model prediction performance

[0190]

[0191]

[0192] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0193] Based on the same inventive concept, the embodiment of the present application also provides a model training and flight operation control core risk detection device for implementing the model training and flight operation control core risk detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the model training and flight operation control core risk detection device provided below can refer to the limitations of the model training and flight operation control core risk detection method above, and will not be repeated here.

[0194] In an exemplary embodiment, Fig.10 As shown, a model training device is provided, the device comprising:

[0195] The sample acquisition module 1002 is used to acquire an indicator sample set and a risk value sample set of the core risk of flight operation control; the indicator sample set includes an indicator feature sample and an indicator risk value sample corresponding to the indicator feature sample, and the risk value sample set includes the indicator risk value sample and a core risk value sample corresponding to the indicator risk value sample;

[0196] The training module 1004 is used to train the first initial model based on the indicator sample set to obtain an indicator risk prediction model; the indicator risk prediction model is used to predict the indicator risk value;

[0197] The training module 1004 is used to train the second initial model based on the risk value sample set to obtain a core risk prediction model; the core risk prediction model is used to predict the flight operation control core risk value based on the indicator risk value.

[0198] In one embodiment, the sample acquisition module 1002 is used to:

[0199] Obtaining a first indicator sample of the flight operation control core risk and a second indicator sample contained in the first indicator sample;

[0200] Obtaining an indicator risk value sample and a core risk value sample corresponding to the first indicator sample, and determining a risk value sample set according to the indicator risk value sample and the core risk value sample;

[0201] An indicator feature sample is determined based on the second indicator sample, and an indicator sample set is determined according to the indicator feature sample and the indicator risk value sample.

[0202] In one embodiment, the generation process of the first indicator sample has multiple sub-processes, the multiple sub-processes are associated with the core risk source of flight operation control, the core risk source of flight operation control has corresponding control measures, and the implementation data of the control measures and the existence of the implementation data belong to the second indicator sample.

[0203] In one embodiment, the training module 1004 is used to:

[0204] Extracting samples from the indicator sample set multiple times to obtain multiple indicator sample subsets;

[0205] According to each of the indicator sample subsets, the first decision tree model is trained to obtain the decision tree model of each of the indicator sample subsets; the random forest model composed of the decision tree models of each of the indicator sample subsets is the indicator risk prediction model;

[0206] Extracting samples from the risk value sample set multiple times to obtain multiple risk value sample subsets;

[0207] According to each of the risk value sample subsets, the second decision tree model is trained to obtain the decision tree model of each of the risk value sample subsets; the random forest model composed of the decision tree models of each of the risk value sample subsets is the core risk prediction model.

[0208] In one embodiment, the training module 1004 is used to:

[0209] In the case where the first decision tree model is a regression tree model, the first decision tree model is trained according to each of the indicator sample subsets and the square error loss function to obtain a regression tree model of each of the indicator sample subsets;

[0210] In the case where the first decision tree belongs to a classification tree model, the Gini index of the first decision tree model in the classification process is determined based on the indicator sample subset, and the first decision tree model is trained according to the Gini coefficient of the first decision tree model in the classification process to obtain the classification tree model of each of the indicator sample subsets;

[0211] In the case where the second decision tree model is a regression tree model, the second decision tree model is trained according to each risk value sample subset and the square error loss function to obtain a regression tree model of each indicator risk value subset;

[0212] In the case where the second decision tree belongs to a classification tree model, the Gini index of the second decision tree model in the classification process is determined based on the indicator sample subset, and the second decision tree is trained according to the Gini coefficient of the second decision tree in the classification process to obtain the classification tree model of each risk value sample subset.

[0213] In an exemplary embodiment, Fig.11 As shown, the present application also provides a flight operation control core risk detection device, the device comprising:

[0214] An acquisition module 1102 is used to acquire the characteristics of indicators to be detected of the core risks of flight operation control;

[0215] The first detection module 1104 is used to determine the indicator risk value corresponding to the to-be-detected data set through an indicator risk prediction model;

[0216] The second detection module 1106 is used to convert the indicator risk value into a flight operation control core risk value through a core risk prediction model; wherein the indicator risk prediction model and the core risk prediction model are trained based on the method of the first aspect.

[0217] Each module in the above-mentioned model training device or flight operation control core risk detection device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0218] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.12As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model training or flight operation control core risk detection method is implemented.

[0219] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not define the limitation of the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0220] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0221] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0222] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0223] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0224] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited thereto.

[0225] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0226] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A model training method, characterized in that: The method comprises: Obtaining an indicator sample set and a risk value sample set of the core risks of flight operation control; the indicator sample set includes an indicator feature sample and an indicator risk value sample corresponding to the indicator feature sample, and the risk value sample set includes the indicator risk value sample and a core risk value sample corresponding to the indicator risk value sample; Based on the indicator sample set, the first initial model is trained to obtain an indicator risk prediction model; the indicator risk prediction model is used to predict the indicator risk value; Based on the risk value sample set, a second initial model is trained to obtain a core risk prediction model; the core risk prediction model is used to predict the flight operation control core risk value based on the indicator risk value.

2. The method according to claim 1, characterized in that The indicator sample set and risk value sample set for obtaining the core risks of flight operation control include: Obtaining a first indicator sample of the flight operation control core risk and a second indicator sample contained in the first indicator sample; Obtaining an indicator risk value sample and a flight operation control core risk value sample corresponding to the first indicator sample, and determining a risk value sample set according to the indicator risk value sample and the core risk value sample; An indicator feature sample is determined based on the second indicator sample, and an indicator sample set is determined according to the indicator feature sample and the indicator risk value sample.

3. The method according to claim 2, characterized in that There are multiple sub-processes in the generation process of the first indicator sample, and the multiple sub-processes are associated with the core risk source of flight operation control. There are corresponding control measures for the core risk source of flight operation control. The implementation data of the control measures and the existence of the implementation data belong to the second indicator sample.

4. The method according to claim 1, characterized in that: The first initial model is trained based on the indicator sample set to obtain an indicator risk prediction model, including: Extracting samples from the indicator sample set multiple times to obtain multiple indicator sample subsets; According to each of the indicator sample subsets, the first decision tree model is trained to obtain the decision tree model of each of the indicator sample subsets; the random forest model composed of the decision tree models of each of the indicator sample subsets is the indicator risk prediction model; The second initial model is trained based on the risk value sample set to obtain a core risk prediction model, including: Extracting samples from the risk value sample set multiple times to obtain multiple risk value sample subsets; According to each of the risk value sample subsets, the second decision tree model is trained to obtain the decision tree model of each of the risk value sample subsets; the random forest model composed of the decision tree models of each of the risk value sample subsets is the core risk prediction model.

5. The method according to claim 4, characterized in that The first decision tree model is trained according to each of the indicator sample subsets to obtain the decision tree model of each of the indicator sample subsets, including: In the case where the first decision tree model is a regression tree model, the first decision tree model is trained according to each of the indicator sample subsets and the square error loss function to obtain a regression tree model of each of the indicator sample subsets; In the case where the first decision tree belongs to a classification tree model, the Gini index of the first decision tree model in the classification process is determined based on the indicator sample subset, and the first decision tree model is trained according to the Gini coefficient of the first decision tree model in the classification process to obtain the classification tree model of each of the indicator sample subsets; The step of training the second decision tree model according to each risk value sample subset to obtain the decision tree model of each risk value sample subset includes: In the case where the second decision tree model is a regression tree model, the second decision tree model is trained according to each risk value sample subset and the square error loss function to obtain a regression tree model of each indicator risk value subset; In the case where the second decision tree belongs to a classification tree model, the Gini index of the second decision tree model in the classification process is determined based on the indicator sample subset, and the second decision tree is trained according to the Gini coefficient of the second decision tree in the classification process to obtain the classification tree model of each risk value sample subset.

6. A flight operation control core risk detection method, characterized in that: include: Obtain the characteristics of indicators to be tested for the core risks of flight operation control; Determine the indicator risk value corresponding to the indicator feature to be detected through the indicator risk prediction model; The indicator risk value is converted into a flight operation control core risk value through a core risk prediction model; Among them, the indicator risk prediction model and the core risk prediction model are trained based on any one of claims 1 to 5.

7. A model training device, characterized in that: The device comprises: A sample acquisition module, used to acquire an indicator sample set and a risk value sample set of the core risk of flight operation control; the indicator sample set includes an indicator feature sample and an indicator risk value sample corresponding to the indicator feature sample, and the risk value sample set includes the indicator risk value sample and a core risk value sample corresponding to the indicator risk value sample; A training module, used to train the first initial model based on the indicator sample set to obtain an indicator risk prediction model; the indicator risk prediction model is used to predict the indicator risk value; The training module is used to train the second initial model based on the risk value sample set to obtain a core risk prediction model; the core risk prediction model is used to predict the flight operation control core risk value based on the indicator risk value.

8. A flight operation control core risk detection device, characterized in that: The device comprises: An acquisition module is used to obtain the characteristics of indicators to be detected for the core risks of flight operation control; A first detection module, used to determine the indicator risk value corresponding to the to-be-detected data set through an indicator risk prediction model; A second detection module is used to convert the indicator risk value into a flight operation control core risk value through a core risk prediction model; Among them, the indicator risk prediction model and the core risk prediction model are trained based on any one of claims 1 to 5.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images