METHOD FOR MAINTAINING AERONAUTICAL EQUIPMENT USING TEXTUAL DESCRIPTIONS

Abstract

The invention relates to a method (10) for maintaining aeronautical equipment that establishes an acceptance threshold (Si) which determines, for each label (LABx), i.e., useful information to be predicted, whether or not it should be predicted. Thus, the method according to the invention does not simply predict, as is generally the case in known models, the label with the highest score, but predicts all the labels (LABx) that have a score greater than or equal to the acceptance threshold. Therefore, the operator is able to diagnose whether a failure has occurred on the aeronautical equipment since, in this case, the number of predicted label(s) is non-zero. Once a failure is observed, the operator is also able to determine what action must be taken on the equipment. Thus, the method according to the invention improves the operational availability of equipment while reducing maintenance costs. Figure for the abstract: Fig. 3

Description

Title of the invention: METHOD FOR MAINTAINING AERONAUTICAL EQUIPMENT USING TEXTUAL DESCRIPTIONS Technical field of the invention

[0001] The invention relates to the field of aircraft maintenance. More specifically, the technical field of the invention relates to the maintenance processes of aircraft equipment. The invention therefore provides a decision support method for identifying faulty equipment on an aircraft.

[0002] However, as will be better understood in the detailed description, the invention can be applied to the maintenance of any type of equipment. Technical background

[0003] An aircraft comprises a wide variety of equipment that can experience failures. The detection and diagnosis of failures in certain aircraft equipment are carried out by automated systems located within the aircraft. However, failures may be detected by these automated systems even if the equipment considered to be faulty is not actually malfunctioning. Furthermore, in many cases, a failure of an aircraft component can only be diagnosed based on human observations, generally made by an operator. Maintenance reports are therefore written in natural language. These maintenance reports contain application-specific terms, abbreviations and acronyms, and potentially grammatical errors.

[0004] The classification of maintenance reports is generally done by humans, which poses a number of problems. When a human classifies an existing database, they define certain classification rules that may need to be replaced by new rules to meet a specific need. These new classification rules may not be feasible for human intervention or may generate significant human work if the database has grown considerably in the meantime. In other words, the time required to reclassify the entire database makes the operation impossible for a human to perform.

[0005] The prior art then proposed using learning methods based on NLP (Natural Language Processing) neural networks. There are two main families of methods: unsupervised learning methods and supervised learning methods.

[0006] Unsupervised learning methods do not require database annotations. Thus, for example, to automatically analyze technical documentation with the aim of automatically labeling test reports of events that occurred during testing, according to anomaly typology, it is possible to implement an unsupervised classification model (known as a "topic modeling" model) of documents. In practice, an automatic classification of test reports by anomaly group can be implemented and then inferred by a business user. Alternatively, it is also possible to use a similarity search approach between each test report and the closest anomaly, provided that both have been coded. The drawback of unsupervised learning methods lies in the inability to quantify the model's performance.This makes it difficult to compare models with each other in terms of performance.

[0007] Supervised learning methods use a pre-trained natural language processing model, then fine-tune it for a specific task such as text classification. The prior art discloses supervised learning methods that mostly use very similar text data classification models.

[0008] Databases progressively enriched by maintenance reports manually compiled by operators are generally smaller compared to a database such as the ASRS (Aviation Safety Reporting System). Furthermore, they include textual data for which grammar, syntax, and spelling are not necessarily consistent and are subject to variations in format between maintenance operators, as well as variations between airlines and equipment manufacturers. In addition, they include specific vocabulary, abbreviations, and technical terms, and / or terms that may be unique to a small community of maintenance operators who have developed their own jargon. For example, an acronym used for a helicopter engine may have a different meaning for an airplane engine.Databases gradually enriched by maintenance reports may also contain information not useful for defining or characterizing the observed failure. Furthermore, they cannot be supplemented by any data other than that contained in maintenance reports. Therefore, the maintenance reports constitute the entirety of the database.

[0009] The document Tikayat Ray, A. et al. aeroBERT -Classifier: Classification of Aerospace Requirements Using BERT. Aerospace 2023, 10, 279 discloses the application of a pre-trained natural language model of the BERT type (from the English "Bidirectional Encoder Representations from Transformers") in the domain Aerospace. A specialized BERT model for this field, called aeroBERT-Classifier, is refined on a small, labeled database to classify textual data. However, because the textual data relates to aerospace requirements, it is well-structured and standardized, which is far removed from real-world use cases where the database is progressively enriched by maintenance reports manually created by operators.

[0010] In general, the aforementioned prior art work does not explain how to obtain a usable database with a pre-trained natural language model derived from real-world data, i.e., data collected in the field or found in archives. Furthermore, the models are refined for a relatively simple classification task. Moreover, the prior art does not propose an end-use application in inference, as it only includes performance studies, without any suggestions for field applications.

[0011] Recently, a system for determining faulty aeronautical equipment has been proposed. The system is a tool to assist in the maintenance of aeronautical equipment, but it does not describe in detail how the natural language processing algorithm integrated into the maintenance tool should be used. Although such documentation relates more specifically to the technical field of the invention, namely, assisting in the maintenance of aircraft equipment, the model(s) used and how they are trained are not described. Thus, currently, to perform maintenance on aeronautical equipment, the maintenance operator often has to refer to instructions in the technical documentation for the aeronautical equipment. However, this documentation is difficult to understand.

[0012] Furthermore, many pieces of equipment are mistakenly sent to the supplier for maintenance, resulting in longer maintenance times and additional costs. Indeed, faced with maintenance time constraints, operators tend to send an excessive number of pieces of equipment for repair or replacement in order to correct the fault as quickly as possible within the allotted time.

[0013] The invention proposes a method for maintaining aeronautical equipment, allowing in particular for faster diagnosis of a failure of one or more pieces of equipment, and for identifying the equipment that is actually faulty and needs to be repaired or replaced. Summary of the invention

[0014] In this respect, the invention proposes a method for maintaining aeronautical equipment comprising the following steps:

[0015] - to prepare a maintenance report based on an equipment inspection,

[0016] - to obtain, through a pre-trained machine learning process, a plurality of labels corresponding to the previously prepared maintenance report,

[0017] - determine, using a linking database comprising data training and calibration, the maintenance actions associated with the labels,

[0018] characterized in that the pre-trained machine learning process includes:

[0019] - refining in classification, from the training data, at least one model of multilayer neural networks for labeled language processing and,

[0020] - to calibrate said at least one neural network model so that it predicts all labels above an acceptance threshold corresponding to a reference score for which a percentage of labels predicted in accordance with the labels from the calibration data is maximal.

[0021] The method according to the invention overcomes the drawbacks of the prior art by assisting the operator in diagnosing and resolving faults. The method establishes an acceptance threshold that determines, for each label, whether or not it should be predicted. Thus, the method according to the invention does not simply predict, as is generally the case in known models, the label with the highest score, but predicts all labels with a score greater than or equal to the acceptance threshold. These labels can correspond to the diagnosis of a fault. Once a fault is observed, the method can also determine what action(s) must be taken on the equipment, for example, whether the equipment must be permanently removed from the aircraft and replaced with other equipment, or whether the equipment can be repaired.Thus, the maintenance process according to the invention makes it possible to improve the operational availability of equipment while reducing maintenance costs.

[0022] In addition, it makes it possible to capture and capitalize on the business knowledge and feedback of aeronautical equipment maintenance operators, which is not the case today, via automatic language processing technologies applied to maintenance reports.

[0023] According to various features of the invention which may be taken together or separately: • the refinement step is repeated for the neural network model(s) for a number of iterations and / or an iteration time; • the linking database is divided into N data partitions, with the neural network model(s) being fine-tuned for each of the data partitions so that a score or average score is produced for each label in each partition; • during the refinement stage, refinement is performed for a plurality of neural network models, with a distinct neural network model being refined for each of the data partitions; • during the refinement stage, refinement is performed for a plurality of neural network models, with one neural network model being refined for each of the data partitions; • the process includes, after the calibration step, a model fusion step for which a percentage of labels predicted in accordance with the labels from the calibration data is maximized; • each data partition includes all the data from the link database, said link database data being partitioned into training data and calibration data so that each data from the link database (DB) is either training data or calibration data; • in each data partition, the data from the first data group is partitioned into training data, calibration data and validation data used to test the previously refined neural network model(s); • the model(s) is chosen from the BERT, SciBERT and SafeAeroBERT, GPT models or their variants; • the process further includes a pre-processing step of the linking database comprising a sub-step consisting of generating or enriching a dictionary of acronyms and abbreviations; • The pre-processing step of the linking database includes a sub-step consisting of replacing the acronyms and abbreviations present in the textual data with their full meaning; • the pre-processing step of the linking database includes a sub-step consisting of generating synonyms for at least part of the textual data composing the linking database from a dictionary of synonyms; • the pre-processing step of the linking database includes a sub-step consisting of generating translations of at least a part of said textual data from at least one translation dictionary; • The pre-processing step of the linking database includes a sub-step consisting of performing automatic spelling correction of the data in the linking database. Brief description of the figures

[0024] Other objects, features and advantages of the invention will become more apparent in the following description, made with reference to the accompanying figures, in which:

[0025] - Figure 1 illustrates the general architecture of a linking database used in a maintenance process according to the invention,

[0026] - Figure 2a illustrates partitions of the linking database used in the maintenance method according to an example of an embodiment of the invention,

[0027] - Figure 2b illustrates partitions of the linking database used in the maintenance method according to another embodiment of the invention,

[0028] - [Fig. 3] is a schematic view of the different stages of a process of maintenance according to a general embodiment of the invention,

[0029] - [Fig. 4] is a detailed schematic view of the different sub-steps of a pre-treatment stage of the maintenance process according to an embodiment of the invention,

[0030] - [Fig. 5] is a detailed schematic view of the optimization of the acceptance threshold in an alternative embodiment of the maintenance process according to the invention,

[0031] - [Fig. 6] is a detailed schematic view of the optimization of the acceptance threshold in another embodiment of the maintenance process according to the invention,

[0032] - [Fig.7] illustrates an example of determining an average acceptance threshold Smoy.

[0033] In all the attached figures, the elements that are indicated by rectangles in Dashed lines are optional. Similarly, the steps of the process according to the invention that are indicated by dotted rectangles are optional. Detailed description of the invention

[0034] The invention relates to a method 10 for maintaining aeronautical equipment.

[0035] Within the scope of the invention, the aircraft can be any type of aircraft, more particularly an airplane or a helicopter.

[0036] Equipment can be any hardware or software equipment of the aircraft (sensors, software, etc.) that may be liable to fail.

[0037] A failed equipment state corresponds to a state of the equipment that prevents it from performing one or more of its functions. The failed equipment state may induce an observable failure or only a lack of performance; in both cases, the equipment must be replaced or repaired according to the thresholds defined in the business protocols.

[0038] The invention particularly aims to limit, or even prevent, sending equipment for maintenance when it is not faulty. This can occur, in particular, when a fault exists in one of the aircraft modules but the actual cause of the fault cannot be identified. When equipment is sent for Incorrect maintenance results in an event called "No Fault Found (NFF)". Sending an excessive number of pieces of equipment for maintenance leads to additional costs for the aircraft operator.

[0039] With reference to [Fig.3], the maintenance process 10 includes a step 100 which consists of preparing an RM maintenance report from an inspection of the equipment.

[0040] Maintenance reports (MRs) are generally written in natural language. They consist of a plurality of textual data (TDs) or TD texts written in natural language. The textual data (TDs) may correspond to words and / or groups of words. The textual data (TDs) typically characterize the aircraft, the aircraft equipment, and the failures affecting the equipment. More generally, the textual data (TDs) provide information useful for diagnosing equipment and thus assist in equipment maintenance, should such maintenance be required. They may also indicate that no maintenance is necessary.

[0041] However, since these RM maintenance reports are generally written by specialized technicians, they are also likely to contain acronyms, abbreviations, and highly specific vocabulary. They are also likely to contain spelling, grammar, and syntax errors, as well as information that is not useful for defining or characterizing the observed failure. Maintenance reports are also generally very susceptible to variations in format between operators, between airlines, and between aircraft types. For example, an acronym used for a helicopter engine may have a different meaning for an airplane engine.

[0042] As also illustrated in [Fig. 3], the method 10 includes a step 200 of obtaining, by means of a pre-trained machine learning process, a plurality of LABX labels corresponding to the maintenance report RM generated in step 100. This step can advantageously be implemented using a digital maintenance assistance system SN pre-trained in the classification of textual data DT from the maintenance reports RM. In this regard, maintenance reports RM are pre-integrated into the digital system SN in order to perform pre-training of the digital system SN. The more maintenance reports RM the digital system SN contains, the more reliable the training and the greater the possibility of precisely defining the LABX labels corresponding to the maintenance report RM.

[0043] More specifically, the invention falls within the framework of supervised learning. Thus, once the RM maintenance reports have been integrated into the SN digital system, the DT textual data are labeled, that is to say, LABX labels are assigned to the textual data (TD). This is referred to as TD-labeled textual data (LABX). The labels represent information within data characterizing aircraft equipment and equipment failures. They are therefore representative of ground truth, i.e., the actual situation on the ground. This preliminary labeling is done manually by a human, preferably someone knowledgeable about the subject matter covered in the maintenance report (RM) (e.g., a maintenance technician), prior to the maintenance of the aircraft equipment. Consequently, it is easier to assess the reliability of the LABX labels obtained during step 200 of the invention's process. It should be noted that several labels can be assigned to the same TD, whereas in the prior art there is generally only one label. This is important for the invention.

[0044] The following table presents an application of labeling to information useful for predictions made within the framework of the present invention.

[0045] Table 1 Identification of information relevant to maintenance support RM Engine Type Airline Type of Failure Cause of Failure Equipment to Repair / Replace Action Maintenance Lorem ipsum dolorista met, consectetur amet. Engine 1 Company 1 Failure A Cause B Equipment 11 Action A Temporary incident ut labore et dolore Engine 2 Company 1 Failure B Cause C Equipment 110 Action N

[0046] Useful information might include, for example, the aircraft engine type, the airline, the equipment to be replaced, and the maintenance action to be performed. LABX labels relating to the type of failure are "Failure A," "Failure B," etc., while LABX labels relating to maintenance actions to be performed are, among others, "Action A," "Action N," etc. For example, an action might be the repair of a component of this equipment. An action might also be the replacement of the equipment itself or a component of this equipment. Labeling thus makes it possible to identify and categorize the textual data. It should be noted that for the same textual data, several labels relating to a class typology TCj can be assigned. For example, a textual data item can have several labels relating to the type of failure (e.g., failure A and failure B), in addition to labels relating to other useful information.

[0047] As illustrated in [Fig. 1], the digital system SN comprises a BA training database including a B Ai link database whose data comes from maintenance reports RM. These maintenance reports are integrated into this BAi link database. They are written by maintenance technicians or operators involved in the maintenance of aircraft equipment, based on their observations. The BAi link database, or BAi link database, used in the context of the invention therefore does not contain textual data as structured and standardized as the data in databases familiar to those skilled in the art. Furthermore, as follows from the above, the BAi link database is, in principle, more specialized than known databases in the technical field of aircraft maintenance, since it is enriched by the maintenance technicians themselves.

[0048] Still with reference to [Fig.1], the B Ai link database includes BATRAin training data and BACal calibration data, the use of which in the context of the maintenance method 10 according to the invention will be better described later.

[0049] Let us now return to [Fig. 3]. The process 10 includes a step 300 of determining, using the B Ai linking database, the maintenance actions associated with the LABX labels. As previously stated, each LABX label is associated with useful information, so it is possible to associate the maintenance action to be performed with the equipment in question.

[0050] According to the invention, the pre-trained machine learning process includes a refinement step 220 in classification, from the BATRA1N training data, of at least one MOD, M0Dk model of a multilayer neural network for labeled language processing.

[0051] The MOD, M0Dk neural network model(s) is / are pre-trained for natural language processing. Within the scope of the invention, a BERT, GPT, or other Large Language Model (LLM) neural network model can advantageously be used. Indeed, although various types of models exist in the prior art, not all are configured to process natural language with such high automatic language processing performance as BERT, GPT, or other LLM language models. As is known, BERT, GPT, or other LLM language models can model language and predict sentences because they are pre-trained to learn words and sentences, as well as the semantic relations to which these words and sentences are connected. Preferably, the MOD, M0Dk model(s) is / are chosen from among the BERT, SciBERT, and SafeAeroBERT models.

[0052] In this respect, the digital system SN takes as input the BATRA1N training data and performs the fine-tuning of the MOD, MODk neural network model(s) based on this BATRA1N training data. Fine-tuning, also called specialization of a neural network model, is an operation by which minor changes are made to the internal parameters of said neural network model. It is thus possible to specialize the neural network model for a specific task without having to fully train the model. In step 220, the MOD, MODk multilayer neural network model(s) is fine-tuned to classify the DT(LABX)-labeled textual data, in particular from the BATRA1N training database, according to the LABX labels.

[0053] In this regard, the TQ class typologies represent information useful to predict by the MOD, M0Dk neural network model(s). Using the example from Table 1 above, the TC1 class typology is the "type of failure" of the aircraft engine on which the presumed faulty equipment was found, the TC2 class typology is the "Cause of the failure," the TC3 class typology is the "equipment to be repaired or replaced," and the TC4 class typology is the "maintenance action" to be performed. Of course, other types of information may be useful to the maintenance technician, and the aforementioned classes are neither exhaustive nor limiting within the scope of the present invention. For example, class typologies could also be provided relating to the procedures implemented during the diagnosis of the equipment failure, as well as to the operational impact (e.g.flight delay or cancellation) due to equipment failure.

[0054] Refining each MOD, M0Dk neural network model using classification 220 allows the MOD, M0Dk model(s) to learn the link between the BATRA1N training data and the LABX labels, and by association, to identify the TQ class typologies associated with these labels—that is, the information useful for predicting failure or the absence of failure and taking the necessary actions. This learning is performed in the iterative process that allows specialization of the neural network model(s) through multi-label refinement of the neural network(s). Each multilayer neural network can then advantageously include an outer layer with as many neurons as there are LABX labels. Each neuron in the outer layer is then capable of learning the link between the BATRA1N training data and the LABX labels to which it is associated.

[0055] In practice, several outer layers of the neural network could be used for refining the MOD, M0Dk model(s). In this case, only the outermost outer layer has as many neurons as there are LABX labels; for the other outer layers, this number of neurons can be variable and is part of the model's meta-parameters. Regardless of the number of outer layers used for refining the MOD, M0Dk model(s), the other layers of the neural network are fixed and do not participate in the refinement. In one embodiment of the invention, the other layers may not be fixed and therefore also participate in the refinement. In one embodiment as in the others, the refinement step 220 can advantageously be repeated for the MOD, M0Dk neural network model(s) according to one or more stopping criteria.

[0056] The stopping of the iterative process can be defined according to one or more stopping criteria chosen from among which: i. reaching a maximum number of iterations, ii. achieving a maximum training time (iteration duration), according to business guidelines.

[0057] That being said, other criteria for stopping the iterative process can be considered when the link base B Ai includes, in addition to the BATRA1N training data and the BACal calibration data, BAVal validation data. These criteria for stopping the iterative process are: i. the observation of overfitting based on the BAval validation database according to business criteria, ii. the observation of a stagnation on the BAVal validation database after a predefined number of iterations according to business criteria.

[0058] Overfitting occurs when performance continues to improve with training data but begins to decline significantly with validation data. In this case, the model(s) has / have learned the training data too well and loses its / their ability to generalize to data not seen during training. Validation data (BAVal) is generally used to stop training before the neural network loses its generalization capacity.

[0059] Stagnation refers to a variability in performance on the BAVal validation data less than epsilon, epsilon being a number fixed according to business criteria, for example on the order of 103. In this regard, the second MVal validation module advantageously allows the iterative process to be stopped at the best iteration, that is to say the iteration for which the variability in performance on the BAVal validation data is less than epsilon.

[0060] Each of the four aforementioned stopping criteria can be used alone or in combination. If stopping criteria i and ii are used alone or only in combination with each other, neither the BAVal validation data nor the second Mval validation module is required. However, if criteria iii and iv are used alone or in combination with one of the other two criteria, the iterative process can only be implemented if the linking base B Ai includes B Aval validation data. Thus, the B Aval validation data serves only to terminate the iterative process and is therefore optional. Other stopping criteria for the iterative process, defined according to business criteria, can be considered by those skilled in the art when implementing the present invention.

[0061] The refined MOD, M0Dk neural network model(s) thus produce, at the output of the iterative process, an SC score for each LABX label. The SC score associated with a LABX label quantifies the level of confidence that the MOD, M0Dk neural network model(s) place in the prediction of the LABX label.

[0062] According to the invention, the pre-trained machine learning process also includes a calibration step 240 of said at least one MOD, M0Dk neural network model so that it predicts all LABX labels above an acceptance threshold Si corresponding to a reference score for which the percentage of labels predicted in accordance with the LABX labels from the BAcal calibration data is maximized. In other words, the acceptance threshold Si is the reference score for which the percentage of predicted LABX labels that correspond to the predetermined LABX labels (i.e., the annotations), and therefore the percentage of LABX labels that correspond to the ground truth, is maximized. This calibration step 240 is implemented after the refinement step 220, as illustrated in [Fig. 3]. This calibration step thus makes it possible to take into account the fact that a textual data DT can have several labels.In this case, if for example a text data DT has two predetermined LABX labels, a threshold that only predicts these two labels will give a better percentage of labels predicted according to ground truth than a threshold that only predicts one of these two labels or a score that predicts these 2 labels and an incorrect label.

[0063] At this stage, it should be noted that the acceptance threshold Si can only be known if the MOD model, M0Dk, has been previously refined 220, i.e., specialized, to classify the BATRAin training data, optionally the BAVal validation data, according to the LABX labels. Indeed, as indicated, the acceptance threshold Si corresponds to the reference score for which the percentage of LABX labels predicted in accordance with the LABX labels, i.e., with ground truth, from the BACal calibration data is maximal, which implies testing each MOD, MODk model(s) on the BACal calibration data.

[0064] Implementing an acceptance threshold Si ensures that the process of the invention produces as output not the LABX label with the highest score, but all LABX labels with an SC score greater than the acceptance threshold Si. Indeed, a LABX label is predicted if its SQ score is greater than or equal to the threshold Si. This makes it possible to determine whether a failure actually affects the equipment in question, and to diagnose a failure of one or more pieces of equipment more quickly.

[0065] From then on, the operator can visually or by any other means inspect the equipment and possibly add supplementary observations to those made in the original RM maintenance report. They can also replace parts after performing associated tests, and then add further observations to those made in the original RM maintenance report. In both cases, the textual data DT entered into the SN digital system allows the operator to prioritize the inspection of any equipment they have not yet inspected, and to identify the components to be replaced and / or tested.

[0066] It should be noted that prior art neural network models output a score associated with each label, and a person skilled in the art usually uses the label with the highest score as the label predicted by the model. Therefore, there is no possibility of rejecting all labels, and thus no possibility of diagnosing the absence of a fault with regard to the application more specifically targeted within the scope of the present invention. Thus, the maintenance method according to the invention limits, or even prevents, sending equipment for maintenance when it does not present any fault, and therefore prevents the occurrence of "No Fault Found (NFF)" events. Indeed, sending an excessive number of pieces of equipment for maintenance results in additional costs for the aircraft operator.

[0067] When the SC scores of all LABX labels are strictly below the acceptance threshold Sj, the digital SN system reaches the following conclusion: "there is no fault or the described fault is not referenced" and offers an operator the following choices: - Do nothing. - Modify the text data DT: the operator can indeed modify the text data DT if he thinks that the system could not conclude because the text data DT was not explicit enough. - Create a new LABX label associated with this text data DT: the operator can then modify the training set by adding the text data DT and the associated LABX label. In this case, this text data DT can only be permanently included in the training set B Ai if the MOD, MODk model(s) is modified to include an additional output neuron corresponding to this new label. - Manually associate an existing label: the operator can therefore modify the training database by adding the text data DT and the associated label.

[0068] It can be advantageous to implement active learning. In this regard, the training database B Ai can, for example, be supplemented by targeting RM maintenance reports that are underrepresented or not represented at all. If the numerical system SN is unable to classify the labeled textual data DT(LABX) from these RM maintenance reports, this textual data is therefore poorly known or unknown and can be manually labeled to enrich the training database. If one or more underrepresented LABX labels have been removed from the B Ai link database and a new RM maintenance report's score exceeds the threshold S; Upon acceptance, a new neural network model incorporating this / these label(s) can be refined to the task of classifying texts from the labeled DT(LAB x) textual data of this / these new report(s). Thus, performance is continuously improved.

[0069] According to a particularly advantageous embodiment, the BAi linking database is divided into N SPLIT data partitions, i a natural number with i > 2, the MOD, M0Dk neural network model being refined for each of the SPLIT data partitions so that a score or an average SQ score is produced for each LABX label in each SPLIT partition. By way of example, the number of SPLIT partitions could be 10 for a BAi linking database comprising between 2,000 and 3,000 data points.

[0070] Each SPLIT; comprises all the textual data DT(LABX) from the linking database. Within each SPLIT;, the data from the linking database is exclusively divided into BATRA1N training data, B Aval validation data (optional), and BACal calibration data. Thus, within a SPLIT;, each data point is used either for training, validation, or calibration. These roles may differ from one SPLIT; to another. For example, in the SPLIT;, 75% of the data may be BATRA1N training data, 15% B Aval validation data, and 10% BACal calibration data. It is preferable (but not mandatory) to maintain these same percentage distributions in all SPLIT;s.

[0071] When the link base BAi does not contain validation data BAVal, each data partition SPLIT; includes all the data from the link base BA;, said data from the link base B Ai being partitioned into training data BATRAiNi and calibration data BAcau so that each data from the link base B Ai is either a training data BATRAiNi or a calibration data BAcalî- This example of an embodiment is illustrated in [Fig.2a].

[0072] When the BAi link base contains the BAVal validation data, in each SPLIT data partition;, the data of the first group of BAi data are partitioned into BATRAiNi training data, BAcalî calibration data and BAVALi validation data. This example of an embodiment is illustrated in [Fig.2b].

[0073] This performs cross-validation, which allows for optimal use of DT(LABX)-labeled textual data, particularly when the BA database contains a small number of data points (e.g., fewer than 10,000). Furthermore, this allows for better data utilization and therefore improved generalization performance (i.e., end-use performance) without having to test all possible data combinations.

[0074] It is preferable that each data point in the BA; link database be classified at least once (i.e., at least in a SPLIT) as BATRAin training data in order to obtain more accurate MOD, M0Dk models after refinement. Furthermore, it is preferable to maintain identical label proportions for each data type to perform cross-validation. For example, if the BATRAin training data includes 5% data with label A, it is preferable to have approximately 5% of the BAVA validation data with label A and 5% of the BACAL calibration data with label A.

[0075] According to an exemplary embodiment, the link database BA contains 1000 data points and there are N=3 splits. The splits could be organized as follows: - Split 1: training data 1 to 750, validation data 751 to 900, calibration data 901 to 1000, - Split 2: training data 1 to 650 and 751 to 850, validation data 851 to 1000, calibration data 651 to 750, - split 3: training data 1 to 550 and 801 to 1000, validation data 651 to 800, calibration data 551 to 650.

[0076] During the refinement step 220, a plurality of M0Dk models, k a natural number, of multilayer neural networks, pre-trained in natural language processing, can be refined in classification. The M0Dk neural network model(s) is then refined for each of the SPLIT partitions; of data so that a score or an average SCj score is produced for each LABX label in each SPLIT partition;.

[0077] Once the M0Dk models have been refined, they can then be tested with the validation data B Aval-. If necessary, the MODi neural network model can be tested using the validation data BAvalu. The M0D2 neural network model can be tested using the validation data BAVal2, and so on for the N partitions. As long as the selected stopping criterion(a) is / are not met, further iterations of the iterative process are performed. Testing with the validation data BAVal is optional.

[0078] Still in relation to this variant of the embodiment, during calibration step 240, the SCj score of each LABX label is compared to the acceptance threshold S of the SPLIT partition considered, or to an average of the acceptance thresholds Si determined for each SPLIT partition, called the average acceptance threshold Smoy, such that: - when the SCj score is strictly less than the S threshold for acceptance of the SPLIT partition considered, or the Smoy average acceptance threshold, the LABX label associated with this SCj score is rejected for this text data. - when the SCj score is greater than or equal to the S; acceptance threshold of the SPLIT partition; considered or the Smoy average acceptance threshold, the LABX label associated with this SCj score is predicted for this text data.

[0079] Figures 5 and 6 illustrate respectively the optimization of the acceptance threshold S; and the average acceptance threshold Smoy.

[0080] According to a first particular implementation, a distinct M0Dk neural network model is refined for each of the SPLIT; partitions of data, the number of M0Dk models being equal to the number of SPLIT; partitions (k = i). However, it should be noted that if there are fewer models to refine than SPLIT; partitions, the SPLIT; partitions to be used are chosen randomly or according to a business criterion as described in the following section. Furthermore, it is preferable that a SPLIT; partition be used for refining only one model.

[0081] According to a second particular implementation, the linking basis BA; is divided into N partitions SPLIT;, SPLIT2, ..., SPLITN. However, refinement 220 is implemented for a plurality of M0Dk models, k a natural number with k < i and k = [1.. .P], of multilayer neural networks, pre-trained for natural language processing. In this case, P SPLIT partitions are chosen from the N, either randomly or according to a business criterion, with an M0Dk neural network model being refined for each of the SPLIT; partitions of data from among the selected partitions (refinement of one model per SPLIT; partition). Thus, in the present implementation, although several MOD;, M0D2, ..., M0DP neural network models are refined in the different SPLIT partitions; of data, the number of these models MODi, MOD2, MODp is less than the number N of SPLIT partitions;.

[0082] It can be useful to train several neural networks, some predicting very general labels, others increasingly specific labels. In this case, if the operator provides a rather vague observation in the initial RM maintenance report, the neural network(s) predicting general labels may be able to make a prediction (label score above the acceptance threshold of that neural network), but the specific neural networks are likely to have scores below their respective acceptance thresholds. They will therefore return nothing. The labels ultimately returned, i.e., ultimately predicted, to the user are thus likely to be very general. The operator can then iteratively modify or supplement their observations by taking into account the responses of the neural network(s) in order to obtain more specific response(s).

[0083] Once the calibration step 240 has been implemented, a M0Dk model fusion step 260 for which a percentage of labels predicted in accordance with the LABX labels from the BACal calibration data is maximized can be performed.

[0084] With reference to Figures 3 and 4, the process 10 may advantageously include a pretreatment step 120 of the BAb binding base. The pretreatment step 120 may include the following substeps: - an optional sub-step 121, consisting of generating or enriching a dictionary of acronyms and abbreviations, - an optional sub-step 122, consisting of replacing the acronyms and abbreviations present in the DT textual data with their full meaning, - an optional sub-step 123, consisting of generating synonyms for at least part of the DT textual data composing the BAi linking database from a synonym dictionary, - an optional sub-step 124, consisting of generating translations of at least a part of said DT textual data from at least one translation dictionary, - an optional sub-step 125, consisting of performing an automatic spelling correction of the data in the BAp link database

[0085] In this respect, the SN digital system can be configured to generate or enrich a dictionary of acronyms and abbreviations, and to replace the acronyms and abbreviations present in the DT(LABX) labeled textual data with their full meaning. Such preprocessing allows the MOD, M0Dk model(s) to neural network(s) to better understand the context of certain textual data labeled DT(LABX) and to better understand certain specific terms.

[0086] Similarly, the SN digital system can be configured to generate synonyms for the DT(LABX)-labeled text data comprising the BAi link database from a synonym dictionary and / or to generate translations of said DT(LABX)-labeled text data from at least one translation dictionary. This makes it possible to increase the amount of text data available for refining the MOD, M0Dk neural network model(s). Indeed, when a large amount of annotated data is available, it is easier to train or specialize a neural network model. However, in practice, in specific applications, there is very little annotated text data. Increasing the amount of text data, or more simply, increasing the data, is highly advantageous.

[0087] Synonym generation consists of replacing certain words with their synonyms according to a predetermined percentage, from at least one public or specialized synonym dictionary, preferably in digital format.

[0088] In the case of translation generation, it is possible to use translators capable of iteratively translating words or phrases from a language L1 used in the RM maintenance reports to another language L2, which is itself translated into yet another language L3, and so on. The final translation must be from language LN to language LL. This has the effect of preserving the meaning of the original text while introducing minor variations. These variations are "minor" because they allow new textual data DT to be introduced into the B Ai link database without altering the nature and interpretation of this textual data. In other words, the textual data DT provided by the translation and the synonyms are data that could have appeared in the RM maintenance report provided by the maintenance operator.

[0089] Automatic spelling correction of DT(LABX) labeled text data contributes to the normalization of text data and also improves the understanding of text data by the MOD model or each MOD model, M0Dk.

[0090] Furthermore, as illustrated in [Fig. 1], in addition to the BAi link database, the BA training database may also include an evaluation database BA2, which allows the performance of the pre-trained and refined model to be assessed on data different from that used for training. In this respect, the BA2 evaluation database may include maintenance data on which the performance of the model(s) is / are evaluated.

Claims

Demands

1. A method (10) for maintaining aeronautical equipment comprising the following steps: - developing (100) a maintenance report (MR) from an inspection of the equipment, - obtaining (200) by means of a machine learning process pre-trained with a digital maintenance assistance system, a plurality of labels (LABX) corresponding to the maintenance report (MR) developed in step (100), - determining (300), using a linking database (BAi) comprising training (BATRAin) and calibration (B AcaiA) data, the maintenance actions to be performed that are associated with the labels, - performing the maintenance actions associated with the labels (LABx), characterized in that the machine learning process pre-trained with the digital maintenance assistance system includes: - refining (220) in classification, from the training data (BATRAin), at least one model (MOD,M0Dk) of multilayer neural network for labeled language processing and, - to calibrate (240) said at least one (MOD, M0Dk) neural network model so that it predicts all labels (LABX) above an acceptance threshold (Si) corresponding to a reference score for which a percentage of labels predicted in accordance with the labels (LABX) from the calibration data (BACal) is maximal.,

2. A maintenance method (10) according to claim 1, wherein the refinement step (220) is repeated for the neural network model(MOD, M0Dk) for a number of iterations and / or an iteration time.

3. A maintenance method (10) according to any one of claims 1 to 2, wherein the linking database (DB) is divided into N data partitions (SPLIT,), i a natural number with i > 2, the neural network model(s) (MOD, M0Dk) being fine-tuned for each of the data partitions (SPLIT,) such that that a score or an average score (SCj) is produced for each label (LABX) in each partition (SPLIT;).

4. Maintenance method (10) according to claim 3, wherein, in step (220), the refinement is performed for a plurality of models (M0Dk), k a natural number with k = i, of neural networks, a distinct neural network model (M0Dk) being refined for each of the data partitions (SPLIT;).

5. Method (10) of maintenance according to claim 3, wherein, in step (220), the refinement is carried out for a plurality of models (M0Dk), k a natural number with k < i and k = [1.. .P], of neural networks, a model (M0Dk) of neural network being refined for each of the partitions (SPLIT;) of data.

6. A maintenance method (10) according to any one of claims 4 to 5, comprising, after the calibration step (240), a model fusion step (260) for which a percentage of labels predicted in accordance with the labels (LABX) from the calibration data (BACal) is maximized.

7. A maintenance method (10) according to any one of claims 3 to 6, wherein each data partition (SPLIT i) comprises all the data in the link base (BAi), said data in the link base (BAi) being partitioned into training data (BATRAiNi) and calibration data (BACalÛ) such that each data in the link base (BA) is either training data (BATRAiNi) or calibration data (BACalÛ)*

8. A maintenance method (10) according to claim 7, wherein, in each data partition (SPLIT;), the data from the link base (BA;) are partitioned into training data (BATRA1Ni), calibration data (BACALi) and validation data (BAVALi), the validation data (BAVALi) being used to test the previously refined neural network model(s) (MOD, M0Dk).

9. Maintenance method (10) according to any one of the preceding claims, wherein the model or each model (MOD, M0Dk) is selected from the BERT, SciBERT and SafeAeroBERT, GPT models or their variants.

10. A maintenance method (10) according to any one of the preceding claims, further comprising a pre-processing step (120) of the linking database (DB) comprising the following sub-step(s): (121) generating or enriching a dictionary of acronyms and abbreviations, (122) replacing the acronyms and abbreviations present in the text data (TD) with their full meaning, (123) generating synonyms for at least a part of the text data (TD) composing the linking database (DB) from a synonym dictionary, (124) generating translations for at least a part of said text data (TD) from at least one translation dictionary, (125) performing automatic spell checking of the data in the linking database (DB).

Images