Aero-engine health management human-computer interaction method and system

Abstract

The application discloses an aero-engine health management man-machine interaction method and system, which comprises the following steps: acquiring related multi-modal data sources for generating an aero-engine health management knowledge graph; completing missing data in the multi-modal data sources by using a mean value interpolation method to obtain multi-modal preprocessed data; constructing a feature fusion model and inputting the multi-modal preprocessed data to obtain multi-modal data triple fusion features; extracting information of the multi-modal data fusion features by using a machine learning method to obtain multi-modal aero-engine health management knowledge and construct a multi-modal aero-engine health management knowledge graph; constructing a multi-modal man-machine interaction system, and realizing man-machine interaction of multi-modal data based on the multi-modal aero-engine health management knowledge graph, which can better display aero-engine health management information, thereby replacing the knowledge of experts in the field of aero-engine health management and realizing the intelligentization of the field of aero-engine health management.

Description

A Human-Computer Interaction Method and System for Aircraft Engine Health Management Technical Field

[0001] This invention relates to the field of robotics, and in particular to a human-computer interaction method and system for aircraft engine health management. Background Technology

[0002] Consulting engine health management experts or building relevant expert knowledge bases for aero-engine health management is not only costly but also time-consuming and labor-intensive. With the rapid development of artificial intelligence technology, aero-engine health management researchers have begun to focus on the application of intelligent technologies such as deep learning in engine fault diagnosis and prediction, and maintenance and repair decisions based on this. Besides revealing the correlation between known fault state indicators, maintenance and repair trigger indicators, and improving diagnostic reliability through information fusion and other methods, intelligent engine health management systems also have a series of more important tasks, such as searching for the most suitable maintenance and repair time points and fusing features or decisions with known faults and maintenance and repair indicators. The completion of all these tasks relies heavily on knowledge graph technology.

[0003] Existing methods for human-computer interaction in aero-engine health management rely too heavily on a single data source to construct a knowledge graph for aero-engine health management. This results in data with low reliability and inherent errors, leading to suboptimal human-computer interaction outcomes. Furthermore, scheduled maintenance is not only resource-intensive and inefficient, but also incurs persistently high maintenance costs.

[0004] The data sources used include structured, semi-structured, and unstructured data, which are used to construct the knowledge graph of aero-engine health management, thereby improving the credibility of the data. When using relevant multimodal data sources to generate the knowledge graph of aero-engine health management, data preprocessing algorithms are used to remove redundant data from the multimodal data sources and to fill in missing data, thereby improving the reliability of the data. Summary of the Invention

[0005] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0006] In view of the above-mentioned problems, the present invention is proposed.

[0007] Therefore, the technical problem solved by this invention is: how to create a health management knowledge graph using multimodal data sources to achieve human-computer interaction with multimodal data.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a human-computer interaction method for aircraft engine health management, comprising the following steps,

[0009] Obtain relevant multimodal data sources for generating a knowledge graph of aircraft engine health management;

[0010] The missing data in the multimodal data source is completed by using the mean interpolation method to obtain multimodal preprocessed data;

[0011] A feature fusion model is constructed and multimodal preprocessed data is input to obtain multimodal data triple fusion features;

[0012] Machine learning methods are used to extract information from the fusion features of multimodal data to obtain multimodal aero-engine health management knowledge and construct a multimodal aero-engine health management knowledge graph.

[0013] A multimodal human-computer interaction system is constructed, and human-computer interaction on multimodal data is realized based on the multimodal aero-engine health management knowledge graph.

[0014] As a preferred embodiment of the human-computer interaction method for aero-engine health management described in this invention, the relevant multimodal data sources used to generate the aero-engine health management knowledge graph include an engine knowledge information database, engine maintenance logs, and engine non-destructive testing and borescope information obtained from airborne sensors.

[0015] As a preferred embodiment of the human-computer interaction method for aircraft engine health management described in this invention, the feature fusion model sets three modalities of data: structured data, semi-structured data, and unstructured data. In order to ensure that each element of each modal input data has sufficient perception ability with its neighboring elements, the input data is passed through a one-dimensional convolutional layer.

[0016] As a preferred embodiment of the human-computer interaction method for aero-engine health management described in this invention, the calculation method of the one-dimensional convolutional layer is as follows:

[0017]

[0018] Where Conv1D represents a one-dimensional convolution operation. k represents input data from three modes: U, S, and W. {U,S,W} and T {U,S,W} d represents the kernel size and data sequence length for the U, S, and W modalities, respectively; d is the feature dimension of the data.

[0019] As a preferred embodiment of the human-computer interaction method for aero-engine health management described in this invention, the method involves: enabling each modality's data to carry time information, and extending the position embedding to the output of a one-dimensional convolutional layer.

[0020]

[0021] Among them, Z {U,V,W} This represents the computation result of adding positional embeddings to the output of a one-dimensional convolutional layer, PE(T). {U,S,W} ,d) represents the position embedding for each position index. Here, the position information of a data sequence of length T and feature dimension d is encoded using sin and cos functions. The frequencies of the sin and cos functions are determined by the feature index. The sequence X∈R is defined as follows: T×d Position embedding as a matrix

[0022]

[0023]

[0024] As a preferred embodiment of the human-computer interaction method for aero-engine health management described in this invention, the output of the position embedding is input to a self-attention layer, and the self-attention layer is used to obtain more effective features of each modality data. The calculation formula for self-attention is as follows:

[0025]

[0026] Where softmax represents the activation function, Y {U,S,W} Q represents the self-attention calculation results for the three modalities U, S, and W. {U,S,W} K {U,V,W} and V {U,S,W} These represent the query, key, and value of self-attention, respectively. This represents the scaling factor.

[0027] As a preferred embodiment of the human-computer interaction method for aero-engine health management described in this invention, the method comprises: extracting information from the fusion features of multimodal data triples using machine learning methods to obtain multimodal aero-engine health management knowledge; extracting feature names and health management trigger indication names from the fusion features of multimodal data triples as nodes in the knowledge graph; calculating the relationship strength between nodes based on the relationship strength and triple statistics contained in the fusion features of multimodal data triples, and using the obtained relationship strength between nodes as edges in the multimodal aero-engine health management knowledge graph; and inputting all the multimodal aero-engine health management knowledge mined based on machine learning methods into an expert system to construct the multimodal aero-engine health management knowledge graph.

[0028] In view of the above-mentioned problems, the invention provides the following technical solution: a human-machine interaction system for aircraft engine health management.

[0029] Therefore, the technical problem solved by this invention is to complete a knowledge graph of aero-engine health management by collecting, processing, extracting and fusing data through multiple modules, so as to realize human-computer interaction.

[0030] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a human-computer interaction system for aero-engine health management, comprising: a multimodal data source acquisition module, a data preprocessing module, a feature extraction module, a feature fusion module, an aero-engine health management knowledge acquisition module, an aero-engine health management knowledge graph construction module, and a human-computer interaction system construction module;

[0031] The multimodal data source acquisition module is used to acquire multimodal data sources for generating a multimodal aero-engine health management knowledge graph;

[0032] The data preprocessing module is used to preprocess the multimodal data source to obtain multimodal preprocessed data;

[0033] The feature extraction module is used to extract features from the obtained multimodal preprocessed data, and extract features of different modal data.

[0034] The feature fusion module is used to perform knowledge fusion on the features of different modal data to obtain multimodal data fusion features;

[0035] The aero-engine health management knowledge acquisition module is used to extract information from the multimodal data fusion features to obtain multimodal aero-engine health management knowledge.

[0036] The aero-engine health management knowledge graph construction module is used to construct the multimodal aero-engine health management knowledge graph based on the multimodal aero-engine health management knowledge.

[0037] The human-computer interaction system construction module is used to construct a multimodal human-computer interaction system and realize human-computer interaction on multimodal data based on the multimodal aero-engine health management knowledge graph.

[0038] A computer device includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

[0039] A computer-readable storage medium storing a computer program thereon, characterized in that, when executed by a processor, the computer program implements a human-computer interaction method for aircraft engine health management based on multi-source information fusion as described in any one of claims 1 to 9.

[0040] The beneficial effects of this invention are as follows: The data sources used in this invention include structured data, semi-structured data, and unstructured data. Compared with relying solely on a single data source to construct the knowledge graph of aircraft engine health management, the reliability of the data is improved. When using relevant multimodal data sources for generating the knowledge graph of aircraft engine health management, data preprocessing algorithms are used to remove redundant data from the multimodal data sources and to complete the missing data in the multimodal data sources, thereby improving the reliability of the data. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0042] Figure 1 is a flowchart illustrating a human-computer interaction method for aircraft engine health management according to an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of the feature fusion model structure in a human-computer interaction method for aircraft engine health management according to an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of the global filtering module structure in a human-computer interaction method for aircraft engine health management according to an embodiment of the present invention.

[0045] Figure 4 is a structural block diagram of a human-computer interaction system for aircraft engine health management provided in an embodiment of the present invention. Detailed Implementation

[0046] The above-mentioned objects, features, and advantages of the present invention will become more apparent and understandable. The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0047] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0048] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0049] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0050] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0051] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0052] Example 1

[0053] Referring to Figures 1, 2, and 3, which illustrate an embodiment of the present invention, a human-computer interaction method for aircraft engine health management is provided, comprising:

[0054] Step S1: Obtain relevant multimodal data sources for generating the knowledge graph of aircraft engine health management;

[0055] Step S2: Complete the missing data in the multimodal data source using the mean interpolation method to obtain multimodal preprocessed data;

[0056] Step S3: Construct a feature fusion model with multimodal global filtering capability, input the obtained multimodal preprocessed data into the fusion model, and obtain multimodal data triplet fusion features;

[0057] Step S4: Use machine learning methods to extract information from the fusion features of multimodal data to obtain multimodal aero-engine health management knowledge;

[0058] Step S5: Input all the multimodal aero-engine health management knowledge mined based on machine learning methods into the expert system to construct the multimodal aero-engine health management knowledge graph;

[0059] Step S6: Construct a multimodal human-computer interaction system and realize human-computer interaction on multimodal data based on the multimodal aero-engine health management knowledge graph.

[0060] The multimodal data sources used to generate the knowledge graph of aircraft engine health management include engine knowledge information databases, engine maintenance logs, and engine non-destructive testing and borescope information obtained from airborne sensors.

[0061] The missing data in the multimodal data source is completed by using the mean interpolation method to obtain multimodal preprocessed data.

[0062] A feature fusion model with multimodal global filtering capability is constructed. The obtained multimodal preprocessed data is input into the fusion model to obtain multimodal data triple fusion features. The specific implementation of the feature fusion model is as follows:

[0063] Given three modalities of data: structured data U, semi-structured data V, and unstructured data W, to ensure that each element of the input data for each modality has sufficient awareness of its neighboring elements, the input data is passed through a one-dimensional convolutional layer, which can be calculated using the following formula:

[0064]

[0065] Where Conv1D represents a one-dimensional convolution operation. k represents input data from three modes: U, S, and W. {U,S,W} and T {U,S,W} represents the kernel size and data sequence length for the U, S, and W modal data, respectively; d is the feature dimension of the data;

[0066] Then, to enable each modality's data to carry temporal information, the position embedding is extended to the output of the one-dimensional convolutional layer, which can be calculated using the following formula:

[0067]

[0068] Among them, Z {U,V,W} This represents the computation result of adding positional embeddings to the output of a one-dimensional convolutional layer, PE(T). {U,S,W} ,d) represents the position embedding for each position index. Here, the position information of a data sequence of length T and feature dimension d is encoded using sin and cos functions. The frequencies of the sin and cos functions are determined by the feature index. Define the sequence X∈R T×d The positions are embedded into a matrix, where:

[0069]

[0070]

[0071] in,

[0072] Next, the output of the location embedding is fed into a self-attention layer. The self-attention layer is used to obtain more effective features for each modality of data. The self-attention can be calculated by the following formula:

[0073]

[0074] Among them, Y {U,S,W} Q represents the self-attention calculation results for the three modalities U, S, and W. {U,S,W} K {U,V,W} and V {U,S,W} These represent the query, key, and value of self-attention, respectively. Indicates the scaling factor;

[0075] In order to obtain global features among the three modalities U, S and W and control the computational complexity of the model, the output features of self-attention are further input into the global filtering module, which is a residual structure composed of a global filtering layer and a feature feedforward layer.

[0076] First, the features of the input global filter layer are normalized. The normalization operation can be calculated using the following formula:

[0077]

[0078] Among them, F LN This represents the layer normalization operation; ε is a very small number, here we take ε = 10. -5 α and β are the scaling factor and bias term for learning, respectively; μ and σ 2 These are the channel mean and variance after batch normalization.

[0079] Next, a two-dimensional fast Fourier transform operation is performed on the layer-normalized output features. The two-dimensional fast Fourier transform operation can be calculated by the following formula:

[0080] D = F FFN (H)(1.7);

[0081] Among them, F FFN This represents a two-dimensional Fast Fourier Transform operation;

[0082] Then, by globally filtering D by multiplying the output of the two-dimensional fast Fourier transform by a learnable filter Z, it can be calculated by the following formula:

[0083] O=Z⊙D(1.8)

[0084] Where ⊙ represents a pixel-by-pixel multiplication operation;

[0085] Then, a two-dimensional inverse fast Fourier transform (IFFT) operation is used to transform O from the frequency domain back to the spatial domain. The output I of the two-dimensional IFFT operation can be calculated by the following formula:

[0086] I = F IFFN (O)(1.9)

[0087] Among them, F IFFN This represents the two-dimensional inverse fast Fourier transform operation;

[0088] The output features of the two-dimensional inverse fast Fourier transform operation are further input into the feature feedforward layer, which consists of layer normalization and multilayer perceptron layers. First, the layer normalization operation can be calculated by the following formula:

[0089] A L =F LN (I L (1.10)

[0090] Among them, F LN Representation layer normalization operation; A L and I LThese represent the Lth feature map channels of A and I, respectively.

[0091] Then, the normalized output features of the layer are input into the multilayer perceptron layer, which can be calculated by the following formula:

[0092] B = F MLP (A)(1.11)

[0093] Among them, F MLP This represents a multilayer perceptron;

[0094] Machine learning methods are used to extract information from the triple fusion features of multimodal data to obtain multimodal aero-engine health management knowledge, including:

[0095] First, feature names and health management trigger indicator names are extracted from the multimodal data triple fusion features and used as nodes in the knowledge graph.

[0096] Then, based on the relation strength and triple statistics contained in the multimodal data triple fusion features, the relation strength between nodes is calculated, and the obtained relation strength between nodes is used as the edge of the multimodal aero-engine health management knowledge graph.

[0097] Input all the multimodal aero-engine health management knowledge mined based on machine learning methods into the expert system to construct the multimodal aero-engine health management knowledge graph.

[0098] Input all the multimodal aero-engine health management knowledge mined based on machine learning methods into the expert system to construct the multimodal aero-engine health management knowledge graph.

[0099] Example 2

[0100] Figure 2 is a structural block diagram of a human-computer interaction system 200 for aircraft engine health management according to an embodiment of the present invention. Referring to Figure 2, the system 200 includes a multimodal data source acquisition module 201, used to acquire multimodal data sources for generating a multimodal aircraft engine health management knowledge graph;

[0101] Data preprocessing module 202 is used to preprocess multimodal data sources to obtain multimodal preprocessed data;

[0102] Feature extraction module 203 is used to extract features from the obtained multimodal preprocessed data and extract features of different modal data;

[0103] The feature fusion module 204 is used to perform knowledge fusion on the features of different modal data to obtain multimodal data fusion features;

[0104] The aircraft engine health management knowledge acquisition module 205 is used to extract information from the multimodal data fusion features to obtain multimodal aircraft engine health management knowledge.

[0105] The aero-engine health management knowledge graph construction module 206 is used to construct the multimodal aero-engine health management knowledge graph based on the multimodal aero-engine health management knowledge.

[0106] The human-computer interaction system construction module 207 is used to construct a multimodal human-computer interaction system and realize human-computer interaction on multimodal data based on the multimodal aero-engine health management knowledge graph.

[0107] Example 3

[0108] As shown in Figure 2, this embodiment relates to a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the human-computer interaction method for aero-engine health management based on multi-source information fusion provided by the embodiment of the present invention as described above.

[0109] The memory can be, for example, hard disk drive memory, non-volatile memory (such as flash memory or other electronically programmable erasure-restricted memory used to form solid-state drives), volatile memory (such as static or dynamic random access memory), etc., and this application embodiment is not limited to these. Specifically, the memory can be an internal storage unit of the electronic device, such as the hard disk or RAM of the electronic device. The memory can also be an external storage device of the electronic device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device. Furthermore, the memory can include both internal storage units and external storage devices of the electronic device. The memory is used to store computer programs and other programs and data required by the terminal. The memory can also be used to temporarily store data that has been output or will be output.

[0110] Furthermore, this embodiment of the invention also provides a computer-readable storage medium, which may be disposed in the verification terminal of the above embodiments. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the human-computer interaction method for aircraft engine health management described in the embodiment shown in FIG2. Furthermore, the computer-readable storage medium may also be a USB flash drive, a portable hard drive, a read-only memory (ROM), RAM, a magnetic disk, or an optical disk, or any other medium capable of storing program code.

[0111] Example 4

[0112] In this embodiment, a specific application experiment of the method of the present invention was conducted. Under the same pre-set experimental environment, three sets of experiments were carried out on the existing conventional method and the method of this embodiment, respectively. The specific experimental results are shown in Tables 1 and 2.

[0113] Table 1 Comparison of Certification Time

[0114] Comparison of detection time: Experiment 1, Experiment 2, Experiment 3; Traditional method: 10.23s, 11.27s, 10.85s; 20.12s, 21.32s, 20.56s surface

[0115] Table 2 Comparison of Certification Accuracy Rates

[0116] Comparison of detection accuracy: Experiment 1, Experiment 2, Experiment 3: Traditional method 97.78%, 98.2%, 98.52%; Detection accuracy of traditional method 86.28%, 83.64%, 87.21%. surface

[0117] The comparative experiments above confirm that the method provided by this invention significantly improves the detection speed, resulting in a marked increase in authentication speed compared to existing technologies. This reduces detection time and ensures authentication results far exceeding those of traditional technologies, while also lowering the error rate.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A human-computer interaction method for aircraft engine health management, characterized in that, include: Obtain relevant multimodal data sources for generating a knowledge graph of aircraft engine health management; The missing data in the multimodal data source is completed by using the mean interpolation method to obtain multimodal preprocessed data; A feature fusion model is constructed and multimodal preprocessed data is input to obtain multimodal data triple fusion features; Machine learning methods are used to extract information from the fusion features of multimodal data to obtain multimodal aero-engine health management knowledge, and a multimodal aero-engine health management knowledge graph is constructed. A multimodal human-computer interaction system is built, and human-computer interaction with multimodal data is realized based on the multimodal aero-engine health management knowledge graph. The feature fusion model sets three data modalities: structured data, semi-structured data, and unstructured data. To ensure that each element of each modal input data has sufficient perception of its neighboring elements, the input data is passed through a one-dimensional convolutional layer. The calculation method of the one-dimensional convolutional layer is as follows: ,in, This represents a one-dimensional convolution operation. Indicates from 、 and Input data for three modalities, and They represent 、 and Kernel size and data sequence length for three modalities; It is the feature dimension of the data; the calculation method of the one-dimensional convolutional layer is as follows: ,in, This represents a one-dimensional convolution operation. Indicates from 、 and Input data for three modalities, and They represent 、 and Kernel size and data sequence length for three modalities; It is a feature dimension of the data; it enables each modality's data to carry temporal information, extending the positional embedding to the output of the one-dimensional convolutional layer. ,in, This represents the computation result of adding positional embeddings to the output of a one-dimensional convolutional layer. This represents the position embedding for each position index, where the sin and cos functions are used to represent the length of... Feature dimension is The positional information of the data sequence is positionally encoded, and the frequencies of the sine and cosine functions are determined by the feature index; the sequence is defined. The positions are embedded into a matrix; ,in, The output of the position embedding is input into the self-attention layer, which extracts more effective features for each modality of data. The formula for calculating self-attention is: Where softmax represents the activation function. express 、 and Self-attention calculation results for three modalities of data. 、 and These represent the query, key, and value of self-attention, respectively. This represents the scaling factor.

2. The human-computer interaction method for aircraft engine health management as described in claim 1, characterized in that: The relevant multimodal data sources used to generate the knowledge graph of aero-engine health management include engine knowledge information databases, engine maintenance logs, and engine non-destructive testing and borescope information obtained from airborne sensors.

3. The human-computer interaction method for aircraft engine health management as described in claim 2, characterized in that: Machine learning methods are used to extract information from the triple fusion features of multimodal data to obtain multimodal aero-engine health management knowledge. Feature names and health management trigger indication names are extracted from the triple fusion features of multimodal data as nodes in the knowledge graph. Then, based on the relation strength and triple statistics contained in the multimodal data triple fusion features, the relation strength between nodes is calculated, and the obtained relation strength between nodes is used as the edge of the multimodal aero-engine health management knowledge graph. Input all the multimodal aero-engine health management knowledge mined based on machine learning methods into the expert system to construct the multimodal aero-engine health management knowledge graph.

4. A human-computer interaction system for aircraft engine health management, employing the human-computer interaction method for aircraft engine health management as described in any one of claims 1 to 3, characterized in that, include: The system comprises a multimodal data source acquisition module (201), a data preprocessing module (202), a feature extraction module (203), a feature fusion module (204), an aero-engine health management knowledge acquisition module (205), an aero-engine health management knowledge graph construction module (206), and a human-computer interaction system construction module (207). The multimodal data source acquisition module (201) is used to acquire multimodal data sources for generating a multimodal aero-engine health management knowledge graph. The data preprocessing module (202) is used to preprocess the multimodal data sources to obtain multimodal preprocessed data. The feature extraction module (203) is used to extract features from the obtained multimodal preprocessed data and extract features of different modal data. The feature fusion module (204) is used to perform knowledge fusion on the features of different modal data to obtain multimodal data fusion features; the aero-engine health management knowledge acquisition module (205) is used to extract information from the multimodal data fusion features to obtain multimodal aero-engine health management knowledge; the aero-engine health management knowledge graph construction module (206) is used to construct the multimodal aero-engine health management knowledge graph based on the multimodal aero-engine health management knowledge. The human-computer interaction system construction module (207) is used to construct a multimodal human-computer interaction system and realize human-computer interaction on multimodal data based on the multimodal aero-engine health management knowledge graph.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a human-computer interaction method for aircraft engine health management as described in any one of claims 1 to 3.

Images