Aviation maintenance experience knowledge management system and method based on multi-source data processing

Abstract

The invention relates to the technical field of aviation maintenance data management, in particular to an aviation maintenance experience knowledge management system and method based on multi-source data processing, and the system comprises a data cleaning and preprocessing module, a storage module, a document analysis module and an index generation module. The data cleaning and preprocessing module is used for performing data cleaning, entity extraction, data verification and vectorization processing on input original multi-source data; the storage module is used for storing the compliant aviation maintenance experience data subjected to data verification and PDF (Portable Document Format) document data of various formed texts; the document analysis module analyzes a document and stores the document into a vector database; the index generation module generates an index file by recording the page number and content of each page of PDF.

Description

Technical Field

[0001] The present invention relates to the technical field of aviation maintenance data management, and in particular to an aviation maintenance experience knowledge management system and method based on multi-source data processing. Background Art

[0002] Aviation maintenance refers to a series of activities to maintain, repair and inspect aircraft (including airplanes, helicopters, drones, etc.) and their related equipment to ensure their safe and reliable operation.

[0003] The complexity of aircraft and the harsh flight environment require that every component be in excellent working condition. Any minor malfunction can lead to serious consequences during flight. With the continuous advancement of aviation technology, new aircraft are constantly emerging, and their structures and systems are becoming increasingly complex, placing higher demands on the technical proficiency of maintenance personnel and maintenance equipment. Maintenance personnel need to continuously learn and master new maintenance techniques and knowledge to adapt to the demands of technological development.

[0004] The following technical pain points exist in the current aviation maintenance field:

[0005] 1. Maintenance history data is scattered and not in a standardized format, making it difficult to use effectively;

[0006] 2. Most reference documents are in unstructured PDF format, resulting in low retrieval efficiency;

[0007] 3. Traditional retrieval methods cannot effectively combine semantic similarity and timeliness;

[0008] 4. Lack of a mechanism to intelligently link maintenance experience with reference manuals. Summary of the Invention

[0009] (1) Technical issues

[0010] The present invention aims to at least solve the problem of intelligent management of aviation maintenance data existing in the prior art.

[0011] (2) Technical content

[0012] This solution provides an aviation maintenance experience knowledge management system based on multi-source data processing, including data cleaning and pre-processing modules, storage modules, document parsing modules, and index generation modules;

[0013] The data cleaning and preprocessing module is used to perform data cleaning, entity extraction, data validation and vectorization processing on the input original multi-source data;

[0014] The document parsing module creates PDF parsing rules and parses PDF content; stores the parsing rules of the same type of PDF into the vector database of the storage module;

[0015] The storage module is used to store aviation maintenance experience data that has passed data validation and compliance, as well as various PDF document data;

[0016] The index generation module generates an index file by recording the page number and content of each PDF page.

[0017] Optimal technical solution 1: also includes an intelligent retrieval engine, which uses semantic similarity calculation values, time-weighted score calculation values ​​for weighted merging and normalization processing to obtain result sorting, and then intercepts the first N data and outputs the processing method in the experience data after arranging in descending order.

[0018] Preferred technical solution 2: The data cleaning process includes null value filtering, ATA chapter parsing, and type mapping.

[0019] Preferred technical solution three: The entity extraction uses large language model technology to extract key entity information from the text, and the extraction objects include fault content and processing content; the entity extraction process is guided by constructing a prompt word template.

[0020] Preferred technical solution four: The data validation includes: checking the integrity of the extracted entity data; identifying and marking missing or formatted entity items; correcting data that does not meet the specifications using text matching rules; and re-extracting missing key entities.

[0021] Preferred technical solution five: The vectorization processing uses Langchain's text embedding model to convert the fault description into a vector and stores it in the vectorized database in the storage module; other data is used as metadata and stored in the vector database according to custom schema rules.

[0022] Preferred technical solution six: The data cleaning and preprocessing module also includes data merging and comparison, and replacement parts data processing.

[0023] Preferred technical solution seven: the document parsing module also includes a program manual parser to parse the program manual.

[0024] This solution also discloses an aviation maintenance experience knowledge management method based on multi-source data processing, including the following steps:

[0025] Step 1: Empirical data cleaning and preprocessing

[0026] The raw data is merged and output after being filtered for null values, field normalized, ATA chapter parsed, type mapped, entity extracted, and data validated;

[0027] Step 2: Document processing

[0028] Parse, extract, process and generate indexes for PDF document data;

[0029] Step 3: Data storage

[0030] Store the vectorized experience data and document data;

[0031] Step 4: Retrieve call

[0032] Use the intelligent retrieval engine to call the data in the vector database and display it on the knowledge application interface.

[0033] (3) Technical effects

[0034] The above structure enables this solution to have the following beneficial effects:

[0035] 1. The utilization rate of maintenance data increased by more than 60%;

[0036] 2. Knowledge retrieval accuracy increased by 45%;

[0037] 3. Average troubleshooting time is shortened by 30%;

[0038] 4. Realize the automated inheritance of maintenance experience;

[0039] 5. Significantly reduce the incidence of human errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0041] Figure 1 This is the system logic block diagram of this solution;

[0042] Figure 2 Flowchart of empirical data cleaning and preprocessing for this scheme;

[0043] Figure 3 This is the data verification flow chart for this solution;

[0044] Figure 4 This is a flow chart of the hybrid scoring algorithm output by the intelligent retrieval engine of this solution for retrieval results. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] See also Figure 1 、 Figure 2 , an aviation maintenance experience knowledge management system based on multi-source data processing, including data cleaning and pre-processing module, storage module, document parsing module, and index generation module;

[0047] The data cleaning and preprocessing module is used to perform data cleaning, entity extraction, data validation and vectorization processing on the input raw multi-source data;

[0048] The storage module is used to store aviation maintenance experience data that has passed data validation and compliance, as well as various written PDF documents (ordinary PDF documents, procedure manuals);

[0049] Data needs to be vectorized before being stored in the vector database in the storage module;

[0050] Vectorization uses Langchain's text embedding model (such as OpenAIEmbeddings or HuggingFaceEmbeddings) to convert fault descriptions into vectors and store them in the vectorized database in the storage module. Other data is stored as metadata in a vector database (such as FAISS or Chroma) according to custom schema rules (field type, length, etc.). An asynchronous method (FastAPI + Huey task queue) is used to improve writing efficiency.

[0051] The document parsing module creates PDF parsing rules, including file path, exclusion page list, and cropping range size; uses PyPDF2 or pdfplumber to parse PDF content (implementing content extraction); stores parsing rules for the same type of PDF in the storage module's vector database, and automatically applies these rules to subsequent PDFs in the same folder;

[0052] The index generation module generates an index file by recording the page number and content of each PDF page. It checks the last modification time of the PDF through a scheduled task (Huey). If it is later than the index generation time, the index is automatically updated and the old file is renamed according to the date to achieve index version control.

[0053] It also includes a program manual parser that sets parsing rules: delimiter, batch character length, overlapping character length, and whether to enable regular expression matching. The program manual parser uses langchain's RecursiveCharacterTextSplitter to split text and store it in a vector database.

[0054] See also Figure 1 、 Figure 2 、 Figure 4The aviation maintenance experience knowledge management system based on multi-source data processing also includes an intelligent retrieval engine. Based on a vector database and index files, it uses semantic similarity calculation values ​​and time-weighted score calculation values ​​to perform weighted merging and normalization processing to obtain result sorting. After sorting in descending order, the first N data are intercepted and output as the processing method in the experience data.

[0055] The intelligent search engine can call large language models (such as DeepSeek) to merge processing methods with high similarity to generate a comprehensive solution; if the output experience data contains a clause number, the corresponding page number is retrieved in the index and the URL of the PDF page is generated for the user to view; the program manual is retrieved through similarity matching, and the summary results and reference PDF page URL are returned.

[0056] See also Figure 1 、 Figure 2 、 Figure 3 ,Aviation maintenance experience knowledge management system based on multi-source data processing,,the data cleaning process includes null value filtering, ATA chapter parsing,,type mapping;

[0057] Null value processing process: Load the fault maintenance record dataset; identify and delete records with empty fault report or treatment action fields; standardize and rename data column names; uniformly replace null values ​​(NaN) in the dataset with empty strings;

[0058] ATA chapter parsing: Segment the ATA chapter number field, using hyphens "-" as separators; generate standardized chapter number segmentation data;

[0059] Type mapping: Establish a mapping relationship table between the digital code and text description of the fault type; establish a mapping relationship table between the digital code and text description of the treatment type; establish a mapping relationship table between the digital code and text description of the occurrence stage; convert the digital code into the corresponding text description according to the mapping relationship table;

[0060] Entity extraction uses large language model technology to extract key entity information from text. The extracted objects include fault content and processing content.

[0061] Fault content extraction includes the following entity elements: fault location, fault component, fault phenomenon description, fault related information, and fault diagnosis code;

[0062] The processing content extraction includes the following entity elements: maintenance release level, applicable clause number, specific processing method, and final processing result;

[0063] Guide the entity extraction process by constructing a prompt word template;

[0064] Data validation includes: checking the integrity of extracted entity data; identifying and marking missing or malformed entity items; correcting non-compliant data using text matching rules; and re-extracting missing key entities.

[0065] The data cleaning and pre-processing module also includes data merging and comparison, and data processing of dismantled and replaced parts;

[0066] Data merging and comparison: Associate and merge the extracted fault content entity with the processed content entity; compare the merged entity data with the original records for consistency; and manually review and adjust any discrepant data items.

[0067] Processing of replacement parts data: Clean and standardize replacement parts records; establish a coding mapping relationship table between replacement parts processing types and operation types; group and integrate replacement parts data according to maintenance record ID; associate and merge processed replacement parts data with the main fault processing records.

[0068] This solution also discloses an aviation maintenance experience knowledge management method based on multi-source data processing, including the following steps:

[0069] Step 1: Empirical data cleaning and preprocessing

[0070] The raw data is merged and output after being filtered for null values, field normalized, ATA chapter parsed, type mapped, entity extracted, and data validated;

[0071] Step 2: Document processing

[0072] Parse, extract, process and generate indexes for PDF document data;

[0073] Step 3: Data storage

[0074] Store the vectorized experience data and document data;

[0075] Step 4: Retrieve call

[0076] Use the intelligent search engine to retrieve data from the vector database and display it on the knowledge application interface.

[0077] Unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; and internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this application.

[0078] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. Aviation maintenance experience knowledge management system based on multi-source data processing, characterized by: Includes data cleaning and preprocessing module, storage module, document parsing module, and index generation module; The data cleaning and preprocessing module is used to perform data cleaning, entity extraction, data validation and vectorization processing on the input original multi-source data; The document parsing module parses the PDF document content; The storage module is used to store the processed aviation maintenance experience data and various written PDF document data; The index generation module is used to generate an index file.

2. The aviation maintenance experience knowledge management system based on multi-source data processing according to claim 1 is characterized by: It also includes an intelligent retrieval engine for searching target data from storage modules and index files.

3. The aviation maintenance experience knowledge management system based on multi-source data processing according to claim 1 is characterized by: The data cleaning process includes null value filtering, ATA chapter parsing, and type mapping.

4. The aviation maintenance experience knowledge management system based on multi-source data processing according to claim 1 is characterized by: The entity extraction uses a large language model technology to extract key entity information from the data.

5. The aviation maintenance experience knowledge management system based on multi-source data processing according to claim 4 is characterized by: The data validation includes: checking the integrity of the extracted entity data; identifying and marking missing or formatted entity items; correcting non-compliant data using text matching rules; and re-extracting missing key entities.

6. The aviation maintenance experience knowledge management system based on multi-source data processing according to claim 1 is characterized by: The original multi-source data and PDF document data are vectorized and stored in a vector database in the storage module.

7. The aviation maintenance experience knowledge management system based on multi-source data processing according to claim 5 is characterized by: The data cleaning and pre-processing module also includes data merging and comparison, and replacement parts data processing.

8. The aviation maintenance experience knowledge management system based on multi-source data processing according to claim 1 is characterized by: The document parsing module also includes a program manual parser for parsing the program manual as a PDF document.

9. An aviation maintenance experience knowledge management method based on multi-source data processing according to claims 1-8, characterized in that: The steps include: Step 1: Empirical data cleaning and preprocessing The raw data is merged and output after being filtered for null values, field normalized, ATA chapter parsed, type mapped, entity extracted, and data validated; Step 2: Document processing Parse, extract, process and generate indexes for PDF document data; Step 3: Data storage Store the vectorized experience data and document data; Step 4: Retrieve call Use the intelligent retrieval engine to call the data in the vector database and display it on the knowledge application interface.

Images