Aviation multi-field data adaptive extraction method and system based on large model

Abstract

The invention provides an aviation multi-field data adaptive extraction method and system based on a large model, and relates to the technical field of data retrieval, and the method comprises the steps: obtaining an aviation field multi-field data set, and carrying out the preprocessing of the data set to obtain a preprocessed data set; constructing a plurality of inverted index tables based on the preprocessed data set; performing semantic feature extraction on each paragraph of each text in the preprocessed data set based on a trained BGE model to obtain a semantic feature vector library; performing semantic feature extraction on a text input by a user based on the trained BGE model to obtain an input feature vector; calculating based on the large model to obtain a cosine similarity set and a morpheme similarity set; carrying out normalization processing on the morpheme similarity set to obtain a normalized value set; and performing fusion sorting on the normalized value set and the cosine similarity set to obtain an extraction result. According to the method, the keyword extraction result and the semantic extraction result are fused and sorted at the same time, so that the extraction richness and the sorting accuracy are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of data retrieval, and in particular to a method and system for adaptively extracting aviation multi-domain data based on a large model. Background Art

[0002] When using the aviation knowledge service platform, one of the key needs of users is to search and use multiple types of aviation data. Due to the different query purposes and query data types of users, the fixed search statements and weights used in traditional search methods are difficult to meet the user's search experience and needs. The problems mainly include the following aspects:

[0003] (1) During the user search process, they need to manually select specific fields for search, which is a low level of intelligence;

[0004] (2) Fixed search statements and search field weights are inconsistent with the user's search purpose, causing the search results to deviate from the user's expectations;

[0005] (3) Traditional keyword search only focuses on character matching and ignores semantic information, resulting in inaccurate search results. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for adaptive extraction of aviation multi-domain data based on a large model, which extracts and compares semantic features of retrieval content and text information, and integrates and sorts the results of keyword extraction and semantic extraction to ensure the richness of extraction and the accuracy of sorting.

[0007] A method for adaptively extracting aviation multi-domain data based on a large model, comprising:

[0008] Acquire a multi-domain dataset in the aviation field, and preprocess the multi-source dataset in the aviation field to obtain a preprocessed dataset;

[0009] Constructing a plurality of inverted index tables based on the preprocessed data set; each of the inverted index tables corresponds to a field; the inverted index tables include a Json key-value pair and a plurality of morphemes, each of the morphemes corresponds to a text in the preprocessed data set; the Json key-value pair includes an index name, index data content and purpose;

[0010] Splitting a number of texts in the preprocessed data set into sentences, and dividing each sentence based on the large model to obtain a first training set, and performing unsupervised training on the BGE model based on the first training set to obtain the initially trained BGE model;

[0011] Using several text paragraphs in the preprocessed data set as a second training set, and asking questions to the several text paragraphs in the preprocessed data set based on the large model to obtain a long and short text annotated data set;

[0012] Using the long and short text annotation dataset as a label set and combining it with the second training set to perform supervised training on the initially trained BGE model to obtain the trained BGE model;

[0013] Extracting semantic features from each paragraph of each text in the preprocessed data set based on the trained BGE model to obtain a semantic feature vector library;

[0014] Extracting semantic features of the text input by the user based on the trained BGE model to obtain an input feature vector;

[0015] Based on the large model, cosine similarity calculation is performed on the input feature vector and each semantic feature vector in the semantic feature vector library to obtain an initial cosine similarity set;

[0016] Based on the Json key-value pairs of each inverted index table, a large model is used to analyze the text input by the user to obtain a demand inverted index table; the demand inverted index table is an inverted index table corresponding to the field corresponding to the text input by the user;

[0017] Based on each morpheme in the demand inverted index table, a large model is used to form a query statement and a weight of each query field in the query statement according to the text input by the user;

[0018] Based on the large model and the weights of each query field in the query statement, similarity calculation is performed on each morpheme in the query statement and the demand inverted index table to obtain a morpheme similarity set; and the morpheme similarity set is normalized to obtain an initial normalized value set;

[0019] If the normalized value in the initial normalized value set and the cosine similarity in the initial cosine similarity set correspond to the same text in the preprocessed data set, the text is taken as the current text, the normalized value and the cosine similarity corresponding to the current text are weighted summed to obtain a fusion value, and the cosine similarity corresponding to the current text is deleted from the initial cosine similarity set, and the normalized value corresponding to the current text is deleted from the initial normalized value set, and the initial normalized value set and the initial cosine similarity set are traversed to obtain a normalized value set, a cosine similarity set and a fusion value set; the fusion value set includes a plurality of fusion values;

[0020] The fusion values ​​in the fusion value set, the normalized values ​​in the normalized value set, and the cosine similarities in the cosine similarity set are sorted from large to small, and corresponding texts are given as extraction results according to the required quantity.

[0021] Optionally, constructing an inverted index table based on the preprocessed data set includes:

[0022] Extract aviation-related terms from the Air Force Aviation Engineering Dictionary, the Commercial Aircraft Professional Terminology Dictionary, the China Aviation Encyclopedia Dictionary, and aviation standards to obtain a term list;

[0023] Splitting the preprocessed data set into words according to the term list and the search engine's own word list to obtain a plurality of morphemes;

[0024] The morpheme is used as the key value, and the text corresponding to the morpheme is used as the value value to construct a key-value pair to obtain the inverted index table.

[0025] Optionally, the cosine similarity expression is:

[0026]

[0027] Where A is the input feature vector, B is the semantic feature vector, n is the dimension of the feature vector, A i is the value of the i-th dimension of the input feature vector, B i Represents the value of the i-th dimension of the semantic feature vector, Sim cos AB Represents the cosine similarity between the input feature vector and the semantic feature vector.

[0028] Optionally, the morpheme similarity expression is:

[0029]

[0030]

[0031] Where: w i Represents a query statement, w j represents the jth morpheme in the demand inverted index table, j = [1,,2,…,N], N is the total number of morphemes in the demand inverted index table, n is the total number of query fields in the query statement, and w ik represents the kth query field in the query statement, IDF(w ik ) indicates w ik The weight of f(w ik ,w j ) indicates that w j Medium ik The number of occurrences, k1 is the first adjustment factor, b is the second adjustment factor, |wj | indicates w j The length of the corresponding text, avgdl represents the average length of all texts in the required inverted index table.

[0032] Optionally, the fusion value expression is:

[0033] score final =W*sigmoid(score bm25 ,c,a)+d*score cosine ;

[0034]

[0035] Where: score final represents the fusion value, W is the weight of the normalized value, d represents the weight of the cosine similarity, c is the first parameter of the simoid function, a is the second parameter of the simoid function, and score cosine is the cosine similarity, score bm25 is the morpheme similarity, sigmoid(score bm25 ,c,a) represents the normalized value of morpheme similarity.

[0036] Optionally, the loss function during unsupervised training and supervised training is:

[0037]

[0038] Where: F is the loss function value, p and q represent text pairs, is a negative sample, τ is the BGE model temperature coefficient, e p is the eigenvector of p, e q is the eigenvector of q, yes The characteristic vector of (p，q) is the sum of the loss function values ​​of all text pairs, min·Σ (p，q) It is the minimum value of the sum of loss function values ​​of all text pairs.

[0039] Optionally, the large model is Qwen2.5-72B.

[0040] The present invention also provides an aviation multi-source data adaptive extraction system based on a large model, which comprises:

[0041] A data acquisition processing module is used to acquire a multi-domain data set in the aviation field, and pre-process the multi-source data set in the aviation field to obtain a pre-processed data set;

[0042] An index table module, used to construct a plurality of inverted index tables based on the preprocessed data set; each of the inverted index tables corresponds to a field; the inverted index table includes a Json key-value pair and a plurality of morphemes, each of the morphemes corresponds to a text in the preprocessed data set; the Json key-value pair includes an index name, index data content and purpose;

[0043] A first training module is used to split the plurality of texts in the preprocessed data set into sentences, and divide each sentence based on the large model to obtain a first training set, and perform unsupervised training on the BGE model based on the first training set to obtain the initially trained BGE model;

[0044] A labeling module, used for taking the paragraphs of several texts in the preprocessed data set as a second training set, and asking questions to the paragraphs of several texts in the preprocessed data set based on the large model to obtain a long and short text labeling data set;

[0045] A second training module is used to use the long and short text annotation dataset as a label set and combine it with the second training set to perform supervised training on the initially trained BGE model to obtain the trained BGE model;

[0046] A vector library module is used to extract semantic features from each paragraph of each text in the preprocessed data set based on the trained BGE model to obtain a semantic feature vector library;

[0047] A feature extraction module is used to extract semantic features of the text input by the user based on the trained BGE model to obtain an input feature vector;

[0048] A cosine similarity module, used for calculating the cosine similarity between the input feature vector and each semantic feature vector in the semantic feature vector library based on a large model to obtain an initial cosine similarity set;

[0049] The demand index table module is used to analyze the text input by the user using a large model based on the Json key-value pairs of each inverted index table to obtain a demand inverted index table; the demand inverted index table is an inverted index table corresponding to the field corresponding to the text input by the user;

[0050] A weight module, for forming a query statement and a weight of each query field in the query statement using a large model according to the text input by the user based on each morpheme in the demand inverted index table;

[0051] A morpheme similarity module is used to calculate the similarity between the query statement and each morpheme in the demand inverted index table based on the large model and the weight of each query field in the query statement to obtain a morpheme similarity set; and normalize the morpheme similarity set to obtain an initial normalized value set;

[0052] A fusion module, for, if the normalized value in the initial normalized value set and the cosine similarity in the initial cosine similarity set correspond to the same text in the preprocessed data set, taking the text as the current text, performing weighted summation on the normalized value and the cosine similarity corresponding to the current text to obtain a fusion value, deleting the cosine similarity corresponding to the current text from the initial cosine similarity set, deleting the normalized value corresponding to the current text from the initial normalized value set, traversing the initial normalized value set and the initial cosine similarity set to obtain a normalized value set, a cosine similarity set and a fusion value set; the fusion value set includes a plurality of the fusion values;

[0053] The text extraction module is used to sort the fusion values ​​in the fusion value set, the normalized values ​​in the normalized value set and the cosine similarities in the cosine similarity set from large to small and provide corresponding texts as extraction results according to the required quantity.

[0054] The effects of the present invention are as follows:

[0055] The invention discloses a large-model-based aviation multi-domain data adaptive extraction method, which can automatically select a retrieval field according to a user's query question.

[0056] The invention discloses a large-model-based adaptive extraction method for aviation multi-domain data, which automatically generates search weights and search statements according to the relationship between user query questions and search fields.

[0057] The invention is based on a large-model adaptive extraction method for aviation multi-domain data, which extracts and compares semantic features of retrieval content and text information, and simultaneously integrates and sorts the results of keyword extraction and semantic extraction to ensure the richness of extraction and the accuracy of sorting. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flow chart of the adaptive extraction method of aviation multi-domain data based on a large model of the present invention. DETAILED DESCRIPTION

[0059] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0060] Figure 1 This is a flow chart of the adaptive extraction method of aviation multi-domain data based on a large model of the present invention. Figure 1As shown, the present invention provides a method for adaptively extracting aviation multi-domain data based on a large model, which includes:

[0061] S1, obtain multi-domain datasets in the aviation field, and preprocess the multi-source datasets in the aviation field to obtain preprocessed datasets.

[0062] Specifically, the papers in the field of aviation are downloaded and the metadata related to the papers are obtained, including title, author, publication date, keywords and abstract. Since most of the data obtained are PDF files, it is necessary to perform text recognition and layout analysis on some files to form chapter titles and the text corresponding to the chapter titles to obtain the preprocessed data set.

[0063] S2, build several inverted index tables based on the preprocessed data set. Each inverted index table corresponds to a field. The inverted index table includes a Json key-value pair and several morphemes, each of which corresponds to a text in the preprocessed data set. The Json key-value pair includes the index name, index data content and purpose.

[0064] Preferably, S2 includes:

[0065] S21, extract aviation-related terms from the Air Force Aviation Engineering Dictionary, the Commercial Aircraft Professional Terminology Dictionary, the China Aviation Encyclopedia Dictionary and aviation standards to obtain a term list.

[0066] S22, splitting the preprocessed data set into words according to the term list and the search engine's own word list to obtain a number of morphemes.

[0067] S23, taking the morpheme as the key value and the text corresponding to the morpheme as the value value to construct a key-value pair, and obtain an inverted index table.

[0068] S3, splitting several texts in the preprocessed data set into sentences, and dividing each sentence based on the large model to obtain a first training set, and performing unsupervised training on the BGE model based on the first training set to obtain an initially trained BGE model. In this embodiment, the large model is a large language model, preferably Qwen2.5-72B.

[0069] S4, taking several text paragraphs in the preprocessed data set as the second training set, and asking questions to several text paragraphs in the preprocessed data set based on the large model to obtain a long and short text annotation data set.

[0070] S5, using the long and short text annotation dataset as a label set and combining it with the second training set to perform supervised training on the initially trained BGE model to obtain a trained BGE model.

[0071] The loss function during unsupervised training and supervised training is:

[0072]

[0073] Where: F is the loss function value, p and q represent text pairs, is a negative sample, τ is the BGE model temperature coefficient, e p is the eigenvector of p, e q is the eigenvector of q, yes The eigenvector of (p，q) is the sum of the loss function values ​​of all text pairs, min·Σ (p，q) It is the minimum value of the sum of loss function values ​​of all text pairs.

[0074] S6, based on the trained BGE model, semantic features of each paragraph of each text in the preprocessed data set are extracted to obtain a semantic feature vector library.

[0075] S7, extracting semantic features of the text input by the user based on the trained BGE model to obtain an input feature vector. The present invention fine-tunes the multi-source data corpus in the aviation field through the BGE model to make the vector extraction and sorting more accurate.

[0076] S8, based on the large model, the cosine similarity between the input feature vector and each semantic feature vector in the semantic feature vector library is calculated to obtain an initial cosine similarity set. The BGE model is a fine-tuning semantic model, which is used to fine-tune the text input by the user so that the length of the semantic feature vector is consistent with that of the input feature vector.

[0077] The cosine similarity expression is:

[0078]

[0079] Where A is the input feature vector, B is the semantic feature vector, n is the dimension of the feature vector, A i is the value of the i-th dimension of the input feature vector, B i Represents the value of the i-th dimension of the semantic feature vector, Sim cos AB Represents the cosine similarity between the input feature vector and the semantic feature vector.

[0080] S9, based on the Json key-value pairs of each inverted index table, a large model is used to analyze the text input by the user to obtain a demand inverted index table. The demand inverted index table is an inverted index table corresponding to the field corresponding to the text input by the user.

[0081] S10, based on each morpheme in the demand inverted index table, a large model is used to form a query statement and a weight of each query field in the query statement according to the text input by the user.

[0082] S11, based on the large model and the weights of each query field in the query statement, the similarity between the query statement and each morpheme in the demand inverted index table is calculated to obtain a morpheme similarity set; and the morpheme similarity set is normalized to obtain an initial normalized value set.

[0083] The expression of morpheme similarity is:

[0084]

[0085]

[0086] Where: w i Represents a query statement, w j represents the jth morpheme in the demand inverted index table, j = [1,,2,…,N], N is the total number of morphemes in the demand inverted index table, n is the total number of query fields in the query statement, and w ik represents the kth query field in the query statement, IDF(w ik ) indicates w ik The weight of f(w ik ,w j ) indicates that w j Medium ik The number of occurrences, k1 is the first adjustment factor, b is the second adjustment factor, |w j | indicates w j The length of the corresponding text, avgdl represents the average length of all texts in the required inverted index table.

[0087] S12, if the normalized value in the initial normalized value set and the cosine similarity in the initial cosine similarity set correspond to the same text in the preprocessed data set, take this text as the current text, perform weighted summation on the normalized value and cosine similarity corresponding to the current text, obtain a fusion value, delete the cosine similarity corresponding to the current text from the initial cosine similarity set, delete the normalized value corresponding to the current text from the initial normalized value set, traverse the initial normalized value set and the initial cosine similarity set, and obtain a normalized value set, a cosine similarity set and a fusion value set. The fusion value set includes several fusion values.

[0088] Specifically, the fusion value expression is:

[0089] score final =W*sigmoid(score bm25 ,c,a)+d*score cosine ;

[0090]

[0091] Where: scorefinal represents the fusion value, W is the weight of the normalized value, d represents the weight of the cosine similarity, c is the first parameter of the simoid function, a is the second parameter of the simoid function, and score cosine is the cosine similarity, score bm25 is the morpheme similarity, sigmoid(score bm25 ,c,a) represents the normalized value of morpheme similarity.

[0092] S13, sorting the fusion values ​​in the fusion value set, the normalized values ​​in the normalized value set, and the cosine similarities in the cosine similarity set from large to small, and providing corresponding texts as extraction results according to the required quantity.

[0093] Specifically, when the user inputs: What are the applicable standards for turbines, compressors, fans, and turbocharger rotors, it corresponds to the aviation standards field, and the demand inverted index table is shown in Table 1.

[0094] Table 1 Inverted index table

[0095] index document engine 4 Rotor 15、13、5 Compressor 14、3、6 fan 13、7、9 … …

[0096] The input feature vector is a 1024-dimensional feature vector. The cosine similarities obtained in documents 1, 8, and 9 are 0.7981214340483506, 0.7781211345683576, and 0.7651324545987272, respectively. The morpheme similarity scores obtained in documents 1, 7, and 9 are 74, 52, and 41, respectively. The values ​​obtained after sigmoid normalization are 0.999997617294756, 0.999929166980283, and 0.9992376924918762. When the weights of morpheme similarity and cosine similarity are both 1, the final scores of documents 1, 7, 8, and 9 are 1.7981190513431065, 0.999929166980283, 0.7781211345683576, and 1.7643701470906035, respectively, and the final ranking results are documents 1, 9, 7, and 8.

[0097] Preferably, metadata corresponding to the text in the extraction result is displayed according to actual needs.

[0098] The method of the present invention is implemented based on the Elasticsearch search engine.

[0099] The present invention also provides an aviation multi-source data adaptive extraction system based on a large model, which comprises:

[0100] The data acquisition and processing module is used to acquire multi-domain data sets in the aviation field and pre-process the multi-source data sets in the aviation field to obtain pre-processed data sets.

[0101] The index table module is used to build several inverted index tables based on the preprocessed data set. Each inverted index table corresponds to a field. The inverted index table includes a Json key-value pair and several morphemes, each of which corresponds to a text in the preprocessed data set. The Json key-value pair includes the index name, index data content and purpose.

[0102] The first training module is used to split several texts in the preprocessed data set into sentences, and divide each sentence based on the large model to obtain a first training set, and perform unsupervised training on the BGE model based on the first training set to obtain an initially trained BGE model.

[0103] The annotation module is used to use several text paragraphs in the preprocessed data set as the second training set, and to ask questions to the several text paragraphs in the preprocessed data set based on the large model to obtain a long and short text annotation data set.

[0104] The second training module is used to use the long and short text annotation dataset as a label set and combine it with the second training set to perform supervised training on the initially trained BGE model to obtain a trained BGE model.

[0105] The vector library module is used to extract semantic features from each paragraph of each text in the preprocessed data set based on the trained BGE model to obtain a semantic feature vector library.

[0106] The feature extraction module is used to extract semantic features of the text input by the user based on the trained BGE model to obtain an input feature vector.

[0107] The cosine similarity module is used to calculate the cosine similarity between the input feature vector and each semantic feature vector in the semantic feature vector library based on the large model to obtain an initial cosine similarity set.

[0108] The demand index table module is used to analyze the text input by the user based on the Json key-value pairs of each inverted index table using a large model to obtain a demand inverted index table. The demand inverted index table is an inverted index table corresponding to the field corresponding to the text input by the user.

[0109] The weight module is used to form a query statement and the weight of each query field in the query statement based on each morpheme in the demand inverted index table using a large model according to the text input by the user.

[0110] The morpheme similarity module is used to calculate the similarity between the query statement and each morpheme in the demand inverted index table based on the large model and the weight of each query field in the query statement to obtain a morpheme similarity set; and normalize the morpheme similarity set to obtain an initial normalized value set.

[0111] The fusion module is used for, if the normalized value in the initial normalized value set and the cosine similarity in the initial cosine similarity set correspond to the same text in the preprocessed data set, taking the text as the current text, performing weighted summation on the normalized value and cosine similarity corresponding to the current text, obtaining the fusion value, deleting the cosine similarity corresponding to the current text from the initial cosine similarity set, deleting the normalized value corresponding to the current text from the initial normalized value set, traversing the initial normalized value set and the initial cosine similarity set, obtaining the normalized value set, the cosine similarity set and the fusion value set. The fusion value set includes several fusion values.

[0112] The text extraction module is used to sort the fusion values ​​in the fusion value set, the normalized values ​​in the normalized value set and the cosine similarities in the cosine similarity set from large to small and give the corresponding text as the extraction result according to the required quantity.

[0113] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for adaptively extracting aviation multi-domain data based on a large model, characterized in that: It includes: Acquire a multi-domain dataset in the aviation field, and preprocess the multi-source dataset in the aviation field to obtain a preprocessed dataset; Constructing a plurality of inverted index tables based on the preprocessed data set; Each of the inverted index tables corresponds to a field; the inverted index table includes a Json key-value pair and a plurality of morphemes, each of which corresponds to a text in the preprocessed data set; the Json key-value pair includes an index name, index data content and purpose; Splitting a number of texts in the preprocessed data set into sentences, and dividing each sentence based on the large model to obtain a first training set, and performing unsupervised training on the BGE model based on the first training set to obtain the initially trained BGE model; Using several text paragraphs in the preprocessed data set as a second training set, and asking questions to the several text paragraphs in the preprocessed data set based on the large model to obtain a long and short text annotated data set; Using the long and short text annotation dataset as a label set and combining it with the second training set to perform supervised training on the initially trained BGE model to obtain the trained BGE model; Extracting semantic features from each paragraph of each text in the preprocessed data set based on the trained BGE model to obtain a semantic feature vector library; Extracting semantic features of the text input by the user based on the trained BGE model to obtain an input feature vector; Based on the large model, cosine similarity calculation is performed on the input feature vector and each semantic feature vector in the semantic feature vector library to obtain an initial cosine similarity set; Based on the Json key-value pairs of each inverted index table, a large model is used to analyze the text input by the user to obtain a demand inverted index table; The demand inverted index table is an inverted index table corresponding to the field corresponding to the text input by the user; Based on each morpheme in the demand inverted index table, a large model is used to form a query statement and a weight of each query field in the query statement according to the text input by the user; Based on the large model and the weight of each query field in the query statement, similarity calculation is performed between the query statement and each morpheme in the demand inverted index table to obtain a morpheme similarity set; and normalizing the morpheme similarity set to obtain an initial normalized value set; If the normalized value in the initial normalized value set and the cosine similarity in the initial cosine similarity set correspond to the same text in the preprocessed data set, take this text as the current text, perform weighted summation on the normalized value and the cosine similarity corresponding to the current text to obtain a fusion value, delete the cosine similarity corresponding to the current text from the initial cosine similarity set, delete the normalized value corresponding to the current text from the initial normalized value set, traverse the initial normalized value set and the initial cosine similarity set, and obtain a normalized value set, a cosine similarity set, and a fusion value set; The fusion value set includes a plurality of fusion values; The fusion values ​​in the fusion value set, the normalized values ​​in the normalized value set, and the cosine similarities in the cosine similarity set are sorted from large to small, and corresponding texts are given as extraction results according to the required quantity.

2. The method for adaptively extracting aviation multi-source data based on a large model according to claim 1 is characterized in that: The step of constructing an inverted index table based on the preprocessed data set includes: Extract aviation-related terms from the Air Force Aviation Engineering Dictionary, the Commercial Aircraft Professional Terminology Dictionary, the China Aviation Encyclopedia Dictionary, and aviation standards to obtain a term list; Splitting the preprocessed data set into words according to the term list and the search engine's own word list to obtain a plurality of morphemes; The morpheme is used as the key value, and the text corresponding to the morpheme is used as the value value to construct a key-value pair to obtain the inverted index table.

3. The method for adaptively extracting aviation multi-source data based on a large model according to claim 1 is characterized in that: The cosine similarity expression is: Where A is the input feature vector, B is the semantic feature vector, n is the dimension of the feature vector, A i is the value of the i-th dimension of the input feature vector, B i Represents the value of the i-th dimension of the semantic feature vector, Sim cos AB Represents the cosine similarity between the input feature vector and the semantic feature vector.

4. The method for adaptively extracting aviation multi-source data based on a large model according to claim 1, characterized in that: The morpheme similarity expression is: Where: w i Represents a query statement, w j represents the jth morpheme in the demand inverted index table, j = [1,,2,…,N], N is the total number of morphemes in the demand inverted index table, n is the total number of query fields in the query statement, and w ik represents the kth query field in the query statement, IDF(w ik ) indicates w ik The weight of f(w ik ,w j ) indicates that w j Medium ik The number of occurrences, k1 is the first adjustment factor, b is the second adjustment factor, |w j | indicates w j The length of the corresponding text, avgdl represents the average length of all texts in the required inverted index table.

5. The method for adaptively extracting aviation multi-source data based on a large model according to claim 1, characterized in that: The fusion value expression is: score final =W*sigmoid(score bm25 ,c,a)·+d*score cosine ; Where: score final represents the fusion value, W is the weight of the normalized value, d represents the weight of the cosine similarity, c is the first parameter of the simoid function, a is the second parameter of the simoid function, and score cosine is the cosine similarity, score bm25 is the morpheme similarity, sigmoid(score bm25 ,c,a) represents the normalized value of morpheme similarity.

6. The method for adaptively extracting aviation multi-source data based on a large model according to claim 1, characterized in that: The loss function during unsupervised training and supervised training is: Where: F is the loss function value, p and q represent text pairs, is a negative sample, τ is the BGE model temperature coefficient, e p is the eigenvector of p, e q is the eigenvector of q, yes The eigenvector of (p，q) is the sum of the loss function values ​​of all text pairs, min·∑ (p，q) It is the minimum value of the sum of loss function values ​​of all text pairs.

7. The method for adaptively extracting aviation multi-source data based on a large model according to claim 1, characterized in that: The large model is Qwen2.5-72B.

8. An adaptive extraction system for aviation multi-source data based on a large model, characterized in that: It includes: A data acquisition processing module is used to acquire a multi-domain data set in the aviation field, and pre-process the multi-source data set in the aviation field to obtain a pre-processed data set; An index table module, used to construct a plurality of inverted index tables based on the preprocessed data set; each of the inverted index tables corresponds to a field; the inverted index table includes a Json key-value pair and a plurality of morphemes, each of the morphemes corresponds to a text in the preprocessed data set; the Json key-value pair includes an index name, index data content and purpose; A first training module is used to split the plurality of texts in the preprocessed data set into sentences, and divide each sentence based on the large model to obtain a first training set, and perform unsupervised training on the BGE model based on the first training set to obtain the initially trained BGE model; A labeling module, used for taking the paragraphs of several texts in the preprocessed data set as a second training set, and asking questions to the paragraphs of several texts in the preprocessed data set based on the large model to obtain a long and short text labeling data set; A second training module is used to use the long and short text annotation dataset as a label set and combine it with the second training set to perform supervised training on the initially trained BGE model to obtain the trained BGE model; A vector library module is used to extract semantic features from each paragraph of each text in the preprocessed data set based on the trained BGE model to obtain a semantic feature vector library; A feature extraction module is used to extract semantic features of the text input by the user based on the trained BGE model to obtain an input feature vector; A cosine similarity module, used for calculating the cosine similarity between the input feature vector and each semantic feature vector in the semantic feature vector library based on a large model to obtain an initial cosine similarity set; The demand index table module is used to analyze the text input by the user using a large model based on the Json key-value pairs of each inverted index table to obtain a demand inverted index table; The demand inverted index table is an inverted index table corresponding to the field corresponding to the text input by the user; A weight module, used to form a query statement and a weight of each query field in the query statement based on each morpheme in the demand inverted index table and a large model according to the text input by the user; A morpheme similarity module, used to calculate the similarity between the query statement and each morpheme in the demand inverted index table based on the large model and the weight of each query field in the query statement, so as to obtain a morpheme similarity set; and normalizing the morpheme similarity set to obtain an initial normalized value set; A fusion module, for, if the normalized value in the initial normalized value set and the cosine similarity in the initial cosine similarity set correspond to the same text in the preprocessed data set, taking the text as the current text, performing weighted summation on the normalized value and the cosine similarity corresponding to the current text to obtain a fusion value, deleting the cosine similarity corresponding to the current text from the initial cosine similarity set, deleting the normalized value corresponding to the current text from the initial normalized value set, traversing the initial normalized value set and the initial cosine similarity set to obtain a normalized value set, a cosine similarity set and a fusion value set; the fusion value set includes a plurality of the fusion values; The text extraction module is used to sort the fusion values ​​in the fusion value set, the normalized values ​​in the normalized value set and the cosine similarities in the cosine similarity set from large to small and provide corresponding texts as extraction results according to the required quantity.

Images