A quality analysis method for aerospace equipment

Abstract

The present invention relates to a quality analysis method for aerospace equipment, belongs to the field of semantic processing technology, and solves the problem of low efficiency and accuracy of quality analysis in the prior art. The method comprises: S1, passing the data to be analyzed into a trained multimodal embedding model to obtain a feature vector to be analyzed; updating the multimodal correlation matrix according to the latest user interaction data to obtain a real-time correlation matrix; S2, outputting the retrieval results using the HNSW graph according to the feature vector to be analyzed and the real-time correlation matrix; S3, passing the data to be analyzed and the selected retrieval results into a trained large language model, and outputting the answer to the question according to the submitted question; if the answer to the question is recognized, it is used as the quality analysis result; otherwise, obtaining supplementary data and using it as new data to be analyzed, iteratively executing steps S1-S3 until the quality analysis result is obtained. Efficient and accurate quality analysis of aerospace equipment data is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of semantic processing, and in particular to a quality analysis method for aerospace equipment. Background Art

[0002] As the cornerstone of space missions, the quality of aerospace equipment is directly related to the success of space missions. Therefore, every component in aerospace equipment, even the smallest bolt, undergoes strict quality inspection and analysis during the production process to promptly detect and repair potential defects, ensure the normal operation of the equipment, reduce the number of defective products to reduce costs, and ensure that these equipment can maintain optimal performance in extreme environments. However, in the quality analysis process, it is still a challenge to efficiently and quickly analyze the causes of product defects and find solutions.

[0003] The existing large language model can quickly process all kinds of data generated during the production process and has strong context understanding capabilities. It can combine structured data (such as production parameters and sensor readings) and unstructured data (such as images) to assist users in discovering subtle details in production quality issues and improve the timeliness and comprehensiveness of problem analysis. After the large language model is trained, it outputs the corresponding answer based on the input question.

[0004] However, considering the particularity and confidentiality of the aerospace field, it is not possible to directly apply the capabilities of existing large language models to data analysis scenarios in the aerospace field. Moreover, the aerospace field includes massive amounts of data, and how to improve query efficiency and how to extract the closest answer from the massive data based on its relevance requires further research. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a quality analysis method for aerospace equipment to solve the problem of low efficiency and accuracy of existing quality analysis.

[0006] An embodiment of the present invention provides a quality analysis method for aerospace equipment, comprising the following steps:

[0007] S1. Pass the data to be analyzed into the trained multimodal embedding model to obtain the feature vector to be analyzed; update the multimodal correlation matrix according to the latest user interaction data to obtain the real-time correlation matrix;

[0008] S2, outputting the search results using the HNSW graph according to the feature vector to be analyzed and the real-time correlation matrix;

[0009] S3. The data to be analyzed and the selected search results are passed into the trained large language model, and the answer to the question is output according to the submitted question; if the answer to the question is recognized, it is used as the quality analysis result; otherwise, the supplementary data is obtained and used as the new data to be analyzed, and steps S1-S3 are iteratively executed until the quality analysis result is obtained.

[0010] As a further improvement of the above method, the multimodal embedding model is obtained by fine-tuning the pre-trained multimodal large model based on historical aerospace equipment data by introducing the multimodal correlation matrix in the Lora algorithm.

[0011] Based on the further improvement of the above method, each business type data in the historical aerospace equipment data is used as a modal data, including: monitoring data, test data, log files, equipment pictures, product profile pictures, problem zeroing reports, equipment manuals, equipment electronic resumes, equipment operation audio and equipment operation video; the multi-modal large model adopts the Yi-VL model.

[0012] Based on the further improvement of the above method, the pre-trained multimodal large model is fine-tuned by introducing the multimodal correlation matrix into the Lora algorithm, including: decomposing the weight matrix of each linear layer in the multimodal large model into the sum of the products of the original weight matrix and three low-rank matrices; the three low-rank matrices include a dimensionality reduction matrix, a multimodal correlation matrix and a dimensionality increase matrix, and the rank of each matrix is ​​the number of modalities; during training, the original weight matrix and multimodal correlation matrix are solidified, and only the dimensionality reduction matrix and the dimensionality increase matrix are updated.

[0013] Based on the further improvement of the above method, the multimodal correlation matrix is ​​an asymmetric r×r matrix constructed with the number of modes r as the row and column dimensions and the unidirectional correlation between each mode calculated according to the user interaction data in a preset time period as the element value.

[0014] Based on the further improvement of the above method, the one-way correlation between the modes is calculated according to the user interaction data in the preset time period. The mode of the data to be analyzed input in the user interaction data is used as the input mode, and the mode of the selected search result is used as the output mode. The selection probability of each output mode corresponding to each input mode in the preset time period is calculated using the following formula:

[0015]

[0016] Among them, C i,j represents the unidirectional correlation between the i-th input mode and the j-th output mode, Indicates the number of selections of the jth output modal data corresponding to the ith input modality; Impression i Represents the number of retrievals for the i-th input modality data.

[0017] Based on the further improvement of the above method, the latest user interaction data is updated by recording the modality of the input data to be analyzed and the modality of the selected retrieval results each time the HNSW graph is retrieved; the multimodal correlation matrix is ​​updated according to the latest user interaction data to obtain the real-time correlation matrix, which is based on the latest user interaction data. On the basis of the multimodal correlation matrix, the corresponding one-way correlation is updated by increasing the number of selections of the corresponding input modality and the number of selections of the output modality, or the corresponding one-way correlation is updated by using the time attenuation factor while increasing the number of retrievals of the corresponding input modality to obtain the real-time correlation matrix.

[0018] Based on the further improvement of the above method, the corresponding one-way correlation is updated by the following formula using the time decay factor while increasing the number of selections of the corresponding input mode:

[0019]

[0020] Among them, C' i,j and C i,j They represent the unidirectional correlation between the i-th input mode and the j-th output mode after and before the update, respectively. α represents the time decay factor, α∈(0,1).

[0021] Based on the further improvement of the above method, the retrieval results are output using the HNSW graph according to the feature vector to be analyzed and the real-time correlation matrix. The feature vector to be analyzed is taken as the target node, and several candidate nodes with the smallest distance to the target node are searched layer by layer from the top layer of the HNSW graph. The original data corresponding to the several candidate nodes are taken as the retrieval results. The distance between the target node and the candidate node is obtained by multiplying the semantic similarity between the target node and the candidate node by the corresponding element value obtained from the real-time correlation matrix according to the mode of the target node and the candidate node as the weight.

[0022] Based on the further improvement of the above method, the data to be analyzed and the selected retrieval results are passed into the trained large language model, and the answer to the question is output according to the submitted question. The data to be analyzed and the selected retrieval results are used as reference data, and the answer to the question is obtained by setting multiple consecutive tasks and injecting instructions of different roles into each task respectively; the roles include: analysis role, review role and description role.

[0023] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0024] 1. Integrating professional knowledge in the aerospace field, building a data lake to integrate structured data and unstructured data effectively solves the problem of data silos, improves data utilization, and thus enhances overall analytical capabilities.

[0025] 2. By introducing an asymmetric modal correlation matrix in the process of Lora fine-tuning the multimodal large model and focusing on the correlation between different modes, the ability of the Lora algorithm to process multimodal data is enhanced, making the generated feature vector more in line with business needs.

[0026] 3. In the HNSW graph construction stage, an asymmetric modal correlation matrix is ​​introduced to determine the connection preference between nodes, which improves the accuracy of semantic connection between modalities. In the HNSW graph query stage, an asymmetric modal correlation matrix is ​​used to weight semantic similarity, improve the relevance of retrieval results, and optimize natural language query capabilities.

[0027] 4. Combine the search results and the data to be analyzed as the initial reference data for the large language model, and set different tasks and roles for the large language model to improve the accuracy of machine question answering. Apply natural language processing, semantic understanding and intelligent machine question answering to the quality analysis process of aerospace equipment, greatly improving the efficiency and accuracy of quality analysis.

[0028] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0030] Figure 1 This is a flow chart of a quality analysis method for aerospace equipment in Example 1 of the present invention;

[0031] Figure 2 Schematic diagram of data interaction in the quality analysis method in Example 1 of the present invention. DETAILED DESCRIPTION

[0032] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0033] A specific embodiment of the present invention discloses a quality analysis method for aerospace equipment, such as Figure 1 As shown, the following steps are included:

[0034] S1. Pass the data to be analyzed into the trained multimodal embedding model to obtain the feature vector to be analyzed; update the multimodal correlation matrix according to the latest user interaction data to obtain the real-time correlation matrix;

[0035] S2, outputting the search results using the HNSW graph according to the feature vector to be analyzed and the real-time correlation matrix;

[0036] S3. The data to be analyzed and the selected search results are passed into the trained large language model, and the answer to the question is output according to the submitted question; if the answer to the question is recognized, it is used as the quality analysis result; otherwise, the supplementary data is obtained and used as the new data to be analyzed, and steps S1-S3 are iteratively executed until the quality analysis result is obtained.

[0037] It should be noted that this embodiment uses steps S1-S3 to perform quality analysis on aerospace equipment, and uses real-time correlation matrix to weight semantic similarity in the HNSW graph query stage to improve the relevance of retrieval results and optimize natural language query capabilities; the retrieval results and the data to be analyzed are combined as initial reference data for the large language model, and the quality analysis results are continuously optimized through a feedback loop mechanism, greatly improving the accuracy of quality analysis.

[0038] It should be noted that in step S1, the multimodal large model adopts the Yi-VL (Yi Vision Language) model, which combines the understanding capabilities of vision (Image) and language (Language) to perform various cross-modal tasks.

[0039] In terms of architectural design, the Yi-VL model is based on the open source LLaVA architecture, including: Vit (Vision Transformer) module, Projection module and large-scale language model.

[0040] Among them, the ViT module is used for image encoding, and uses the open source OpenClip ViT-H / 14 model to initialize trainable parameters. By learning to extract features from large-scale "image-text" pairs, the model is able to process and understand images.

[0041] The Projection module is used to align image features with text features. It consists of a multilayer perceptron (MLP) with layer normalization, which enables the model to effectively fuse and process visual and text information, improving the accuracy of multimodal understanding and generation.

[0042] The large-scale language model uses Yi-34B-Chat and Yi-6B-Chat, and with the help of advanced natural language processing technology, the model has strong language understanding and the ability to generate coherent and relevant text.

[0043] It should be noted that the training of the multimodal large model is to divide the historical multimodal data into two parts, one for pre-training and the other for fine-tuning training. The historical multimodal data of aerospace equipment is divided into each business type as a modal data according to the business division, including but not limited to: monitoring data, test data, log files, equipment pictures, product profile pictures, problem zeroing reports, equipment manuals, equipment electronic records, equipment operation audio and equipment operation video.

[0044] like Figure 2 As shown in the figure, historical multimodal data is preprocessed, such as data cleaning, formatting and standardization, and the preprocessed historical data is stored in the data lake. The data lake is a data storage and management service platform, which includes relational databases (such as Oracle, MySQL, etc.), document databases (such as MongoDB, CouchDB, etc.), distributed file storage systems (such as HDFS, PVFS, PanFS, etc.) and graph databases (such as Neo4j, Cayley, GrapgDB, etc.). Apache Hive and Delta Lake tools are used to interact with data in the data lake. Apache Hive provides data warehouse capabilities and SQL query interfaces on top of Hadoop, and Delta Lake adds support for ACID transactions, further enhancing data consistency and reliability.

[0045] It should be emphasized that this embodiment mines the user's search preferences from the user interaction data and focuses on the different correlations between the modalities. For example, when a user inputs a device picture (device abnormality picture) to search, he or she most wants to obtain the device manual from the search results, and when inputting the device manual to search for related data, he or she most wants to obtain the corresponding device operation video from the search results. In other words, after entering the i-th modality, the user often chooses to browse the j-th modality data from the search results. In this embodiment, the correlation between the i-th modality and the j-th modality is considered to be high.

[0046] Therefore, the method of this embodiment also includes: collecting the search behavior data in the user search process as user interaction data, including: the data to be searched and its modality input by each user, the output search results and their modality, and the search results selected by the user (i.e., the search results of the detailed information browsed by the user) and their modality. The preset time period can be a certain period of time in history, or the trial operation stage of the method of this embodiment, the internal test user inputs the data to be searched according to the conventional business needs, obtains the search results by step S2, and selects and displays the detailed information from the search results according to the actual situation, so as to collect the user interaction data.

[0047] Furthermore, an asymmetric multimodal correlation matrix is ​​established based on the user interaction data in the preset time period. The multimodal correlation matrix is ​​an asymmetric r×r matrix constructed with the number of modes r as the row and column dimensions and the unidirectional correlation between the modes calculated based on the user interaction data in the preset time period as the element value.

[0048] Specifically, the one-way correlation between each modality is calculated based on the user interaction data within a preset time period, including: taking the modality of the data to be analyzed input in the user interaction data as the input modality, and the modality of the selected search results as the output modality, counting the number of searches for each input modality, and the number of selections of each output modality corresponding to each input modality; respectively calculating the ratio of the number of selections of each output modality to the number of selections of the corresponding input modality, and obtaining the selection probability of each output modality corresponding to each input modality within the preset time period in turn as the one-way correlation between the modalities. The corresponding formula is expressed as follows:

[0049]

[0050] Among them, C i,j It represents the unidirectional correlation between the i-th input mode and the j-th output mode, and also represents the selection probability of the j-th modal data corresponding to the i-th modal data. Indicates the number of selections of the jth output modal data corresponding to the ith input modality; Impression i Represents the number of retrievals for the i-th input modality data.

[0051] Within a preset time period, a multimodal correlation matrix is ​​constructed according to the calculation results of formula (1), wherein if the kth output modal data is not retrieved or selected based on the i-th input modal data, a unidirectional correlation of 0 is calculated with a selection number of 0, or is preset to a value close to 0.

[0052] Furthermore, after the multimodal large model is pre-trained, the pre-trained multimodal large model is fine-tuned again by introducing the multimodal correlation matrix into the Lora algorithm, so that the fine-tuned multimodal large model pays attention to the correlation between different modalities and extracts feature vectors that are more in line with business needs.

[0053] Specifically, the multimodal correlation matrix is ​​introduced in the Lora (Low-Rank Adaption) algorithm, which decomposes the weight matrix of each linear layer in the multimodal large model into the sum of the products of the original weight matrix and three low-rank matrices. The output h of each linear layer is expressed by the following formula (the bias vector is omitted for simplicity):

[0054] h l =Wl x l +ΔW l x l =W l x l +A l MB l x l Formula (2)

[0055] Among them, x l represents the input of the lth linear layer, W l represents the original lth weight matrix (i.e. the weight matrix after pre-training), d and k represent the matrix dimensions, ΔW l represents the weight update matrix of the lth linear layer during back propagation, which is decomposed into three low-rank matrices: A l represents the dimension reduction matrix of the lth linear layer, M represents the multimodal correlation matrix, B l represents the dimension-raising matrix of the l-th linear layer, The rank of these three low-rank matrices is the number of modes r, r<<min(d,k).

[0056] For the reduced dimension matrix A l Use a random Gaussian distribution to initialize the dimension-raising matrix B l Initialize to zero matrix to ensure that ΔW l = 0. When fine-tuning, the original weight matrix and multimodal correlation matrix are solidified, that is, the original weight matrix W l and the multimodal correlation matrix M do not participate in fine-tuning, only the low-rank matrix A l and B l Perform training updates, significantly reducing the number of parameters and computation required for model fine-tuning, and making the low-rank matrix A l and B l Learning the correlation probability between modalities enhances the Lora algorithm's ability to process multimodal data, making the generated feature vectors more in line with business needs. These feature vectors are projected into a common semantic space for subsequent retrieval.

[0057] After fine-tuning the multimodal embedding model, the multimodal data of aerospace equipment in the data lake are transferred into the multimodal embedding model in batches. Each linear layer is based on the original weight matrix W l , multimodal correlation matrix M and trained dimension reduction matrix A l And the dimension-raising matrix B l The corresponding output is obtained without adding additional reasoning time. Finally, the feature vector output by the multimodal embedding model is obtained, stored in the data lake, and associated with the corresponding original multimodal data.

[0058] In practical applications, the data to be analyzed is the data to be analyzed of aerospace equipment acquired by the user, such as one or more of monitoring data, equipment operation video, equipment operation audio, and pictures of a certain component of the equipment. In step S1, the data to be analyzed is passed into the trained multimodal embedding model to obtain the feature vector to be analyzed.

[0059] At the same time, in order to improve the accuracy of the search results and accurately adapt to the user's search needs, before executing step S2, the multimodal correlation matrix is ​​updated according to the latest user interaction data to obtain a real-time correlation matrix.

[0060] It should be noted that the latest user interaction data is updated by recording the modality of the input data to be analyzed and the modality of the selected search results each time the HNSW graph is searched. The real-time correlation matrix is ​​obtained by updating the multimodal correlation matrix according to the latest user interaction data. This is based on the latest user interaction data. On the basis of the multimodal correlation matrix, ① the corresponding one-way correlation is updated by increasing the number of selections of the corresponding input modality and the number of selections of the output modality, or ② the corresponding one-way correlation is updated by using the time decay factor while increasing the number of searches of the corresponding input modality, so as to obtain the real-time correlation matrix.

[0061] Specifically, ① the corresponding one-way correlation is updated by increasing the number of selections of the corresponding input modality and the number of selections of the output modality, including: obtaining the input modality according to the data to be retrieved each time; taking the output modality of the retrieval result corresponding to the input modality as the modality to be updated, and increasing the number of retrievals of the input modality by 1 according to formula (1); if the modality to be updated is selected, the number of selections of the corresponding output modality is increased by 1 according to formula (1), otherwise the number of increments remains unchanged. In other words, the one-way correlation of the output modality that has been retrieved and selected increases, the one-way correlation of the output modality that has been retrieved but not selected decreases, and the one-way correlation of the modality that has not been retrieved remains unchanged.

[0062] Specifically, while increasing the number of selections of the corresponding input mode, the corresponding one-way correlation is updated using the time decay factor through the following formula:

[0063]

[0064] Among them, C' i,j and C i,j They represent the one-way correlation between the i-th input mode and the j-th output mode after and before the update, respectively. α represents the time decay factor, α∈(0,1). i +1 means that each time the data to be retrieved is retrieved, the search count of the corresponding input mode increases by 1.

[0065] It should be noted that the time decay factor is obtained by inverse calculation based on the data impact factor and the time length, where the data impact factor f(t) is calculated by the following formula:

[0066] f(t)=(1-α) t Formula (4)

[0067] Based on formula (4), the appropriate time decay factor is calculated inversely according to the set time length and the expected data impact factor. For example, when t=30 days and the expected data impact factor f(t)≤0.05 (dropped below 5%), α=0.079.

[0068] In step S2, the HNSW (Hierarchical Navigable Small World) graph is a hierarchical graph structure, and the search efficiency is improved by performing an approximate nearest neighbor search on each layer. When constructing the HNSW graph, this embodiment uses the semantic similarity and multimodal correlation matrix based on the feature vector of the historical multimodal data in the data lake to determine the connection preference between nodes, thereby improving the accuracy of the semantic connection between modalities.

[0069] Specifically, constructing the HNSW graph includes the following steps: taking each feature vector as a node and the element value in the modal correlation matrix as the weight between the corresponding two nodes; when inserting the node to be inserted into its target layer in turn, by calculating the node distance between the node to be inserted and the inserted node in the target layer, the nearest neighbor inserted node is selected according to the preset number of neighbors to establish a connection with the node to be inserted; the distance between nodes is obtained according to the product of the semantic similarity between nodes and the weight; inserting the same node to be inserted in each layer below the target layer and establishing the same connection; when all nodes are inserted, the HNSW graph is constructed.

[0070] Since the structure of the HNSW graph is layered, each node to be inserted must be inserted into the target layer and each layer below it, making each layer of the network from top to bottom more and more dense, thereby achieving fast approximate search at the top layer while maintaining high search accuracy at the bottom layer. This embodiment introduces a multimodal correlation matrix during the graph construction process, so that nodes tend to establish connections with nodes with higher weights in the multimodal correlation matrix, while ensuring search efficiency and search accuracy, and improving search accuracy.

[0071] Specifically, the target layer L for inserting the node is obtained by the following formula:

[0072] L = -log(rnd) / log(m) Formula (5)

[0073] Among them, rnd represents a random number in the range of (0,1), and m represents the preset number of neighbors.

[0074] The distance between nodes is calculated using the following formula:

[0075] d(p,q)=Similarity(p,q)×M Type(p),Type(q) Formula (6)

[0076] Where b(p,q) represents the distance between node p and node q, Similarity(p,q) represents the semantic similarity between node p and node q, and M Type(p),Type(q) Represents the weight between node p and node q, that is, the corresponding element values ​​of the modal Type(p) of node p and the modal Type(q) of node q in the modal correlation matrix.

[0077] Preferably, the semantic similarity between nodes is calculated by cosine similarity.

[0078] When searching from the HNSW graph according to the feature vector to be analyzed and the real-time correlation matrix in step S2, the feature vector to be analyzed also obtains the weights between various modality types from the real-time correlation matrix according to its corresponding modality, and weights the semantic similarity to obtain better retrieval results.

[0079] Specifically, according to the feature vector to be analyzed and the multimodal correlation matrix, the retrieval results are output using the HNSW graph. The feature vector to be analyzed is taken as the target node, and several candidate nodes with the smallest distance to the target node are searched layer by layer from the top layer of the HNSW graph. The original data corresponding to the several candidate nodes are taken as the retrieval results. The distance between the target node and the candidate node is obtained by multiplying the semantic similarity between the target node and the candidate node by the corresponding element value obtained from the real-time correlation matrix according to the modality of the target node and the candidate node as the weight.

[0080] It should be noted that, according to the preset search quantity p, p candidate nodes with the smallest distance to the target node are selected from all candidate nodes, that is, p candidate feature vectors similar to the feature vector to be retrieved, and then the associated original multimodal data is obtained as the search result based on the candidate feature vector, and the search results are listed for users to browse selectively, and the user interaction data is updated according to the user's choice.

[0081] It is understandable that the real-time correlation matrix in this embodiment can be shared by all users and updated according to the interaction data of all users; or each user can correspond to a real-time correlation matrix and update it according to their own interaction data. In addition, the user can adjust the real-time correlation matrix in a specific task to increase the preference for retrieving corresponding modal data from the perspective of professional knowledge.

[0082] Preferably, since the real-time relevance matrix is ​​updated according to the latest user interaction data before retrieval, the multimodal relevance matrix can also be updated regularly according to the real-time relevance matrix.

[0083] Exemplarily, if all users share the same real-time correlation matrix, the latest real-time correlation matrix is ​​directly updated to the multimodal correlation matrix; if each user corresponds to a real-time correlation matrix, the updated multimodal correlation matrix is ​​obtained by averaging all real-time correlation matrices.

[0084] It can be understood that the method of this embodiment also includes: after the multimodal correlation matrix is ​​regularly updated, the multimodal embedding model is fine-tuned and updated according to the updated multimodal correlation matrix, and the feature vector of the multimodal data in the data lake is updated using the updated multimodal embedding model; based on the updated feature vector of the multimodal data and the multimodal correlation matrix, the HNSW graph is reconstructed.

[0085] This embodiment focuses on the user's retrieval preferences, constructs a multimodal correlation matrix through user interaction data, and introduces the multimodal correlation matrix into the multimodal model training, HNSW mapping and retrieval process, weights the semantic similarity, and improves the accuracy of semantic connections between modalities through real-time and periodic updates, improves the accuracy of retrieval results, and optimizes natural language query capabilities.

[0086] Finally, in step S3, the data to be analyzed and the selected search results are passed into the trained large language model, and the answer to the question is output according to the submitted question. The data to be analyzed and the selected search results are used as reference data, and the answer to the question is obtained by setting multiple consecutive tasks and injecting instructions of different roles into each task respectively; the roles include: analysis role, review role and description role.

[0087] It should be noted that the large language model selects a large pre-trained language model that performs outstandingly in logical reasoning and multi-step thinking tasks. Preferably, the chatGPT model is selected.

[0088] If the knowledge graph in the aerospace field has been built, the query interface of the knowledge graph can be defined by integrating the Langchain framework. In the process of generating answers with the large language model, the relevant aerospace knowledge in the knowledge graph can be queried and used to improve the natural language processing capabilities.

[0089] In this embodiment, multiple roles are set up for collaborative work in the large language model, including: the analysis role AnalyzeWorker, which is used to deeply analyze the questions input by the user and generate preliminary conclusions; the review role ReviewWorker, which is used to review the conclusions of the Analyze Worker and verify the professional knowledge to ensure the correctness and professionalism of the conclusions; the description role Describe Worker, which is used to explain the entire reasoning and interaction process, make the decision path of the model transparent, and increase the user's trust in the results.

[0090] In this step, instructions for different roles are injected into each task, where the instructions include but are not limited to: setting different roles for the large language model, explaining the data that each role needs to refer to and examples of question answers.

[0091] Specifically, this step performs the following tasks in sequence to output the answer to the question:

[0092] ① Select the search results from the search results output in step S2, and use them together with the data to be analyzed as the first reference data, and obtain the question submitted by the user, such as: Is the sensor monitoring data (data to be analyzed) normal? After injecting the instructions of the analysis role, a preliminary conclusion is obtained. Exemplarily, the instruction is: Generate a preliminary conclusion of the question {question} based on the first reference data with the analysis role; an example of a preliminary conclusion is: {demo1}. The information in the curly brackets should be replaced according to the actual data.

[0093] ②According to the questions and similarities submitted by the user, historical answers to similar questions and relevant knowledge in the knowledge graph are obtained as the second reference data, and the approval conclusion is obtained after the review role instruction is injected. For example, the instruction is: the review role reviews the preliminary conclusion based on the second reference data to generate an approval conclusion; an example of an approval conclusion is: {demo2}.

[0094] ③ After injecting the description role instruction, the final answer to the question is obtained. For example, the instruction is: use the description role to sort out the answer to the question containing the reasoning path based on the first reference data, the second reference data and the recognition conclusion, and list the first reference data for the user to browse; an example of the answer to the question is: {demo3}.

[0095] It should be noted that the quality analysis results of the answers to questions output by the large language model are continuously optimized using a feedback loop mechanism.

[0096] Specifically, it includes: if the answer to the question output by the large language model is not recognized, then obtaining supplementary data for the quality analysis result, such as newly acquired sensor monitoring data, as new data to be analyzed, using step S1 to update the feature vector to be analyzed, using step S2 to output a new retrieval result, and finally using step S3 to output a new answer to the question again. If it is still not recognized, then repeatedly feeding back the supplementary data until a recognized answer to the question is obtained as the quality analysis result, thereby improving the accuracy and reliability of the quality analysis result.

[0097] Compared with the prior art, the quality analysis method for aerospace equipment provided in this embodiment integrates professional knowledge in the aerospace field, integrates structured data and unstructured data by building a data lake, effectively solves the problem of data islands, improves data utilization, and thus improves the overall analysis capability; by introducing an asymmetric modal correlation matrix in the process of fine-tuning the multimodal large model in Lora, and paying attention to the correlation between different modalities, the ability of the Lora algorithm to process multimodal data is enhanced, so that the generated feature vector is more in line with business needs; in the HNSW graph construction stage, an asymmetric modal correlation matrix is ​​introduced to determine the connection preference between nodes, thereby improving the accuracy of semantic connections between modalities; in the HNSW graph query stage, an asymmetric modal correlation matrix is ​​used to weight semantic similarity, improve the relevance of retrieval results, and optimize natural language query capabilities; the retrieval results and the data to be analyzed are combined as the initial reference data of the large language model, and different tasks and different roles are set for the large language model to improve the accuracy of machine question answering, and the processing of natural language, semantic understanding and intelligent machine question answering are applied in the quality analysis process for aerospace equipment, greatly improving the efficiency and accuracy of quality analysis.

[0098] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0099] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A quality analysis method for aerospace equipment. It is characterized in that The following steps are involved: S1, passing the data to be analyzed into the trained multimodal embedding model to obtain the feature vector to be analyzed; The multimodal correlation matrix is ​​updated according to the latest user interaction data to obtain a real-time correlation matrix; The multimodal correlation matrix is ​​based on the number of modes r As the row and column dimensions, the one-way correlation between the modes is calculated based on the user interaction data in the preset time period as the asymmetric element value constructed r × r The one-way correlation between the modes is calculated based on the user interaction data within the preset time period by taking the mode of the data to be analyzed input in the user interaction data as the input mode, and the mode of the selected search result as the output mode, and calculating the selection probability of each output mode corresponding to each input mode within the preset time period using the following formula: ， in, Indicates i The first input mode and the j One-way correlation of the output modes, Indicates i The input mode corresponds to the j The number of selections for output modal data; Indicates i The number of retrievals for the input modal data; The latest user interaction data is updated by recording the modality of the input data to be analyzed and the modality of the selected search results each time the HNSW graph is used for retrieval; updating the multimodal correlation matrix according to the latest user interaction data to obtain the real-time correlation matrix is ​​based on the latest user interaction data, on the basis of the multimodal correlation matrix, by increasing the number of selections of the corresponding input modality and the number of selections of the output modality to update the corresponding unidirectional correlation, or, while increasing the number of retrievals of the corresponding input modality, using the time decay factor to update the corresponding unidirectional correlation by the following formula to obtain the real-time correlation matrix: ， in, and Respectively represent i The first input mode and the j The one-way correlation between the output modality before and after the update, represents the time decay factor, ; S2. Outputting the search results using the HNSW graph according to the feature vector to be analyzed and the real-time correlation matrix; S3. The data to be analyzed and the selected search results are passed into the trained large language model, and the answer to the question is output according to the submitted question; if the answer to the question is recognized, it is used as the quality analysis result; otherwise, the supplementary data is obtained and used as the new data to be analyzed, and steps S1-S3 are iteratively executed until the quality analysis result is obtained.

2. The quality analysis method for aerospace equipment according to claim 1, It is characterized in that The multimodal embedding model is based on historical aerospace equipment data and is obtained by introducing a multimodal correlation matrix into the Lora algorithm to fine-tune the pre-trained multimodal large model.

3. The quality analysis method for aerospace equipment according to claim 2, It is characterized in that Each type of business data in the historical aerospace equipment data is a kind of modal data, including: monitoring data, test data, log files, equipment pictures, product profile pictures, problem zeroing reports, equipment manuals, equipment electronic resumes, equipment operation audio and equipment operation video; the multimodal large model adopts the Yi-VL model.

4. The quality analysis method for aerospace equipment according to claim 2, It is characterized in that The method introduces a multimodal correlation matrix into the Lora algorithm to fine-tune the pre-trained multimodal large model, including: decomposing the weight matrix of each linear layer in the multimodal large model into the sum of the products of the original weight matrix and three low-rank matrices; the three low-rank matrices include a dimensionality reduction matrix, a multimodal correlation matrix and a dimensionality increase matrix, and the rank of each matrix is ​​the number of modes; during training, the original weight matrix and multimodal correlation matrix are solidified, and only the dimensionality reduction matrix and the dimensionality increase matrix are updated.

5. The quality analysis method for aerospace equipment according to claim 1, It is characterized in that Outputting the search result using the HNSW graph according to the feature vector to be analyzed and the real-time correlation matrix is ​​to use the feature vector to be analyzed as the target node, and search from the top layer of the HNSW graph downward layer by layer for several candidate nodes with the shortest distance from the target node, and the original data corresponding to the several candidate nodes are used as the search result; The distance between the target node and the candidate node is obtained by obtaining the corresponding element value from the real-time correlation matrix as a weight according to the modes of the target node and the candidate node, and multiplying it by the semantic similarity between the target node and the candidate node.

6. The quality analysis method for aerospace equipment according to claim 1, It is characterized in that The data to be analyzed and the selected search results are passed into the trained large language model, and the answer to the question is output according to the submitted question. The data to be analyzed and the selected search results are used as reference data, and the answer to the question is obtained by setting multiple consecutive tasks and injecting instructions of different roles into each task respectively; the roles include: analysis role, review role and description role.

Images