Training and information retrieval method of multilingual embedding model and apparatus thereof

Abstract

The application provides a multilingual embedding model training and information retrieval method and device, belonging to the technical field of data information processing, comprising: acquiring training batches corresponding to first and second language data respectively and inputting the multilingual embedding model to obtain corresponding first and second loss values; updating model parameters of the multilingual embedding model based on the loss values; dynamically adjusting sampling probabilities of the first and second language data according to difference characteristics of the first and second loss values; extracting the next training batch according to the adjusted sampling probabilities and iteratively circulating until the model meets a preset convergence condition. The multilingual embedding model training method provided by the application can monitor convergence gradient differences of different languages in real time through microscale, adaptively regulate data ratios of different languages online, solve long-tail data representation underfitting and cross-language training imbalance problems caused by traditional fixed ratios, and improve cross-language accurate representation capability of mixed knowledge bases.

Description

Technical Field

[0001] This invention relates to the field of data information processing technology, and in particular to a method and apparatus for training and information retrieval of a multilingual embedding model. Background Technology

[0002] With the deepening application of natural language processing technology in various professional fields, in the face of multilingual hybrid knowledge bases containing massive heterogeneous data, retrieval enhancement generation systems need to rely on high-quality multilingual embedding models to accurately represent the semantics of documents in different languages, so as to meet the cross-language accurate information retrieval needs in complex professional scenarios.

[0003] When training embedding models for multi-source knowledge bases, existing technologies typically employ a pre-defined fixed data ratio strategy, such as maintaining a fixed sampling ratio for Chinese and foreign language data to construct training batches; or, after a set interval of training steps, using independent offline validation sets to evaluate the performance metrics of different language tasks, thereby making static, coarse-grained adjustments to the corpus distribution ratio in subsequent training stages.

[0004] However, in real-world multilingual training scenarios, there are significant objective differences in the semantic space formation and convergence speed of different language corpora. Existing training schemes based on fixed ratios cannot adapt to these differences, easily leading to both underfitting of long-tail specialized vocabulary in specific languages ​​and overfitting of high-frequency corpora, resulting in a severe imbalance in the model's cross-lingual representation capabilities. Furthermore, schemes relying on independent validation sets for offline evaluation and adjustment not only introduce enormous additional computational overhead, but the severe lag in evaluation results also prevents the model from responding promptly to differences in training micro-dynamics, making it difficult to guarantee synchronous and balanced convergence of multilingual joint training. Summary of the Invention

[0005] This invention provides a training and information retrieval method and apparatus for a multilingual embedding model, which addresses the shortcomings of existing technologies that use fixed data ratios or rely on offline verification in multilingual model training. These shortcomings result in an inability to promptly account for the differences in learning difficulty among different languages, leading to insufficient training of specialized vocabulary in specific languages, an imbalance in cross-language retrieval capabilities, and significant computational overhead from offline evaluation. The invention achieves dynamic adjustment of the sampling ratio of data for each language based on real-time loss feedback during training, effectively balancing the training process for each language without increasing additional computational burden, and significantly improving the model's accurate retrieval capability for mixed professional knowledge bases.

[0006] This invention provides a training method for a multilingual embedding model, comprising iteratively executing the following steps until a preset training termination condition is met, thereby obtaining a multilingual embedding model: Obtain the first language data batch and the second language data batch for the current training iteration step; Obtain a first loss value obtained by performing processing on the first language data batch based on the multilingual embedding model to be trained, and a second loss value obtained by performing processing on the second language data batch based on the multilingual embedding model; Based on the first loss value and the second loss value, update the model parameters of the multilingual embedding model and determine the sampling scale factor; Construct a new first language data batch and a new second language data batch for the next training iteration, wherein the data volume ratio between the new first language data batch and the new second language data batch is determined based on the sampling ratio factor; Update the next training iteration step to the current training iteration step.

[0007] According to the training method of a multilingual embedding model provided by the present invention, the sampling scaling factor is determined based on the following steps: Eliminate the dimensional difference between the first loss value and the second loss value, and obtain the first normalized loss corresponding to the first loss value and the second normalized loss corresponding to the second loss value respectively; Calculate the numerical ratio of the first normalized loss to the second normalized loss, and determine the numerical ratio as the sampling scale factor.

[0008] According to a training method for a multilingual embedding model provided by the present invention, determining the data volume ratio between the new first language data batch and the new second language data batch based on the sampling ratio factor includes: According to the sampling ratio factor, determine the target sampling weight ratio of the new first language data and the new second language data in the next training iteration step; Based on the target sampling weight ratio, the data volume is allocated between the new first language data batch and the new second language data batch to obtain the data volume allocation ratio.

[0009] According to a training method for a multilingual embedding model provided by the present invention, the model parameters of the multilingual embedding model are updated based on the first loss value and the second loss value, including: Based on the first loss value, calculate the first gradient vector of the first language data with respect to the shared parameters in the multilingual embedding model; Based on the second loss value, calculate the second gradient vector of the second language data with respect to the shared parameters in the multilingual embedding model; Store the first gradient vector and the second gradient vector in a preset cache area; Based on the first gradient vector and the second gradient vector in the cache, a joint update process is performed on the shared parameters of the multilingual embedding model.

[0010] According to the present invention, a training method for a multilingual embedding model is provided, wherein the multilingual embedding model includes a dual-tower network model based on the Transformer architecture, and the dual-tower network model is used to independently encode data in different languages.

[0011] According to a training method for a multilingual embedding model provided by the present invention, the first language data includes aviation maintenance experience data in the first language, and the second language data includes original equipment manufacturer (OEM) technical specifications data in the second language. The first language is Chinese, and the aviation maintenance experience data includes at least one of emergency response plan data, historical fault maintenance log data, and minimum equipment list data; The second language is English, and the original equipment manufacturer (OEM) technical specifications data includes fault isolation manual data.

[0012] The present invention also provides an information retrieval method, comprising obtaining a target query request and performing the following retrieval evaluation iterative steps until a preset process exit condition is met: The target query request is input into the multilingual embedding model to obtain the query feature vector output by the multilingual embedding model; The query feature vector is used to perform matching and retrieval in the knowledge base to obtain the corresponding recall fragment set; The recalled fragment set is input into a large language model for semantic evaluation, and the evaluation result output by the large language model is obtained. If the evaluation result indicates that the recalled fragment set meets the preset semantic consistency condition, the retrieval result is output based on the recalled fragment set, and the process exit condition is triggered; If the evaluation result indicates that the recalled fragment set does not meet the semantic consistency condition and has not reached the preset fault tolerance termination condition, the target query request is input into the query rewriting model, the updated target query request output by the query rewriting model is obtained, and the updated target query request is used as a new target query request to enter the next round of retrieval evaluation iteration steps. If the evaluation result indicates that the recalled fragment set does not meet the semantic consistency condition and the fault tolerance termination condition has been reached, a fallback response result is generated based on the acquired recalled fragment set and the process exit condition is triggered. The multilingual embedding model is trained using any of the multilingual embedding model training methods described above.

[0013] According to an information retrieval method provided by the present invention, the query rewriting model is obtained based on the following training steps: Obtain the original query sample set, which contains multiple basic query samples; According to the preset sample expansion strategy, data transformation processing is performed on the basic query samples to obtain the corresponding derived query samples; the sample expansion strategy includes at least one of the following: keyword masking strategy, multi-query intent fusion strategy, and context information simplification strategy. The derived query samples are mapped and combined with the corresponding basic query samples to construct the target fine-tuning dataset; The query rewriting model is obtained by performing supervised fine-tuning training on the preset base language model using the target fine-tuning dataset.

[0014] According to an information retrieval method provided by the present invention, the step of performing data transformation processing on the basic query samples according to a preset sample expansion strategy to obtain corresponding derived query samples includes at least one of the following: When the sample expansion strategy includes the keyword masking strategy, a masking replacement process is performed on the target keywords in the basic query sample to obtain the corresponding derived query sample; When the sample expansion strategy includes the multi-query intent fusion strategy, at least two basic query samples are obtained, and the at least two basic query samples are subjected to text fusion processing to obtain the corresponding derived query samples. When the sample expansion strategy includes the context information simplification strategy, some context text fragments in the basic query sample are deleted to obtain the corresponding derived query sample.

[0015] According to an information retrieval method provided by the present invention, the step of performing a matching retrieval in a knowledge base using the query feature vector to obtain a corresponding recall fragment set includes: Determine the target recall number corresponding to the current retrieval evaluation iteration step; The query feature vector is used to perform matching and retrieval in the knowledge base to obtain the recall fragment set according to the target recall quantity; Wherein, the number of target recalls in the next round of retrieval evaluation iteration step in which the updated target query request is located is greater than the number of target recalls in the current retrieval evaluation iteration step.

[0016] According to an information retrieval method provided by the present invention, the step of outputting retrieval results based on the recalled fragment set when the evaluation result indicates that the recalled fragment set meets a preset semantic consistency condition includes: Determine the semantic correlation between each recalled fragment in the recalled fragment set and the target query request; Based on the semantic relevance, each recalled fragment in the recalled fragment set is sorted to obtain the target sorted fragment set; The target sorted fragment set is output as the retrieval result.

[0017] According to an information retrieval method provided by the present invention, the step of inputting the recalled fragment set into a large language model for semantic evaluation and obtaining the evaluation result output by the large language model includes: The retrieved fragment set and the target query request are combined and input into the large language model for logical analysis to obtain the semantic relevance score and logical sufficiency score output by the large language model. If the semantic relevance score reaches a first preset threshold and the logical sufficiency score reaches a second preset threshold, the evaluation result is determined to be that the recalled fragment set satisfies the preset semantic consistency condition. If the semantic relevance score does not reach the first preset threshold and / or the logical sufficiency score does not reach the second preset threshold, the evaluation result is determined to be that the recalled fragment set does not meet the semantic consistency condition.

[0018] The present invention also provides a training device for a multilingual embedding model, comprising the following modules: The corpus acquisition module is used to acquire the first language data batch and the second language data batch in the current training iteration step; The loss calculation module is used to obtain a first loss value obtained by performing processing on the first language data batch based on the multilingual embedding model to be trained, and a second loss value obtained by performing processing on the second language data batch based on the multilingual embedding model. The parameter update module is used to update the model parameters of the multilingual embedding model and determine the sampling scale factor based on the first loss value and the second loss value. The sampling control module is used to dynamically adjust the sampling probability for the first language data and the second language data based on the difference characteristics between the first loss value and the second loss value; The iteration control module is used to construct a new first language data batch and a new second language data batch for the next training iteration step, update the next training iteration step to the current training iteration step, and return to the corpus acquisition module to perform the steps of acquiring the first language data batch and the second language data batch for the current training iteration step, until the preset training termination condition is met.

[0019] The present invention also provides an information retrieval device, comprising the following modules: The request retrieval module is used to retrieve the target query request and trigger the execution of the retrieval evaluation iteration operation of the corresponding module below until the preset process exit condition is met; The matching and retrieval module is used to input the target query request into the multilingual embedding model, obtain the query feature vector output by the multilingual embedding model, and perform matching and retrieval in the knowledge base based on the query feature vector to obtain the corresponding recall fragment set; wherein, the multilingual embedding model is trained by any of the training methods of the multilingual embedding model described above. The semantic evaluation module is used to input the recalled fragment set into the large language model for semantic evaluation and obtain the evaluation result output by the large language model. The first output module is used to output retrieval results based on the recalled fragment set and trigger the process exit condition when the evaluation result indicates that the recalled fragment set meets the preset semantic consistency condition. The intent rewriting module is used to input the target query request into the query rewriting model when the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and the preset fault tolerance termination condition is not met, obtain the updated target query request output by the query rewriting model, and use the updated target query request as a new target query request to enter the next round of retrieval evaluation iteration steps. The second output module is used to generate a fallback response result based on the acquired recall fragment set and trigger the process exit condition when the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and the fault tolerance termination condition has been reached.

[0020] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a training method for a multilingual embedding model as described above, or implements an information retrieval method as described above.

[0021] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a multilingual embedding model as described above, or implements an information retrieval method as described above.

[0022] The multilingual embedding model training and information retrieval method and apparatus provided by this invention monitors the convergence gradient differences of different languages ​​in real time at the micro scale, and adaptively adjusts the data ratio of each language online. This solves the problems of underfitting long-tail data representation and cross-language training imbalance caused by traditional fixed ratios, and improves the cross-language accurate representation capability of the hybrid knowledge base. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the training method for the multilingual embedding model provided by the present invention.

[0025] Figure 2 This is a schematic diagram of the training process of a multilingual embedding model based on parallel Chinese and English tasks provided by the present invention.

[0026] Figure 3 This is one of the flowcharts illustrating the information retrieval method provided by the present invention.

[0027] Figure 4 This is a schematic diagram of the training process of the query rewriting model provided by the present invention.

[0028] Figure 5 This is the second flowchart of the information retrieval method provided by the present invention.

[0029] Figure 6 This is a schematic diagram of the structure of the training device for the multilingual embedding model provided by the present invention.

[0030] Figure 7 This is a schematic diagram of the information retrieval device provided by the present invention.

[0031] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0033] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0034] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0035] To more clearly illustrate the training and information retrieval method and apparatus for the multilingual embedding model provided by this invention, a detailed description will be provided below in conjunction with specific embodiments and application scenarios. It should be noted beforehand that the training and information retrieval method for the multilingual embedding model provided by this invention is essentially a low-level computer data processing and intelligent retrieval algorithm architecture, which can be widely applied to various natural language processing, intelligent question answering, and retrieval-augmented generation (RAG) systems containing heterogeneous multilingual data.

[0036] To facilitate a full understanding of the core concept of this invention and the technical challenges it addresses, the following will use aircraft maintenance as a representative and complex scenario to elaborate on the application background, limitations of existing technologies, and improvement ideas of this invention. It is understood that this specific application scenario of aircraft maintenance is merely an example to illustrate the solution of this invention and does not constitute any limitation on the broader scope of protection defined by this invention.

[0037] In one specific embodiment, the multilingual embedding model training and information retrieval method and apparatus provided by the present invention can achieve accurate semantic matching of a Chinese-English hybrid knowledge base through multilingual adaptive representation learning and retrieval quality closed-loop optimization. Its core value lies in: improving the recall accuracy of original Chinese and English technical documents such as Fault Isolation Manuals (FIMs) in aviation maintenance scenarios, ensuring the reliability of knowledge supply of the RAG system in complex technical terminology environments, effectively solving the problems of representation imbalance and semantic drift in cross-language knowledge base retrieval, and providing efficient and accurate knowledge support for core tasks such as fault diagnosis and manual retrieval by aircraft maintenance engineers.

[0038] In practical applications such as the construction of smart civil aviation, specialized fields often generate large-scale, heterogeneous, multilingual hybrid knowledge assets. For example, in the context of aircraft maintenance, the knowledge base includes emergency response plans, historical fault maintenance logs, and Minimum Equipment Lists (MELs) in the first language (e.g., Chinese), as well as FIMs (Firmware Instructions) composed of original manufacturer technical documents in the second language (e.g., English). Professionals, such as maintenance engineers, need to perform frequent cross-referencing and logical comparisons between records and troubleshooting logic in different languages ​​when making decisions. This requires the system to not only handle unstructured experience assets but also align with concise original manufacturer technical specifications containing numerous compound terms and abbreviations, placing extremely high demands on the accuracy of knowledge retrieval and cross-language adaptability.

[0039] At the technological evolution level, RAG technology has been widely used in intelligent question-answering assistants, with its core links covering embedding model vectorization, near nearest neighbor retrieval, fine ranking, and large language model generation. However, when this general technical architecture is directly applied to the aviation maintenance field, two deep-seated structural defects are exposed: First, the multilingual representation capabilities are severely unbalanced. The existing fixed-ratio training strategy results in the embedding model's semantic modeling ability for English professional long-tail terms, such as aircraft system component codes and fault codes, being significantly weaker than that for Chinese. This leads to a biased recall of FIM manuals and fails to meet the accurate retrieval needs of original English manufacturer documents. Second, existing data retrieval methods lack fault tolerance and self-healing mechanisms. Faced with non-standard colloquial queries or vague fault descriptions input by engineers, they lack introspective quality assessment and dynamic correction capabilities based on large language models, easily falling into a vicious cycle of "recalling faults with faults" and making it difficult to support highly reliable troubleshooting decisions.

[0040] A deeper challenge lies in the highly specialized nature of aviation maintenance knowledge and the strong logical connections between data. The information resources required for maintenance decisions are scattered across three heterogeneous dimensions: first, unstructured English original equipment manufacturer (OEM) technical specifications (including FIM manuals), which are highly condensed in language, contain a large number of compound nouns and abbreviations, and have strict causal logic and procedural constraints; second, unstructured Chinese experience data, namely the hidden fault modes and handling techniques contained in historical troubleshooting records; and third, structured airborne data, such as Quick Access Recorder (QAR) decoding parameters. This distribution pattern requires maintenance engineers to frequently switch between manual reference systems, maintenance logbooks, and airborne data analysis platforms when tracing faults, manually aligning English troubleshooting logic with Chinese experience data. This fragmented workflow not only significantly prolongs troubleshooting time but also easily induces serious operational risks, such as damage to aircraft components due to misoperation caused by failure to accurately recall key troubleshooting steps in the FIM manual, or the adoption of incorrect handling plans due to ignoring specific configuration differences information in Chinese historical logs.

[0041] Currently, retrieval enhancement generation systems for multi-source knowledge bases widely adopt dual-tower models based on the Transformer architecture for embedding representation, supplemented by vector retrieval engines to achieve semantic-level matching. In the embedding model training stage, existing technologies generally employ pre-defined fixed data sampling ratios, such as constructing training batch data for bilingual languages ​​at a 1:1 or 2:1 ratio, and optimizing the model by comparing learning losses, such as InfoNCE Loss. In the retrieval recall stage, a strategy of directly outputting a fixed number (e.g., Top-K) of candidate fragments in a single vector retrieval is often used. In the query preprocessing stage, some systems introduce general large language models to rewrite the original query using prompt word engineering, and standardize the input text through methods such as semantic dereference and spoken-to-written language conversion, thereby improving the matching degree between the query and the document.

[0042] In practical applications, existing technologies have various variations, each attempting to alleviate problems such as multilingual training imbalance, but with limited effectiveness. For example, embedding models using a fixed-ratio training scheme are pre-trained on a general corpus and then fine-tuned to maintain a superficial language balance by uniformly sampling multilingual data. Some research systems attempt to introduce offline ratio adjustment strategies based on validation sets, where a fixed training step size is used at each interval to evaluate the loss or accuracy of each language task on an independent validation set, and the language distribution ratio of the training data in the next stage is manually adjusted based on the relative differences in the evaluation results. For multi-task learning scenarios, existing solutions often employ a linear weighting method with fixed loss weights, adjusting the hyperparameters of each language task to balance the gradient contribution of the total loss function, attempting to alleviate the multilingual training imbalance problem.

[0043] However, while the application of the aforementioned general-purpose large language model and vector retrieval technology in vertical domain cross-language retrieval enhancement generation systems has improved basic semantic understanding capabilities, when applied to scenarios such as aircraft maintenance that have both high professional barriers and strong logical constraints, the existing technical architecture still has significant rigidity defects and insufficient adaptability. Mainstream embedding training and recall strategies generally face key technical bottlenecks such as multilingual representation imbalance, inability to self-heal retrieval failures, and weak generalization ability of professional query rewriting. These defects directly result in the retrieval performance on the original English manuals being unable to support highly reliable troubleshooting assistance decisions. While existing solutions attempt to balance multilingual training through fixed ratios or offline adjustments, they are essentially static or semi-static coarse-grained interventions that fail to perceive the micro-dynamic differences in convergence speeds between different languages ​​during training. In aviation maintenance corpora, the semantic space of high-frequency Chinese words can be formed quickly, while low-frequency English compound terms, such as learning "Integrated Drive Generator Voltage Phase Imbalance," require more continuous gradient injection. The adjustment lag caused by fixed ratios and offline evaluation often results in a situation where English representations are underfitted and Chinese representations are overfitted. Furthermore, repeated validation set evaluations bring significant additional computational overhead, making it impossible to achieve efficient and dynamic training optimization.

[0044] Specifically, the recall mechanisms of existing retrieval enhancement generation systems are mostly executed in an "open-loop" manner, meaning that the results are unconditionally sent to the large model after the retrieval is performed, lacking an effective quality verification process. In the aviation maintenance scenario, the queries of aircraft maintenance engineers are often accompanied by problems such as ambiguous intent (e.g., "the engine is shaking a bit") and non-standard terminology (e.g., using colloquial terms instead of standard manual names). Existing information retrieval methods lack a recall quality self-assessment mechanism based on the reasoning ability of large language models, and cannot intelligently identify whether the current recall results contain effective troubleshooting logic, nor can they trigger targeted correction branches based on the assessment conclusions, such as in-depth intent clarification or rewriting of professional query requests.

[0045] Furthermore, the strategy of a fixed recall number fails when faced with complex fault descriptions due to insufficient coverage, and when faced with simple queries, it introduces a large number of noise fragments that interfere with the generation of large models, which seriously reduces the reliability and operating efficiency of the system.

[0046] While general-purpose language model rewriting schemes possess some semantic smoothing capabilities, they face severe domain knowledge drift in the aviation maintenance field. They struggle to understand the specific system connections behind abbreviations like "ENG 1 FAIL" and "PACK OFF," often resulting in rewritings that lack professional rigor and may even introduce factual errors. Rule-based rewriting methods, on the other hand, are impractical due to the exponentially complex combination of failure modes and component relationships in the aviation domain. A more critical challenge lies in the lack of a systematic approach to constructing training data for rewriting models specific to this aviation maintenance scenario. There is a lack of automated methods to extract mappings between vague colloquial descriptions and standard maintenance actions from massive amounts of unstructured troubleshooting records. This makes fine-tuning of professional query rewriting models difficult, hindering the accurate standardization of non-standard query requests.

[0047] Based on the analysis of the underlying defects of current information retrieval methods, such as the imbalance of multilingual representations, the lack of fault tolerance in open-loop retrieval, and the lack of data support for professional rewriting, this invention proposes an adaptive embedding model training and intelligent recall end-to-end closed-loop optimization method. By constructing a multilingual dynamic sampling mechanism based on real-time loss feedback, an introspective recall evaluation branch correction architecture based on a large language model, and a corresponding data expansion fine-tuning strategy, this method fundamentally improves the cross-language accurate semantic matching capability and system robustness of the multilingual hybrid knowledge base.

[0048] This invention provides a training method for a multilingual embedding model, which can be executed by an electronic device with data processing capabilities and computing resources. This electronic device can be a standalone server, a distributed cloud computing cluster, or an intelligent computing platform containing a central processing unit (CPU) and a graphics processing unit (GPU). In the following description of embodiments, for the sake of brevity and consistency, the server will be used as the primary execution entity to elaborate on each execution step of the training method.

[0049] Figure 1 This is a flowchart illustrating the training method for the multilingual embedding model provided by the present invention, as shown below. Figure 1 As shown, this method primarily relies on iterative loop processing logic to gradually improve the cross-lingual representation capability of the multilingual embedding model through continuous batch data input and parameter optimization. Specifically, the method includes executing training iteration steps until a preset training termination condition is met. In each training iteration step, the following processing steps are executed: Step 1: Obtain the first language data batch and the second language data batch for the current training iteration step.

[0050] The current training iteration step refers to the current forward and backward propagation cycle of the multilingual embedding model during continuous optimization. The first language data batch and the second language data batch represent structured or unstructured text sets extracted from corpora with two different language features, respectively.

[0051] As a specific data acquisition implementation method, data can be loaded in batches by establishing a direct read channel with the local storage array, or it can be pulled in real time in the form of a data stream by periodically calling the application programming interface (API) of an external knowledge database through a network interface, or it can be sampled in parallel using a data loader in a distributed file system. To understand this more intuitively, taking the complex field of aircraft maintenance as an example, the first language data batch can be Chinese troubleshooting record text extracted from emergency response plan databases, historical fault maintenance log databases, and minimum equipment lists (MELs) containing a large amount of Chinese context; while the second language data batch can be English technical specification fragments containing a large number of compound terms and component codes extracted from original manufacturer technical documents such as FIM manuals, which are entirely in English.

[0052] Step 2: Obtain the first loss value obtained by performing processing on the first language data batch based on the multilingual embedding model to be trained, and the second loss value obtained by performing processing on the second language data batch based on the multilingual embedding model.

[0053] Specifically, forward propagation processing is performed on the first language data batch and the second language data batch based on the multilingual embedding model to be trained, thereby obtaining the corresponding first loss value and second loss value.

[0054] The multilingual embedding model involved in this invention refers to a deep neural network architecture capable of mapping natural language texts in different languages ​​to a unified high-dimensional continuous vector space. Its specific network structure design is diverse; for example, it can employ a single-tower shared model based on a self-attention mechanism, or a multi-tower model architecture containing independent elements with interactive mechanisms. Forward propagation here refers to the process of inputting the extracted first language data batch and the second language data batch into the multilingual embedding model to be trained, followed by feature extraction and nonlinear transformation at each network layer to calculate feature vectors or prediction results.

[0055] The first and second loss values ​​are used to quantitatively characterize the degree of error between the output feature distribution of the multilingual embedding model under training and the true learning target in the current state. In the specific calculation path, depending on the task type, cross-entropy loss, InfoNCELoss, or Mean Squared Error (MSE) can be used for precise calculation. The calculated first and second loss values ​​directly reflect the semantic modeling difficulty and feature mastery level of the multilingual embedding model under training at the current training iteration step.

[0056] Step 3: Based on the first loss value and the second loss value, update the model parameters of the multilingual embedding model and determine the sampling scale factor.

[0057] During the parameter update phase, the backpropagation algorithm can be used to calculate the gradient of the loss function with respect to each learnable weight. In combination with optimizers such as stochastic gradient descent (SGD) and Adam optimizer, the matrix parameters inside the multilingual embedding model can be iteratively updated along the inverse direction of the gradient in order to reduce the prediction error overall.

[0058] Simultaneously, this invention compares and correlates the obtained first and second loss values ​​in terms of numerical dimensions, thereby calculating a sampling ratio factor. This sampling ratio factor can be used to quantitatively reflect the relative learning difficulty gap between the two language tasks in the current model state. For example, if the calculation finds that the first loss value is significantly greater than the second loss value, it indicates that the feature extraction capability for the first language in the current model state is currently lagging behind that for the second language. In this case, the calculated sampling ratio factor will be shifted accordingly, indicating that more attention needs to be paid to the first language in subsequent data input stages.

[0059] Step 4: Construct a new first language data batch and a new second language data batch for the next training iteration. The data volume ratio between the new first language data batch and the new second language data batch is determined based on the sampling ratio factor.

[0060] In one specific implementation, there are several methods for the server to dynamically adjust the data volume allocation. For example, a soft allocation change can be achieved by dynamically modifying the probability sampling weight distribution of each language queue in the underlying data loading pool. Alternatively, under the premise of limiting the fixed total batch size in a single training cycle, the exact number of samples can be calculated directly based on the sampling ratio factor, and then a truncated hard combination can be performed. Through the above reconstruction actions, the language tasks with larger current loss values, i.e., those that are more difficult to learn and converge more slowly, can obtain higher data sampling priority and a larger sample coverage in the next training iteration, thereby driving the multilingual embedding model to be trained to inject more gradient update magnitude into the weaker languages ​​in the next training iteration.

[0061] Step 5: Update the next training iteration step to the current training iteration step.

[0062] Before the end of each training iteration step, the present invention updates the next training iteration step to the current training iteration step and determines whether the current running state meets the preset training end conditions.

[0063] The training termination condition primarily serves as a trigger to automatically terminate the entire model training process. The specific trigger threshold can be varied based on actual computing resources and accuracy requirements. For example, the training termination condition can be set when the number of global model training epochs reaches a pre-configured maximum limit; or it can be set by monitoring the change in the total loss value over several adjacent training iterations, triggering an early stop mechanism if the change consistently falls below a preset minimum value; or it can be used as a termination signal when the accuracy, recall, and other evaluation metrics of the trained multilingual embedding model on an independent test evaluation set reach the expected passing standard.

[0064] If the current state does not meet the preset training termination condition, the control execution logic continues... Figure 1 The feedback loop shown returns to step 1 and starts a new round of looping; if the judgment result is that the preset training termination condition has been met, the loop will automatically exit, and the final training converged multilingual embedding model will be output.

[0065] The training method for a multilingual embedding model provided in this invention obtains the loss value generated by the multilingual embedding model performing forward propagation on the first and second language data in real time during the training iteration process, and dynamically calculates the sampling ratio factor based on the loss value. This adjusts the sampling ratio of different language data in the next training iteration step. By directly utilizing the loss feedback of the model on the features of specific language data under the current parameter state, the proportion of high-loss languages ​​in the data input of the next training batch is dynamically increased. This allows for targeted improvement of the gradient update strength of the weaker language in the subsequent backpropagation stage. It eliminates the need for periodic lag evaluation using additional offline validation sets, effectively avoiding the problems of inconsistent convergence speeds of different languages ​​and underfitting of long-tail data representation caused by using static fixed data ratios. This achieves balanced training and synchronous convergence of features of different languages, thereby improving the cross-language unified representation capability of the multilingual embedding model for hybrid knowledge bases.

[0066] Based on the above embodiments, as an optional embodiment, a further implementation method for calculating the sampling scale factor is provided, which mainly includes, but is not limited to, the following steps: Step 1: Eliminate the dimensional difference between the first loss value and the second loss value, and obtain the first normalized loss corresponding to the first loss value and the second normalized loss corresponding to the second loss value respectively; Step 2: Calculate the numerical ratio of the first normalized loss to the second normalized loss, and determine the numerical ratio as the sampling scale factor.

[0067] Specifically, due to the inherent objective differences in text length, vocabulary size, and data distribution between the first language data batch and the second language data batch, even if the same loss function is used, such as the information noise comparison evaluation loss, the first loss value and the second loss value calculated for the output of the multilingual embedding model may not be on the same order of magnitude in absolute numerical range.

[0068] To ensure direct comparability between the first and second loss values, this embodiment normalizes the initial loss values. Specifically, the initial first and second loss values ​​generated by the multilingual embedding model in the first training iteration can be extracted as a benchmark. The first loss value of the current training iteration is divided by this initial first loss value to obtain the first normalized loss value; similarly, the second loss value of the current training iteration is divided by this initial second loss value to obtain the second normalized loss value. Alternatively, a sliding window mechanism can be used to calculate the moving average of the first loss values ​​over multiple consecutive training iterations, and this average is used as a benchmark for division normalization, thereby converting the loss value into a dimensionless relative rate of change as the normalized loss.

[0069] After obtaining the first and second normalized loss values, this embodiment further performs a ratio calculation. Referring to the aforementioned aircraft maintenance scenario example, assuming the first language data is Chinese corpus from historical troubleshooting records and the second language data is English corpus from the FIM manual, the calculated first normalized loss is denoted as L. zh Let the calculated second normalized loss be denoted as L. en Then, the sampling scale factor K can be determined based on the numerical ratio between the two: K=L zh / L en .

[0070] The calculated sampling ratio factor K quantitatively reflects the learning difficulty of the multilingual embedding model under the current model state for the Chinese task relative to the English task: if the sampling ratio factor K is greater than 1, it indicates that the learning on the Chinese corpus side is lagging behind, and the multilingual embedding model has more difficulty in extracting features from the first language data; if the sampling ratio factor K is less than 1, it indicates that the learning on the English corpus side is lagging behind; if the sampling ratio factor K is close to 1, it indicates that the learning progress on both sides is relatively balanced.

[0071] This embodiment obtains a first normalized loss value and a second normalized loss value by performing normalization processing on the first loss value and the second normalized loss value respectively, and determines the ratio of the first normalized loss value to the second normalized loss value as the sampling scale factor. The normalization processing directly eliminates the absolute dimensional differences between different language tasks caused by the different underlying data distribution characteristics. This allows the calculated sampling scale factor to objectively and accurately reflect the relative learning difficulty of the multilingual embedding model for different language features under the current parameter state. This ensures the accuracy of adjusting the data volume ratio based on the sampling scale factor in the next training iteration step, avoids misjudgment of sampling probability caused by inconsistency in the absolute scale of the values, and thus improves the scientific nature of the multilingual embedding model training state evaluation and the stability of training convergence.

[0072] Based on the above embodiments, as an optional embodiment, the present invention further provides a specific implementation method for determining the data volume ratio between the new first language data batch and the new second language data batch based on the sampling ratio factor, which mainly includes, but is not limited to: Step 1: Determine the target sampling weight ratio of the new first language data and the new second language data in the next training iteration step according to the sampling ratio factor.

[0073] Step 2: Based on the target sampling weight ratio, allocate the data volume between the new first language data batch and the new second language data batch to obtain the data volume allocation ratio.

[0074] The target sampling weight ratio is mainly used to quantify the relative size of the first language data batch and the second language data batch in the new data batch to be constructed.

[0075] In specific implementations, there are various methods for converting the calculated sampling ratio factor into the target sampling weight ratio.

[0076] As a feasible implementation method, the value of the sampling ratio factor can be directly set as the weight of the first language data, and the constant 1 can be set as the weight of the second language data, thereby forming a direct ratio mapping.

[0077] As another feasible implementation, in order to prevent extreme data sampling imbalance caused by short-term drastic fluctuations in the loss value during a single training iteration, a temperature hyperparameter or sliding smoothing factor can be introduced to perform exponential smoothing or logarithmic scaling on the initial sampling ratio factor before calculating the target sampling weight ratio.

[0078] In the context of aircraft maintenance, assuming that the sampling ratio factor calculated between the first language data and the second language data is 1.5, a direct mapping method can be used to determine that the target sampling weight ratio of the new first language data and the new second language data in the next training iteration step is 1.5:1. If converted to a standard probability distribution, this corresponds to a weight share of 60% and 40%.

[0079] Furthermore, the implementation methods for obtaining the final data volume allocation based on the target sampling weight ratio in this embodiment are diverse.

[0080] As one implementation method, if the total global data capacity of a single training iteration of the multilingual embedding model is predetermined, for example, if the global batch size is fixed at 1000 samples, a multiplication truncation operation can be performed according to the target sampling weight ratio of 1.5:1 determined above. The amount of data required to acquire a new first language data batch is 600 samples, and the amount of data required to acquire a new second language data batch is 400 samples. This forms a precise data volume ratio at the integer number level, and quota extraction is performed in the respective language databases accordingly.

[0081] As another implementation, in a streaming training architecture that does not set a strict absolute upper limit on the number of data entries in a single batch, the target sampling weight ratio can be directly configured as the polling probability parameter of the underlying distributed data loading channel (DataLoader). For example, by using a weighted random sampling algorithm, the frequency of sampling from the first language database and the second language database per unit time can be controlled, so that the first language data and the second language data that actually enter the next training iteration step meet the expected data volume ratio in terms of data volume.

[0082] This embodiment transforms the sampling ratio factor calculated based on loss feedback into a clear target sampling weight ratio. Based on this target sampling weight ratio, it precisely allocates data volume between new first-language data batches and new second-language data batches to obtain the data volume ratio. This implements dynamic feedback control of the micro-convergence gradient at the underlying data loading control logic. This allocation mechanism allows specific language tasks with higher feature extraction difficulty and slower convergence under the current parameter state to be accurately and directly allocated more data volume in the next training batch. Furthermore, without introducing additional offline evaluation computational overhead, it spontaneously guides data flow towards weaker languages, effectively eliminating the defects of underfitting long-tail specialized data representation and overfitting high-frequency data caused by static fixed allocation ratios.

[0083] Based on the above, as an optional embodiment, updating the model parameters of the multilingual embedding model based on the first loss value and the second loss value may include, but is not limited to: Step 1: Based on the first loss value, calculate the first gradient vector of the first language data for the shared parameters in the multilingual embedding model.

[0084] Step 2: Based on the second loss value, calculate the second gradient vector of the second language data with respect to the shared parameters in the multilingual embedding model.

[0085] Step 3: Store the first gradient vector and the second gradient vector in a preset cache.

[0086] Step 4: Based on the first gradient vector and the second gradient vector in the cache, perform joint update processing on the shared parameters of the multilingual embedding model.

[0087] In the network architecture design of multilingual embedding models, shared parameters can refer to the underlying network weight matrices or bias terms that are used together by the first language data and the second language data during feature extraction or semantic mapping, regardless of the input language. Examples include shared word embedding projection layers or underlying self-attention multi-head weights.

[0088] In one specific implementation, the backpropagation algorithm can be used to independently calculate the partial derivative matrix of the first loss value with respect to the shared parameters to form the first gradient vector, and to calculate the partial derivative matrix of the second loss value with respect to the same set of shared parameters to form the second gradient vector. This step can be executed concurrently in different threads within the same computing node, or asynchronously in different computing units of a distributed computing cluster.

[0089] In step 3, this embodiment stores the first gradient vector and the second gradient vector into a preset cache. The preset cache refers to a physical or virtual storage address pre-allocated by the server in the GPU's video memory or the CPU's high-speed system memory specifically for temporarily storing tensor data. In traditional single-task training mode, gradients are usually used immediately for parameter updates once calculated. However, in the multi-task parallel representation learning scenario of this embodiment, to prevent gradient updates of the first language data from directly changing the network state and thus polluting the gradient calculation benchmark of the second language data, this embodiment suspends the calculated first and second gradient vectors and temporarily stores them in the preset cache, awaiting subsequent synchronization scheduling.

[0090] Finally, the server can retrieve these two sets of gradient vectors from the preset cache at any time and perform a joint update on the shared parameters of the multilingual embedding model. In specific implementations, there are various feasible mathematical methods for performing the joint update process. For example, element-wise matrix addition can be performed directly on the first and second gradient vectors to obtain a global accumulated gradient vector, which can then be input into an optimizer, such as a stochastic gradient descent optimizer (SGD) or an Adam optimizer, to drive the optimizer to update the shared parameters along the direction of overall error descent.

[0091] As another feasible implementation, the first gradient vector and the second gradient vector can be multiplied by the corresponding weighting coefficients according to the sampling ratio factor calculated in the foregoing embodiments or the actual number of samples in each language batch, and a weighted average calculation can be performed. Then, the fused weighted gradient vector can be used to perform a single, synchronous model weight iteration on the shared parameters.

[0092] The training method for the multilingual embedding model provided in this embodiment calculates the first and second gradient vectors for the shared parameters in the multilingual embedding model independently based on the first and second loss values, respectively. These two sets of gradient vectors are first stored in a preset cache, and then extracted and jointly updated. This achieves gradient synchronization and fusion optimization for multilingual representation tasks at the underlying computational logic level. It effectively avoids the problem of mutual interference and numerical oscillation of gradient directions caused by the alternating serial updates of model parameters from different languages ​​when processing mixed corpora. This ensures that the underlying shared parameters can simultaneously absorb the feature distribution patterns of the first and second language data, thereby improving the feature alignment accuracy of the multilingual embedding model in a unified cross-lingual semantic space.

[0093] Based on the above embodiments, as an optional embodiment, the present invention provides a specific underlying network topology and feature extraction architecture for a multilingual embedding model. Specifically, the multilingual embedding model includes a dual-tower network model based on the Transformer architecture, which is used to independently encode data in different languages.

[0094] In a specific implementation, the Transformer-based architecture refers to the fact that the underlying construction of the multilingual embedding model used in this invention relies on a self-attention mechanism and a feedforward neural network component. This architecture can break the distance limitations of traditional sequence models when processing long texts, accurately capturing long-distance contextual dependencies between various words in the sequence through global computation. The dual-tower network model refers to the fact that the multilingual embedding model is structurally divided into two parallel and independent encoding branches. These two encoding branches maintain physical or logical isolation in the data processing flow, and are used to receive and process text features of different language dimensions from the input, respectively.

[0095] The dual-tower network model used in this embodiment can independently encode data in different languages, and there are multiple feasible architectural configurations for its implementation.

[0096] As an alternative implementation, this dual-tower network model can adopt a fully parameter-shared Siamese Network structure, meaning that the network weight matrices of the two towers are completely identical. The server feeds the first language data and the second language data into the two weight-shared towers for independent forward propagation. During the intermediate hidden layer stages, there is no cross-attention interaction between the two, and they rely entirely on the shared underlying pre-trained multilingual feature space to extract their respective semantic vectors. Finally, the top layer outputs the dimensionality-reduced high-dimensional dense features.

[0097] As another implementation, to better accommodate the significant differences in grammatical structure and specialized vocabulary between the first and second languages, the dual-tower network model can also employ an asymmetric structure with partial parameter decoupling. For example, two completely independent low-level vocabulary lists and embedding mapping layers can be configured for the first and second language data respectively to specifically capture word roots or abbreviation features of specific languages, while parameter sharing is restored in the high-level Transformer encoding blocks. Alternatively, two completely independent Transformer encoders can be used for mapping, and spatial distance constraints are applied to the output vectors of the two only at the level of calculating the final loss function of the first and second loss values.

[0098] The multilingual embedding model training method provided in this embodiment constructs the multilingual embedding model as a dual-tower network model based on the Transformer architecture. This dual-tower network model performs independent encoding on data from different languages, thus eliminating potential vocabulary conflicts and grammatical feature interference that may occur in the early network layers when extracting data from different languages. The powerful contextual feature extraction capability of the Transformer architecture ensures the accuracy of semantic capture of complex professional texts, while the dual-tower independent encoding mechanism allows text fragments from different languages ​​to be mapped to a unified continuous vector space. This not only provides the network architecture foundation for accurate semantic alignment across languages ​​but also enables offline vectorization caching of a large monolingual knowledge base in subsequent online retrieval applications, thereby greatly improving the system's online retrieval response efficiency and feature matching accuracy.

[0099] Figure 2 This is a schematic diagram of the training process of a multilingual embedding model based on parallel Chinese and English tasks provided by the present invention, as shown below. Figure 2 As shown, the data in the first language mainly includes aviation maintenance experience data in the first language, and the data in the second language mainly includes original equipment manufacturer (OEM) technical specifications data in the second language. The first language is Chinese, and the aviation maintenance experience data includes at least one of emergency response plan data, historical fault maintenance log data, and MEL data; The second language is English, and the original equipment manufacturer (OEM) technical specifications data includes FIM manual data.

[0100] This embodiment applies the training method of the multilingual embedding model provided in the previous embodiment to a specific professional scenario for aircraft maintenance, in order to solve the highly representative problem of joint representation of complex cross-linguistic terms in this field.

[0101] In one specific implementation, the acquisition and construction of first language data and second language data can be achieved by connecting and integrating the dispersed heterogeneous knowledge assets in the field of aircraft maintenance.

[0102] On the one hand, for the Chinese context, structured or unstructured first-language data can be extracted from the underlying maintenance records and experience base. This Chinese aviation maintenance experience data contains information on hidden fault patterns and specific configuration differences accumulated by aircraft maintenance engineers in the process of troubleshooting. Its characteristics include a large number of high-frequency Chinese business terms, and its semantic space can usually be clustered relatively quickly during model training.

[0103] On the other hand, for English-language contexts, second-language data can be imported from the original equipment manufacturer (OEM) system. These English OEM technical specifications, especially the FIM manuals, have a highly condensed professional language style, containing a large number of system component codes, compound nouns, and complex English abbreviations, such as low-frequency English professional long-tail terms like "Integrated Drive Generator VoltagePhase Imbalance." Moreover, the syntax has strict causal logic and troubleshooting step constraints, making feature extraction and semantic modeling significantly more difficult than Chinese log text, requiring more continuous and intensive gradient injection to achieve convergence.

[0104] Combination Figure 2 The underlying data flow logic shown in this invention, and the specific implementation steps for training iteration in the scenario of aircraft maintenance, may include: First, the operation to start the training iteration is initiated, and the first language data batch and the second language data batch of the current training iteration step are obtained in parallel through the underlying configured data loader.

[0105] Subsequently, by inputting the Chinese batch and the English batch into the multilingual embedding model to be trained, and performing forward propagation calculations within the model, the first loss value for the Chinese feature space and the second loss value for the English feature space are output respectively.

[0106] Next, the first and second loss values ​​are correlated to obtain the sampling scaling factor K. This sampling scaling factor K truly reflects the relative learning difficulty gap between the Chinese experience logs and the English original manual in the current model state of the multilingual embedding model.

[0107] Finally, the target sampling weight ratio for the next training iteration is dynamically adjusted to a K:1 correspondence, and this target sampling weight ratio is used as a real-time feedback signal, directly fed back to the data loader through the control loop. Upon receiving this real-time feedback, the data loader automatically reconstructs the sampling probabilities of the next batch of Chinese and English data streams based on the K:1 ratio. If, during a specific training phase, the second loss value is significantly larger due to a severe lag in learning compound terms and fault codes in the English FIM manual, the real-time feedback K:1 weight ratio will directly drive the data loader to significantly increase the number of samples in the English batch during the next extraction, thereby injecting more gradient update magnitude into the English feature space during backpropagation.

[0108] The training method for the multilingual embedding model provided in this embodiment utilizes a dynamic data allocation to spontaneously increase the data input of weaker technical terms. This gives higher sampling priority to tasks with larger loss values, driving the multilingual embedding model to correct biases in these weaker tasks, ultimately achieving dynamic equilibrium convergence of Chinese and English representation capabilities. This effectively eliminates the structural representation imbalance caused by differences in vocabulary frequency and logical density in cross-language knowledge bases within the specific domain of aircraft maintenance, enabling the multilingual embedding model to simultaneously and accurately map both historical experience and skills in Chinese and rigorous troubleshooting logic in English.

[0109] Figure 3 This is one of the flowcharts illustrating the information retrieval method provided by the present invention, such as... Figure 3 As shown, after the server receives the target query request, it begins to execute the retrieval evaluation iteration steps until the preset process exit conditions are met.

[0110] The target query request can refer to the received raw input text expressing the user's specific intention to obtain information. Taking an aircraft maintenance scenario as an example, this target query request might be a text instruction entered by a maintenance engineer at the troubleshooting site, containing colloquialisms, non-standard technical terminology, or ambiguous intent, such as "the engine is shaking a bit" or a query using colloquial terms instead of standard parts names from the manual. The retrieval evaluation iteration step here refers to the cyclical processing logic that involves multiple trials, evaluations, and corrections before the final response is output. The preset process exit condition is the trigger condition that controls the safe termination of this cyclical processing logic; it can be set to successfully obtain a high-confidence response or reach a preset processing limit.

[0111] In each iteration of the retrieval and evaluation process, this embodiment inputs the target query request into a multilingual embedding model to obtain the query feature vector output by the model. The multilingual embedding model is trained using the training method provided in any of the preceding embodiments. During actual operation, the network parameters of this multilingual embedding model are used to perform forward propagation encoding on the target query request, mapping it into a high-dimensional query feature vector.

[0112] The knowledge base mentioned in this embodiment can be a professional domain corpus that has been pre-processed by text segmentation and vectorization, such as the aforementioned Chinese-English mixed maintenance manual, troubleshooting log, etc.

[0113] There are various specific implementation methods for using the query feature vector to perform matching retrieval in the knowledge base. As a feasible implementation method, commonly used graph-based hierarchical navigation small worlds (HNSW) or inverted file with product quantization (IVF-PQ) and other approximate nearest neighbor (ANN) retrieval algorithms can be used to calculate the cosine similarity or inner product distance between the query feature vector and the vectors of each knowledge segment in the knowledge base, and extract several text segments with the highest similarity to form the recall segment set.

[0114] As another implementation, a dual-path recall mechanism that combines dense vector retrieval with keyword-based sparse retrieval can be used to expand the coverage of candidate texts and merge them to generate the recall fragment set.

[0115] like Figure 3 As shown, this embodiment further inputs the recalled fragment set into a large language model for semantic evaluation, obtaining the evaluation result output by the large language model. The large language model used in this step is obtained after supervised training of the base large language model and serves as an introspective quality verification center. By constructing a specific prompt word project, the obtained recalled fragment set and the current target query request can be input into the large language model, driving the large language model to utilize its parameter knowledge and logical reasoning capabilities to perform quality diagnosis on whether the recalled content truly answers the user's query intent. The evaluation result is a structured judgment conclusion output by the large language model, used to indicate the current recall quality status.

[0116] After obtaining the evaluation results output by the large language model, the subsequent execution logic in this embodiment is divided into three differentiated processing branches: As the first branch, if the evaluation result indicates that the recalled fragment set meets the preset semantic consistency condition, the retrieval result will be output based on the recalled fragment set and the process exit condition will be triggered.

[0117] The preset semantic consistency condition refers to the benchmark used to measure whether the recalled content meets the standard. For example, the large language model determines that the recalled content clearly contains valid troubleshooting logic and is highly consistent with the query intent. When the semantic consistency condition is met, it means that the retrieval is successful, and the recalled fragment set can be directly integrated or formatted as the retrieval result and output to the user. At the same time, the process exit condition is triggered to end the current retrieval task.

[0118] In the second branch, if the evaluation result indicates that the recalled fragment set does not meet the above semantic consistency condition and does not reach the preset fault tolerance termination condition, this embodiment will input the target query request into the query rewriting model to obtain the updated target query request output by the query rewriting model, and use the updated target query request as a new target query request to enter the next round of retrieval evaluation iteration steps.

[0119] The preset fault-tolerant termination condition can be a security defense mechanism to prevent the system from falling into an infinite retry loop. For example, it can be set as a maximum number of rewrite retries (e.g., 3 times) or a maximum allowed time threshold for a single retrieval task. In this case, it indicates that the current recalled content is of poor quality, for example, because the input from the maintenance engineer is too colloquial, and the knowledge base cannot directly match the corresponding professional terminology. However, the system still has a trial-and-error allowance. At this time, the server calls a dedicated query rewriting model to standardize or enhance the original non-standard target query request. For example, the colloquial "start switch inoperative" is rewritten to "Engine Start Switch -Switch Does Not Hold In GRD Position" which conforms to the maintenance manual specifications. Subsequently, the updated target query request can be used to follow... Figure 3 The feedback loop shown returns to the starting point to perform the next round of feature vector extraction and matching retrieval.

[0120] In the final branch, if the evaluation result indicates that the recalled fragment set does not meet the semantic consistency condition and the fault tolerance termination condition has been reached, a fallback response result will be generated based on the acquired recalled fragment set, triggering the process exit condition. In this branch, since multiple rewrites and re-recalls have failed to obtain the desired high-quality content, and the fault tolerance limit has been reached (e.g., retries exhausted), to ensure online response rate and avoid a vicious cycle of "recalling the wrong thing with the wrong thing," retries will stop. The system will directly utilize the currently collected set of recalled fragments with the highest relative relevance to perform a safety summary using a large language model, generating a fallback response result containing disclaimers or a general direction for investigation, outputting it to the user, and triggering the process exit condition to forcibly terminate the process.

[0121] This embodiment adds an introspective semantic evaluation layer driven by a large language model between the traditional retrieval and generation processes. Combined with preset semantic consistency and fault-tolerant termination conditions, it constructs a three-branch processing mechanism: direct output, rewrite retry, and fallback protection. This overcomes the shortcomings of traditional retrieval-enhanced generation architectures, such as open-loop retrieval and unconditional trust in recall results. When faced with queries with ambiguous intent or non-standard terminology, the large language model can detect logical failures in the recalled fragments, spontaneously triggering a query rewrite model to standardize and reconstruct the original request. After multiple unsuccessful retries, a fallback mechanism is used for circuit breaking, effectively preventing the cascading amplification of errors caused by retrieval failures.

[0122] Figure 4 This is a schematic diagram of the training process of the query rewriting model provided by the present invention, as follows: Figure 4 As shown, this embodiment provides a training method for complex professional scenarios, specifically for the query rewriting model that plays a core role in the fault-tolerant retry branch. This method addresses the issues of knowledge drift and weak generalization ability in general query rewriting models within specialized domains, and includes, but is not limited to: Step 1: Obtain the original query sample set, which contains multiple basic query samples.

[0123] The original query sample set refers to a standardized set of reference benchmark data prepared for training rewriting capabilities. The basic query samples in it represent standardized and regulated search expressions that conform to specific professional fields, such as aircraft maintenance.

[0124] There are several ways to obtain the basic query samples. As one optional implementation, a general large language model can be used to perform distillation extraction, automatically extracting standard questions or search queries that conform to the troubleshooting logic from the structured content of professional technical documents. For example, standard questions or search queries that conform to the troubleshooting logic can be extracted from the structured content such as document titles, chapter names, and fault description logic in FIM manuals in the aviation field.

[0125] To ensure the professionalism and accuracy of these basic query samples, a manual review process can be introduced. The extracted query statements are then handed over to business experts for random sampling and verification, filtering out unqualified samples. Finally, the qualified samples that pass the verification are accumulated to form the original query sample set.

[0126] Step 2: According to the preset sample expansion strategy, perform data transformation processing on the basic query samples to obtain the corresponding derived query samples. The sample expansion strategy includes at least one of the following: keyword masking strategy, multi-query intent fusion strategy, and context information simplification strategy.

[0127] The pre-defined sample expansion strategy can be a series of data degradation and variation rules designed to simulate non-standard user input in real business scenarios. Derived query samples are query texts generated after processing by the sample expansion strategy, simulating real-world spoken or non-standard user input.

[0128] In practical implementation, multiple sample expansion strategies can be executed in parallel: The first sample expansion strategy can be a keyword masking strategy, which uses a random algorithm to randomly mask and replace the core entities or key terms in the basic query samples, thereby constructing derived query samples with ambiguous intents; the second sample expansion strategy can be a multi-query intent fusion strategy, which calls a general large language model to merge two or more basic query samples with certain logical connections into a complex, colloquial long sentence containing multiple intents; the third sample expansion strategy can be a context information simplification strategy, for example, by deleting preconditions, model backgrounds, and other modifiers in the basic query samples, constructing extremely short query samples lacking context information, such as retaining only a short fault code.

[0129] Step 3: Map and combine the derived query samples with the corresponding basic query samples to construct the target fine-tuning dataset.

[0130] The target fine-tuning dataset refers to a supervised learning alignment dataset consisting of non-standard inputs to standard outputs.

[0131] In a specific implementation, the derived query samples of simulated non-standard queries generated in the preceding steps are used as input features of the model, and the standard and normative basic query samples of the same origin are used as output labels of the model, forming a one-to-one mapping training data pair.

[0132] To prevent the introduction of factual errors during the degradation generation process, this embodiment can also perform a dual quality inspection mechanism before constructing the final target fine-tuning dataset. This includes: firstly, using a preset automatic quality inspection model to batch filter the semantic consistency and syntactic correctness of the data pairs, and then manually reviewing a portion of the samples to ensure that the final mapping combination pairs meet professional standards.

[0133] Step 4: Use the target fine-tuning dataset to perform supervised fine-tuning training on the preset base language model to obtain the query rewriting model.

[0134] The pre-set base language model can be a general-purpose deep neural network model that has been pre-trained on a large-scale unsupervised corpus and has basic natural language generation capabilities, such as a base language model with tens of billions of parameters.

[0135] In a specific implementation, a Supervised Fine-Tuning (SFT) algorithm can be used. Derived query samples from the target fine-tuning dataset are input into the base language model, and the cross-entropy loss between its output sequence and the corresponding base query samples is calculated. This loss is then used to update the network parameters of the base language model through backpropagation. After multiple rounds of iterative fine-tuning, the base language model converges and is finalized, outputting the final query rewriting model.

[0136] The information retrieval method provided in this embodiment extracts standard basic query samples from professional documents and uses targeted preset sample expansion strategies such as keyword masking, multi-query intent fusion, and context simplification to reverse-generate derived query samples simulating real-world non-standard scenarios. These samples are then combined to construct a target fine-tuning dataset for supervised fine-tuning of the base language model, thus filling the gap in high-quality rewritten alignment data in professional domains at the underlying data construction logic level. This query rewriting model training method enables the generated query rewriting model to directly learn the specific mapping relationships of fuzzy intent completion, multi-intent splitting, and context expansion at the parameter level. When invoked in the information retrieval process, it can accurately identify fragmented and colloquial non-standard queries from business personnel and precisely transform them into standardized retrieval expressions containing professional terminology, effectively overcoming the problem of domain knowledge drift that easily occurs in general rewriting models in professional scenarios.

[0137] As an optional embodiment, the step of performing data transformation processing on the basic query samples according to a preset sample expansion strategy to obtain corresponding derived query samples includes at least one of the following: In the first scenario, if the sample expansion strategy includes the keyword masking strategy, a masking replacement process is performed on the target keywords in the basic query sample to obtain the corresponding derived query sample.

[0138] In practice, named entity recognition (NER) algorithms or part-of-speech tagging tools in natural language processing can be used to automatically scan and identify proper nouns, core verbs, or specific component codes in the basic query samples as target keywords. Subsequently, these target keywords can be replaced using specific placeholders, general pronouns, or pre-collected fuzzy colloquial vocabulary.

[0139] In the context of aircraft maintenance, assuming the basic query sample obtained is the standardized expression "Engine Start Switch - Switch Does Not Hold In GRDPosition", the server identifies the target keywords "engine start switch" and "ground position" and performs mask replacement processing on them to generate derivative query samples with ambiguous intent, such as "that start switch is not holding" or "a certain switch is not in the specified position".

[0140] In the second scenario, if the sample expansion strategy includes the multi-query intent fusion strategy, at least two basic query samples are obtained, and the at least two basic query samples are subjected to text fusion processing to obtain the corresponding derived query samples.

[0141] In a specific implementation, two or more basic query samples with potential business connections can be randomly selected from the original query sample set, or extracted based on the underlying fault association graph. Subsequently, through a specific concatenation algorithm, or by calling a large language model and supplementing it with specific fusion prompts, these originally independent query samples are rewritten into a continuous composite text containing multiple rows of intentional graphs.

[0142] For example, by obtaining the basic query sample A "Engine 1 failure isolation (ENG 1 FAIL)" and the basic query sample B "Air conditioning component tripped and troubleshooting (PACK OFF)", we can generate a derived query sample that matches the anxious tone of the maintenance personnel through text fusion processing: "Engine 1 has failed, and the air conditioning component has also tripped. What should we do?"

[0143] In another scenario, where the sample expansion strategy includes the context information simplification strategy, some context text fragments in the basic query sample are deleted to obtain the corresponding derived query sample.

[0144] In practical implementation, extraction rules such as syntactic dependency parsing trees or regular expressions can be used to identify and remove modifiers such as preconditions, machine background, and system status descriptions from the basic query sample, retaining only the core fault code or the most basic nouns. For example, for the basic query sample "Fault isolation and troubleshooting steps triggered under cold weather conditions (Maintenance Message 31-00242 - FaultIsolation)," a deletion operation can be directly performed to remove contextual text fragments such as weather conditions and action descriptions, obtaining the extremely fragmented derived query sample "31-00242".

[0145] This embodiment employs sample expansion strategies such as keyword masking, multi-query intent fusion, and contextual information simplification. By intentionally injecting semantically missing noise and structural variation features into the model input space during deep learning training data construction, it reverses the degradation of pure, standardized retrieval queries into fuzzy, complex, and incomplete text data. This forces the base language model to learn the inverse mapping of this degradation process during supervised fine-tuning. This effectively solves the problem of highly non-standardized long-tail data in real-world applications, resulting in a query rewriting model with powerful information completion, multi-intent decoupling, and logical reconstruction capabilities. It can not only accurately restore missing context but also break down complex composite queries into atomic-level standard retrieval instructions, thus ensuring high recall and matching accuracy even under extremely non-standardized input conditions.

[0146] Based on the above embodiments, as an optional embodiment, the step of using the query feature vector to perform matching retrieval in the knowledge base to obtain the corresponding recall fragment set includes: Determine the target recall number corresponding to the current retrieval evaluation iteration step; The query feature vector is used to perform matching and retrieval in the knowledge base to obtain the recall fragment set according to the target recall quantity; Wherein, the number of target recalls in the next round of retrieval evaluation iteration step in which the updated target query request is located is greater than the number of target recalls in the current retrieval evaluation iteration step.

[0147] Figure 5 This is the second flowchart illustrating the information retrieval method provided by the present invention, as shown below. Figure 5 As shown, this embodiment provides a retrieval execution process based on adaptive recall and ranking optimization, which specifically includes: Step 1: Determine the target recall for the current retrieval evaluation iteration step. Target recall refers to the maximum number of candidate knowledge fragments that the retrieval engine can extract and return from the vast knowledge base in a single query action, denoted as the Top-K value. Combined with... Figure 5 As shown, when a user enters a target query request and triggers the first round of coarse-ranking recall, a relatively small value is configured as the target recall quantity to ensure response efficiency and reduce interference from irrelevant information, for example, it is set to Top-8. The first round of coarse-ranking recall refers to the first iteration of the current retrieval evaluation.

[0148] Step 2: Use the query feature vector to perform matching retrieval in the knowledge base to obtain the recall fragment set according to the target recall number. In a specific implementation, a high-dimensional vector indexing engine, such as a graph-based approximate nearest neighbor search framework, can be used to calculate the distance between the current query feature vector and the document vectors in the knowledge base. Subsequently, based on the determined target recall number (e.g., 8), the fragments are sorted and truncated, and the top 8 closest text fragments are extracted and combined to obtain the current recall fragment set.

[0149] Step 3: According to the system architecture design of this embodiment, the number of target recalls in the next round of retrieval evaluation iteration step where the updated target query request is located is greater than the number of target recalls in the current retrieval evaluation iteration step.

[0150] In a specific operational scenario, if the Top-8 fragments obtained in the initial recall are determined to be unsatisfactory after semantic consistency evaluation by the large language model (i.e., no semantically consistent knowledge fragments exist), the target query request will be marked as a hard case, and the query rewriting model will be invoked. Relying on the capabilities injected during the training phase, the query rewriting model will perform operations such as clarifying fuzzy intents, splitting multiple intents, and filling in context on the original request.

[0151] After the rewriting is complete, the updated target query request can be used to restart the next round of retrieval evaluation iteration steps, i.e., perform secondary recall retrieval. At this time, this embodiment dynamically adjusts the underlying retrieval strategy to expand the target recall quantity in this round. For example... Figure 5 As shown, the target recall number for the next round can be significantly expanded from the original Top-8 to Top-20.

[0152] Subsequently, the optimized and rewritten target query request can be used to perform a re-retrieval. Since the target recall size is relaxed, a wider set of recalled fragments will be obtained according to the Top-20 scale, and submitted to the large language model to perform a secondary evaluation of the recalled correct content.

[0153] The information retrieval method provided in this embodiment dynamically configures an increasing target recall quantity in different retrieval evaluation iterations. At the underlying retrieval execution logic, it constructs an adaptive capacity control method that achieves accurate initial screening in the first round and expands capacity in the second round. This effectively overcomes the static configuration defects of traditional information retrieval methods that use a fixed recall quantity. Using a smaller recall quantity in the first round of retrieval directly avoids introducing a large number of noise fragments due to forcibly recalling too much content when facing simple, routine queries, thus ensuring the purity of information during the generation stage and the system's operational efficiency. In the retry stage for handling non-standard queries or complex and difficult cases, by significantly expanding the target recall quantity, combined with the rewritten enhanced query, it provides a broader underlying knowledge coverage for complex intent expressions, thereby greatly increasing the probability of retrieving long-tail troubleshooting steps in aviation original documents.

[0154] Based on the above embodiments, as an optional embodiment, when the evaluation result indicates that the recalled fragment set meets a preset semantic consistency condition, the step of outputting retrieval results based on the recalled fragment set specifically includes, but is not limited to: Determine the semantic correlation between each recalled fragment in the recalled fragment set and the target query request; Based on the semantic relevance, each recalled fragment in the recalled fragment set is sorted to obtain the target sorted fragment set; The target sorted fragment set is output as the retrieval result.

[0155] Semantic relevance is used to characterize the degree to which knowledge fragments match the user's query intent within deep logic and contextual context. In the initial coarse-grained recall stage, spatial cosine distance can be calculated using independent feature vectors output by the dual-tower network model. While this spatial cosine distance can quickly reflect basic similarity, it often overlooks complex contextual interactions. Therefore, this embodiment can further employ various other methods to determine this semantic relevance.

[0156] As a feasible implementation method, a pre-deployed cross-encoder fine ranking model can be invoked. The target query request and each recalled fragment in the recall fragment set are concatenated into the fine ranking model and then input into the fine ranking model. The model performs deep word-level feature interaction calculation through a self-attention mechanism and outputs a precise score between 0 and 1 as the semantic relevance.

[0157] As another feasible implementation method, the logical relevance probability distribution features or scoring indicators generated incidentally by the underlying layer of the large language model during the semantic consistency evaluation in the preceding steps can be directly extracted, quantified, and determined as the semantic relevance degree.

[0158] Furthermore, based on the calculated semantic relevance values, the text fragments within the recalled fragment set can be repositioned according to a descending order of semantic relevance. In an aircraft maintenance scenario, if the target query request involves troubleshooting a specific fault code, after the above reordering, the English FIM manual troubleshooting steps fragment directly corresponding to the fault code and possessing causal logical constraints will be prioritized at the head of the queue due to its highest semantic relevance; while the relatively broad Chinese historical fault maintenance log fragments may be placed at the back of the queue. In this way, the new sequence after position reordering constitutes the target sorted fragment set.

[0159] If the current retrieval system functions solely as a knowledge base query engine, the target ranked fragment set can be directly rendered onto the interactive terminal interface of maintenance personnel, allowing users to browse it in order of importance. If the current retrieval system's front-end is integrated with a large language model's generation module, the obtained target ranked fragment set can be fed as high-quality contextual prompts to the large language model, driving it to generate the final coherent troubleshooting guidance text based on these rigorously ranked professional knowledge fragments.

[0160] This embodiment, after confirming that the recalled fragment set meets the semantic consistency condition, further determines the semantic relevance between each recalled fragment and the target query request, and performs a secondary reordering based on the semantic relevance to obtain the target sorted fragment set for output. This effectively makes up for the deficiency of insufficient sorting accuracy in single vector retrieval when processing complex logical text, and ensures that the most core and logically close key knowledge fragments are placed with the highest priority before the final output or input into the large language model generation stage.

[0161] Based on the above embodiments, as an optional embodiment, the step of inputting the recalled fragment set into a large language model for semantic evaluation and obtaining the evaluation result output by the large language model specifically includes, but is not limited to: The retrieved fragment set and the target query request are combined and input into the large language model for logical analysis to obtain the semantic relevance score and logical sufficiency score output by the large language model. If the semantic relevance score reaches a first preset threshold and the logical sufficiency score reaches a second preset threshold, the evaluation result is determined to be that the recalled fragment set satisfies the preset semantic consistency condition. If the semantic relevance score does not reach the first preset threshold and / or the logical sufficiency score does not reach the second preset threshold, the evaluation result is determined to be that the recalled fragment set does not meet the semantic consistency condition.

[0162] In a specific implementation, the user's target query request and the newly retrieved recall fragment set can be structurally spliced ​​together using a preset prompt word template to construct a composite text sequence that includes query background and reference facts.

[0163] Subsequently, the complex text sequence is input into a large language model with logical reasoning capabilities. Upon receiving the input, the large language model performs a scoring task from two independent dimensions: first, it calculates a semantic relevance score, which characterizes the degree to which the recalled fragments match the target query request in terms of topic and entity, such as whether they all mention the same aircraft system component code; second, it calculates a logical sufficiency score, which characterizes whether the recalled fragments contain complete business logic or operational steps that can actually solve the user's query problem.

[0164] In the context of aircraft maintenance, if a maintenance worker queries "the engine start switch cannot be held in the ground position", and the retrieval system only retrieves the basic physical parameters of the switch, the large language model will give a high semantic relevance score after logical analysis. However, due to the lack of specific fault isolation and troubleshooting steps, it will give a very low logical sufficiency score.

[0165] The first and second preset thresholds can be pre-configured numerical indicators. For example, the first preset threshold can be set to 80 points and the second preset threshold to 85 points. In practice, only when both scores simultaneously reach or exceed their respective threshold limits can the current recall content be considered relevant and useful in the judgment logic, thus providing a basis for guiding troubleshooting decisions. This formally confirms the evaluation result as meeting the semantic consistency condition, triggering the subsequent direct sorting and generation process.

[0166] Conversely, if the recalled fragment set exhibits any of the following conditions: semantic irrelevance, empty content, lack of operability, or insufficient logic, the evaluation conclusion of the large language model is deemed unsatisfactory. At the execution level, the evaluation result that does not meet the preset semantic consistency conditions is output accordingly, thereby triggering the difficult query rewriting and re-recall or fallback fault-tolerant branch in the aforementioned embodiments.

[0167] This embodiment introduces the reasoning capabilities of a large language model, combines the recalled fragment set with the target query request for logical analysis, and decouples the output semantic relevance score and logical sufficiency score. Through dual-dimensional threshold verification, the recall results are required to simultaneously meet the dual requirements of topic matching and complete troubleshooting logic. This ensures that low-quality knowledge fragments can be identified and intercepted before the final response is generated, fundamentally eliminating the risk of generating incorrect guidance due to blindly delivering low-quality recalled content.

[0168] Figure 6This is a schematic diagram of the structure of the training device for the multilingual embedding model provided by the present invention, as shown below. Figure 6 As shown, it mainly includes, but is not limited to: Corpus acquisition module 11 is used to acquire the first language data batch and the second language data batch in the current training iteration step; The loss calculation module 12 is used to obtain a first loss value obtained by performing processing on the first language data batch based on the multilingual embedding model to be trained, and a second loss value obtained by performing processing on the second language data batch based on the multilingual embedding model. The parameter update module 13 is used to update the model parameters of the multilingual embedding model and determine the sampling scale factor based on the first loss value and the second loss value. The sampling control module 14 is used to dynamically adjust the sampling probability for the first language data and the second language data based on the difference characteristics between the first loss value and the second loss value. The iteration control module 15 is used to construct a new first language data batch and a new second language data batch for the next training iteration step, update the next training iteration step to the current training iteration step, and return to the corpus acquisition module to perform the steps of acquiring the first language data batch and the second language data batch for the current training iteration step, until the preset training termination condition is met.

[0169] It should be noted that the training device for the multilingual embedding model provided by the present invention can execute the training method for the multilingual embedding model described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0170] Figure 7 This is a schematic diagram of the information retrieval device provided by the present invention, as shown below. Figure 7 As shown, it mainly includes, but is not limited to: The request acquisition module 21 is used to acquire the target query request and trigger the execution of the retrieval evaluation iteration operation of the corresponding module below until the preset process exit condition is met. The matching and retrieval module 22 is used to input the target query request into the multilingual embedding model, obtain the query feature vector output by the multilingual embedding model, and perform matching and retrieval in the knowledge base based on the query feature vector to obtain the corresponding recall fragment set; wherein, the multilingual embedding model is trained by the training method of the multilingual embedding model according to any one of claims 1 to 6; The semantic evaluation module 23 is used to input the recalled fragment set into the large language model for semantic evaluation and obtain the evaluation result output by the large language model. The first output module 24 is used to output retrieval results based on the recalled fragment set and trigger the process exit condition when the evaluation result indicates that the recalled fragment set meets the preset semantic consistency condition. The intent rewriting module 25 is used to input the target query request into the query rewriting model when the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and the preset fault tolerance termination condition is not met, obtain the updated target query request output by the query rewriting model, and use the updated target query request as a new target query request to enter the next round of retrieval evaluation iteration steps. The second output module 26 is used to generate a fallback response result based on the acquired recall fragment set and trigger the process exit condition when the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and the fault tolerance termination condition has been reached.

[0171] It should be noted that the information retrieval device provided by the present invention can execute the information retrieval method described in any of the above embodiments during specific operation, and this embodiment will not elaborate on this.

[0172] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communications bus 840. The processor 810 can call logical instructions in the memory 830 to execute a training method and / or an information retrieval method for a multilingual embedding model. The training method for the multilingual embedding model includes iteratively executing the following steps until a preset training termination condition is met to obtain a multilingual embedding model: obtaining a first language data batch and a second language data batch for the current training iteration step; obtaining a first loss value obtained by processing the first language data batch based on the multilingual embedding model to be trained, and a second loss value obtained by processing the second language data batch based on the multilingual embedding model; updating the model parameters of the multilingual embedding model and determining a sampling ratio factor based on the first loss value and the second loss value; constructing a new first language data batch and a new second language data batch for the next training iteration step, wherein the data volume ratio between the new first language data batch and the new second language data batch is determined based on the sampling ratio factor; and updating the next training iteration step to the current training iteration step.

[0173] This information retrieval method includes obtaining a target query request and performing the following retrieval evaluation iterative steps until a preset process exit condition is met: inputting the target query request into a multilingual embedding model to obtain a query feature vector output by the multilingual embedding model; using the query feature vector to perform a matching retrieval in a knowledge base to obtain a corresponding recall fragment set; inputting the recall fragment set into a large language model for semantic evaluation to obtain an evaluation result output by the large language model; if the evaluation result indicates that the recall fragment set meets a preset semantic consistency condition, outputting a retrieval result based on the recall fragment set and triggering the process exit condition .... If the recalled fragment set does not meet the semantic consistency condition and has not reached the preset fault tolerance termination condition, the target query request is input into the query rewriting model to obtain the updated target query request output by the query rewriting model, and the updated target query request is used as a new target query request to enter the next round of retrieval evaluation iteration steps; if the evaluation result indicates that the recalled fragment set does not meet the semantic consistency condition and has reached the fault tolerance termination condition, a fallback response result is generated based on the obtained recalled fragment set and the process exit condition is triggered; the multilingual embedding model is trained by any of the multilingual embedding model training methods described above.

[0174] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is capable of executing a training method and / or an information retrieval method for a multilingual embedding model, the training method for the multilingual embedding model comprising iteratively executing the following steps until a preset training termination condition is met to obtain a multilingual embedding model: obtaining a first language data batch and a second language data batch for the current training iteration step; obtaining a first loss value obtained by processing the first language data batch based on the multilingual embedding model to be trained, and a second loss value obtained by processing the second language data batch based on the multilingual embedding model; updating the model parameters of the multilingual embedding model and determining a sampling ratio factor based on the first loss value and the second loss value; constructing a new first language data batch and a new second language data batch for the next training iteration step, wherein the data volume ratio between the new first language data batch and the new second language data batch is determined based on the sampling ratio factor; and updating the next training iteration step to the current training iteration step.

[0176] The information retrieval method includes obtaining a target query request to perform the following retrieval evaluation iteration steps until a preset process exit condition is met: inputting the target query request into a multilingual embedding model to obtain the query feature vector output by the multilingual embedding model; The query feature vector is used to perform matching and retrieval in the knowledge base to obtain the corresponding recall fragment set; the recall fragment set is input into a large language model for semantic evaluation, and the evaluation result output by the large language model is obtained; if the evaluation result indicates that the recall fragment set meets the preset semantic consistency condition, the retrieval result is output based on the recall fragment set, and the process exit condition is triggered; if the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and has not reached the preset fault tolerance termination condition, the target query request is input into the query rewriting model, the updated target query request output by the query rewriting model is obtained, and the updated target query request is used as the new target query request to enter the next round of retrieval evaluation iteration steps; if the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and has reached the fault tolerance termination condition, a fallback response result is generated based on the obtained recall fragment set, and the process exit condition is triggered; the multilingual embedding model is trained by any of the multilingual embedding model training methods described above.

[0177] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a training method and / or an information retrieval method for a multilingual embedding model. The training method for the multilingual embedding model includes iteratively executing the following steps until a preset training termination condition is met to obtain a multilingual embedding model: obtaining a first language data batch and a second language data batch for the current training iteration step; obtaining a first loss value obtained by processing the first language data batch based on the multilingual embedding model to be trained, and a second loss value obtained by processing the second language data batch based on the multilingual embedding model; updating the model parameters of the multilingual embedding model and determining a sampling ratio factor based on the first loss value and the second loss value; constructing a new first language data batch and a new second language data batch for the next training iteration step, wherein the data volume ratio between the new first language data batch and the new second language data batch is determined based on the sampling ratio factor; and updating the next training iteration step to the current training iteration step.

[0178] The information retrieval method includes obtaining a target query request to perform the following retrieval evaluation iteration steps until a preset process exit condition is met: inputting the target query request into a multilingual embedding model to obtain the query feature vector output by the multilingual embedding model; The query feature vector is used to perform matching and retrieval in the knowledge base to obtain the corresponding recall fragment set; the recall fragment set is input into a large language model for semantic evaluation, and the evaluation result output by the large language model is obtained; if the evaluation result indicates that the recall fragment set meets the preset semantic consistency condition, the retrieval result is output based on the recall fragment set, and the process exit condition is triggered; if the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and has not reached the preset fault tolerance termination condition, the target query request is input into the query rewriting model, the updated target query request output by the query rewriting model is obtained, and the updated target query request is used as the new target query request to enter the next round of retrieval evaluation iteration steps; if the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and has reached the fault tolerance termination condition, a fallback response result is generated based on the obtained recall fragment set, and the process exit condition is triggered; the multilingual embedding model is trained by any of the multilingual embedding model training methods described above.

[0179] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0180] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a multilingual embedding model, characterized in that, This includes iteratively executing the following steps until the preset training termination condition is met to obtain a multilingual embedding model: Obtain the first language data batch and the second language data batch for the current training iteration step; Obtain a first loss value obtained by performing processing on the first language data batch based on the multilingual embedding model to be trained, and a second loss value obtained by performing processing on the second language data batch based on the multilingual embedding model; Based on the first loss value and the second loss value, update the model parameters of the multilingual embedding model and determine the sampling scale factor; Construct a new first language data batch and a new second language data batch for the next training iteration, wherein the data volume ratio between the new first language data batch and the new second language data batch is determined based on the sampling ratio factor; Update the next training iteration step to the current training iteration step.

2. The training method for the multilingual embedding model according to claim 1, characterized in that, The sampling scaling factor is determined based on the following steps: Eliminate the dimensional difference between the first loss value and the second loss value, and obtain the first normalized loss corresponding to the first loss value and the second normalized loss corresponding to the second loss value respectively; Calculate the numerical ratio of the first normalized loss to the second normalized loss, and determine the numerical ratio as the sampling scale factor.

3. The training method for the multilingual embedding model according to claim 2, characterized in that, Determining the data volume ratio between the new first language data batch and the new second language data batch based on the sampling ratio factor includes: According to the sampling ratio factor, determine the target sampling weight ratio of the new first language data and the new second language data in the next training iteration step; Based on the target sampling weight ratio, the data volume is allocated between the new first language data batch and the new second language data batch to obtain the data volume allocation ratio.

4. The training method for the multilingual embedding model according to claim 1, characterized in that, Based on the first loss value and the second loss value, the model parameters of the multilingual embedding model are updated, including: Based on the first loss value, calculate the first gradient vector of the first language data with respect to the shared parameters in the multilingual embedding model; Based on the second loss value, calculate the second gradient vector of the second language data with respect to the shared parameters in the multilingual embedding model; Store the first gradient vector and the second gradient vector in a preset cache area; Based on the first gradient vector and the second gradient vector in the cache, a joint update process is performed on the shared parameters of the multilingual embedding model.

5. The training method for the multilingual embedding model according to claim 1, characterized in that, The multilingual embedding model includes a dual-tower network model based on the Transformer architecture, which is used to independently encode data in different languages.

6. The training method for a multilingual embedding model according to any one of claims 1 to 5, characterized in that, The first language data includes aircraft maintenance experience data in the first language, and the second language data includes original equipment manufacturer (OEM) technical specifications data in the second language. The first language is Chinese, and the aviation maintenance experience data includes at least one of emergency response plan data, historical fault maintenance log data, and minimum equipment list data; The second language is English, and the original equipment manufacturer (OEM) technical specifications data includes fault isolation manual data.

7. An information retrieval method, characterized in that, This includes obtaining the target query request to perform the following retrieval evaluation iteration steps until the preset process exit conditions are met: The target query request is input into the multilingual embedding model to obtain the query feature vector output by the multilingual embedding model; The query feature vector is used to perform matching and retrieval in the knowledge base to obtain the corresponding recall fragment set; The recalled fragment set is input into a large language model for semantic evaluation, and the evaluation result output by the large language model is obtained. If the evaluation result indicates that the recalled fragment set meets the preset semantic consistency condition, the retrieval result is output based on the recalled fragment set, and the process exit condition is triggered; If the evaluation result indicates that the recalled fragment set does not meet the semantic consistency condition and has not reached the preset fault tolerance termination condition, the target query request is input into the query rewriting model, the updated target query request output by the query rewriting model is obtained, and the updated target query request is used as a new target query request to enter the next round of retrieval evaluation iteration steps. If the evaluation result indicates that the recalled fragment set does not meet the semantic consistency condition and the fault tolerance termination condition has been reached, a fallback response result is generated based on the acquired recalled fragment set and the process exit condition is triggered. The multilingual embedding model is trained using the training method for the multilingual embedding model according to any one of claims 1 to 6.

8. The information retrieval method according to claim 7, characterized in that, The query rewriting model was obtained based on the following training steps: Obtain the original query sample set, which contains multiple basic query samples; According to the preset sample expansion strategy, data transformation processing is performed on the basic query samples to obtain the corresponding derived query samples; the sample expansion strategy includes at least one of the following: keyword masking strategy, multi-query intent fusion strategy, and context information simplification strategy. The derived query samples are mapped and combined with the corresponding basic query samples to construct the target fine-tuning dataset; The query rewriting model is obtained by performing supervised fine-tuning training on the preset base language model using the target fine-tuning dataset.

9. The information retrieval method according to claim 8, characterized in that, The step of performing data transformation processing on the basic query samples according to a preset sample expansion strategy to obtain corresponding derived query samples includes at least one of the following: When the sample expansion strategy includes the keyword masking strategy, a masking replacement process is performed on the target keywords in the basic query sample to obtain the corresponding derived query sample; When the sample expansion strategy includes the multi-query intent fusion strategy, at least two basic query samples are obtained, and the at least two basic query samples are subjected to text fusion processing to obtain the corresponding derived query samples. When the sample expansion strategy includes the context information simplification strategy, some context text fragments in the basic query sample are deleted to obtain the corresponding derived query sample.

10. The information retrieval method according to claim 7, characterized in that, The step of using the query feature vector to perform matching and retrieval in the knowledge base to obtain the corresponding recall fragment set includes: Determine the target recall number corresponding to the current retrieval evaluation iteration step; The query feature vector is used to perform matching and retrieval in the knowledge base to obtain the recall fragment set according to the target recall quantity; Wherein, the number of target recalls in the next round of retrieval evaluation iteration step in which the updated target query request is located is greater than the number of target recalls in the current retrieval evaluation iteration step.

11. The information retrieval method according to claim 7, characterized in that, When the evaluation result indicates that the recalled fragment set meets a preset semantic consistency condition, the retrieval result is output based on the recalled fragment set, including: Determine the semantic correlation between each recalled fragment in the recalled fragment set and the target query request; Based on the semantic relevance, each recalled fragment in the recalled fragment set is sorted to obtain the target sorted fragment set; The target sorted fragment set is output as the retrieval result.

12. The information retrieval method according to claim 7, characterized in that, The step of inputting the recalled fragment set into a large language model for semantic evaluation and obtaining the evaluation result output by the large language model includes: The retrieved fragment set and the target query request are combined and input into the large language model for logical analysis to obtain the semantic relevance score and logical sufficiency score output by the large language model. If the semantic relevance score reaches a first preset threshold and the logical sufficiency score reaches a second preset threshold, the evaluation result is determined to be that the recalled fragment set satisfies the preset semantic consistency condition. If the semantic relevance score does not reach the first preset threshold and / or the logical sufficiency score does not reach the second preset threshold, the evaluation result is determined to be that the recalled fragment set does not meet the semantic consistency condition.

13. A training device for a multilingual embedding model, characterized in that, include: The corpus acquisition module is used to acquire the first language data batch and the second language data batch in the current training iteration step; The loss calculation module is used to obtain a first loss value obtained by performing processing on the first language data batch based on the multilingual embedding model to be trained, and a second loss value obtained by performing processing on the second language data batch based on the multilingual embedding model. The parameter update module is used to update the model parameters of the multilingual embedding model and determine the sampling scale factor based on the first loss value and the second loss value. The sampling control module is used to dynamically adjust the sampling probability for the first language data and the second language data based on the difference characteristics between the first loss value and the second loss value; The iteration control module is used to construct a new first language data batch and a new second language data batch for the next training iteration step, update the next training iteration step to the current training iteration step, and return to the corpus acquisition module to perform the steps of acquiring the first language data batch and the second language data batch for the current training iteration step, until the preset training termination condition is met.

14. An information retrieval device, characterized in that, include: The request retrieval module is used to retrieve the target query request and trigger the execution of the retrieval evaluation iteration operation of the corresponding module below until the preset process exit condition is met; The matching and retrieval module is used to input the target query request into the multilingual embedding model, obtain the query feature vector output by the multilingual embedding model, and perform matching and retrieval in the knowledge base based on the query feature vector to obtain the corresponding recall fragment set; wherein, the multilingual embedding model is trained by the training method of the multilingual embedding model according to any one of claims 1 to 6; The semantic evaluation module is used to input the recalled fragment set into the large language model for semantic evaluation and obtain the evaluation result output by the large language model. The first output module is used to output retrieval results based on the recalled fragment set and trigger the process exit condition when the evaluation result indicates that the recalled fragment set meets the preset semantic consistency condition. The intent rewriting module is used to input the target query request into the query rewriting model when the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and the preset fault tolerance termination condition is not met, obtain the updated target query request output by the query rewriting model, and use the updated target query request as a new target query request to enter the next round of retrieval evaluation iteration steps. The second output module is used to generate a fallback response result based on the acquired recall fragment set and trigger the process exit condition when the evaluation result indicates that the recall fragment set does not meet the semantic consistency condition and the fault tolerance termination condition has been reached.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the training method for the multilingual embedding model as described in any one of claims 1 to 6, or the information retrieval method as described in any one of claims 7 to 12.

16. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the multilingual embedding model as described in any one of claims 1 to 6, or the information retrieval method as described in any one of claims 7 to 12.

Images