A cross-language understanding method, apparatus, and device

Abstract

The embodiment of the specification discloses a cross-language understanding method, device and equipment, relates to the technical field of artificial intelligence, and is used for solving the problem that the existing cross-language understanding is difficult to cope with high requirement scenes such as small languages and professional fields. The method comprises the following steps: determining a pre-constructed cross-language knowledge base; the construction of the cross-language knowledge base is: the source language original data of the current professional field is disassembled to obtain the source language question and answer pair in combination with external authoritative information; based on translation quality verification, the target language question and answer pair after translation of the source language question and answer pair is obtained to construct the cross-language knowledge base in combination with the source language question and answer pair; receiving a target language user question to be processed, disassembling the question, obtaining a sub-question set, and searching in the cross-language knowledge base to obtain the knowledge matched with the sub-question set in semantics; and generating a current target language answer based on the knowledge to complete the cross-language understanding.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a cross-language understanding method, apparatus, and device. Background Technology

[0002] Cross-language intelligent question answering is a technology that eliminates language barriers and enables knowledge dissemination, allowing users to directly access high-quality professional knowledge in other languages ​​using their native language.

[0003] Traditional cross-language question answering processes typically rely on a sequential workflow of translation, retrieval, and generation. First, the user's question is translated into the source language; then, the answer is retrieved from a source language knowledge base; and finally, the answer is translated back into the target language. However, in complex scenarios involving less commonly spoken languages ​​or specialized fields such as finance, translation errors related to technical terminology and complex logic accumulate and amplify during language conversion, leading to distorted answers. Furthermore, traditional solutions rely on general-domain machine translation engines, which struggle to guarantee translation quality for specialized texts.

[0004] In practical applications, user questions in specialized fields of less commonly taught languages ​​often exhibit ambiguity or non-standard terminology due to limited language resources. In such cases, the traditional approach of translating first and then retrieving is prone to initial translation errors, leading to incorrect search directions and difficulty in extracting the desired content from a vast amount of highly specialized multilingual knowledge. Therefore, relying solely on a monolingual knowledge base or superficially bilingual comparisons through simple translation alignment lacks semantic relevance within the specialized field. This makes it difficult for the system to understand the same professional content expressed in different languages, hindering accurate location of the required knowledge fragments and resulting in generated answers that deviate from the user's true intent.

[0005] Therefore, there is a need for accurate cross-language understanding methods that can handle high-requirement scenarios such as less commonly spoken languages ​​and specialized fields. Summary of the Invention

[0006] This specification provides one or more embodiments of a cross-language understanding method, device, and medium to solve the following technical problem: the need for an accurate cross-language understanding method capable of handling high-requirement scenarios such as less commonly spoken languages ​​and specialized fields.

[0007] To solve the above-mentioned technical problems, one or more embodiments of this specification are implemented as follows: This specification provides a cross-language understanding method through one or more embodiments, including: A pre-constructed cross-language knowledge base is determined; the construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, combining it with external authoritative information to obtain source language question-answer pairs; based on translation quality verification, obtaining target language question-answer pairs translated from the source language question-answer pairs; the translation quality verification includes at least: bidirectional semantic alignment verification; and constructing the cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs. Receive user questions to be processed in the target language, break down the user questions to be processed to obtain a set of sub-questions, and retrieve knowledge that semantically matches the set of sub-questions from the cross-language knowledge base; Based on the knowledge provided, a response in the target language is generated, thus achieving cross-language understanding.

[0008] This specification provides a cross-language understanding device according to one or more embodiments, comprising: A cross-language knowledge base determination unit is used to determine a pre-constructed cross-language knowledge base. The construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, combining it with external authoritative information to obtain source language question-answer pairs; obtaining target language question-answer pairs translated from the source language question-answer pairs based on translation quality verification; the translation quality verification includes at least: bidirectional semantic alignment verification; and constructing the cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs. The problem retrieval unit is used to receive user problems to be processed in the target language, decompose the user problems to be processed to obtain a set of sub-problems, and retrieve knowledge that semantically matches the set of sub-problems from the cross-language knowledge base; The generation unit is used to generate a response in the current target language based on the knowledge, thereby completing cross-language understanding.

[0009] This specification provides one or more embodiments of a cross-language understanding device, comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: A pre-constructed cross-language knowledge base is determined; the construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, combining it with external authoritative information to obtain source language question-answer pairs; based on translation quality verification, obtaining target language question-answer pairs translated from the source language question-answer pairs; the translation quality verification includes at least: bidirectional semantic alignment verification; and constructing the cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs. Receive user questions to be processed in the target language, break down the user questions to be processed to obtain a set of sub-questions, and retrieve knowledge that semantically matches the set of sub-questions from the cross-language knowledge base; Based on the knowledge provided, a response in the target language is generated, thus achieving cross-language understanding.

[0010] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: By decomposing the original source language data and combining it with authoritative external information to generate source language question-answer pairs, the problem of insufficient coverage of specialized content in less commonly taught languages ​​is effectively addressed. A translation quality verification mechanism, including bidirectional semantic alignment verification, verifies the translation quality of the source language question-answer pairs to obtain corresponding target language question-answer pairs, ensuring consistency between the target language and source language pairs and reducing the risk of further incorrect answers due to translation errors. Furthermore, by decomposing user questions into sub-question sets and retrieving matching knowledge from a cross-language knowledge base, question decoupling is achieved, making this approach applicable to complex question scenarios. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart illustrating a cross-language understanding method provided in an embodiment of this specification; Figure 2 This is a schematic diagram of a bidirectional semantic alignment verification process in an application scenario provided by an embodiment of this specification; Figure 3 This specification provides a schematic diagram of a cross-language understanding logic flow in an application scenario, as illustrated in an embodiment of the present specification. Figure 4 This is a schematic diagram of the structure of a cross-language understanding device provided in an embodiment of this specification; Figure 5 This is a schematic diagram of the structure of a cross-language understanding device provided in an embodiment of this specification. Detailed Implementation

[0012] This specification provides an embodiment of a cross-language understanding method, apparatus, and device.

[0013] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0014] In the context of the background technology, cross-language intelligent question answering is a technology that eliminates language barriers and achieves knowledge dissemination, enabling users to directly access high-quality professional knowledge in other languages ​​using their native language. With technological advancements, users of less commonly spoken languages, limited by language resources, also need technologies that allow them to perform cross-language searches based on their native language to obtain relevant professional knowledge. However, existing solutions primarily rely on a simple concatenation of general machine translation and static knowledge bases. When dealing with cross-language understanding of less commonly spoken languages ​​and highly specialized questions in specific fields, these solutions suffer from limitations in achieving sufficient search accuracy and yielding low reliability of search results.

[0015] Because texts in specialized fields, such as finance, are characterized by dense terminology, rigorous logic, and low tolerance for errors, and because less commonly spoken languages ​​suffer from limited training data and weak pre-trained model support due to language resource constraints, the following methods also have limitations. To illustrate these problems more clearly, the existing methods are explained below: One existing approach involves a sequential process of translation, retrieval, and generation based on general machine translation or a static knowledge base. This method translates the user's specific language problem into a high-resource language like English or Chinese using a general translation engine. Then, the answer is retrieved from a relevant language-specific knowledge base, such as a financial knowledge base in the financial field, and translated back into the user's native language. While simple to implement, this approach struggles to guarantee consistency in financial terminology in practice. Furthermore, translation errors are amplified at each stage of the translation process, leading to incorrect questions and answers. Moreover, the knowledge bases relied upon are often monolingual or only superficially translated and aligned, lacking deep cross-linguistic semantic mapping and creating semantic silos. This prevents the system from understanding the same financial concepts behind different languages, resulting in low reliability of the generated answers.

[0016] Another existing approach involves direct retrieval based on cross-lingual pre-trained models or multilingual embeddings. This method utilizes multilingual models to encode text from different languages ​​into the same vector space for semantic matching, or employs cross-lingual retrieval-augmented generation (RAG) technology. However, this approach has limitations in understanding specialized terminology and logical reasoning within specific domains. While current mainstream models have achieved good performance in high-resource languages ​​like Chinese and English through large-scale instruction fine-tuning and domain adaptation, high-quality labeled data is scarce in specialized scenarios involving less common languages ​​like Indonesian. Without verification mechanisms for semantic consistency and professional accuracy, training samples may introduce noise, leading to a decline in model performance and low relevance in retrieval results. Furthermore, if the translation quality of the knowledge base itself is low, direct use will also introduce noise, causing a chain reaction where correct questions lead to incorrect information.

[0017] Based on the above, there is a current need for a cross-language understanding method that does not rely entirely on massive amounts of annotated data in less commonly spoken languages ​​or large-scale models in specialized domains, but rather addresses the entire process from source language knowledge construction to target language answer generation. This would provide a highly reliable and interpretable cross-language understanding method for specialized scenarios involving less commonly spoken languages. To this end, this application proposes a cross-language understanding method. It is based on constructing a closed-loop translation framework that integrates multi-model collaboration, reward scoring, and back-translation verification. This framework ensures semantic fidelity and terminological consistency during the source-to-target language conversion process. Furthermore, based on a high-quality aligned cross-language knowledge base, it achieves accurate retrieval and generates answers that match the user's question. The specific implementation plan of this application will be described in detail below based on this overall approach.

[0018] Figure 1 This diagram illustrates a cross-language understanding method provided in one or more embodiments of this specification. The method can be applied to application areas involving multilingual information processing, such as financial services, legal services, medical services, and scientific research services. The process can be executed using computing devices relevant to the specific field (e.g., the server of an intelligent question-answering system in financial services). Furthermore, certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0019] The following detailed description of each step of the method, with reference to specific embodiments, illustrates the method, which includes the following steps: S102: Determine a pre-constructed cross-language knowledge base; the construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, combining it with external authoritative information to obtain source language question-answer pairs; based on translation quality verification, obtaining target language question-answer pairs translated from the source language question-answer pairs; the translation quality verification includes at least: bidirectional semantic alignment verification; and constructing a cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs.

[0020] The current professional field refers to the application field corresponding to the current question and answer, including but not limited to finance, law, and medicine, which have specific knowledge categories and are characterized by high specialization, standardized terminology, and rigorous logic. This specification uses the financial field, characterized by highly specialized language, dense terminology, and rigorous logic, as an example for subsequent explanations, but this specification is not limited to the financial field.

[0021] The target language is the user's native language, which is usually a less common language or the local language. The source language is the original data, which is written in a high-resource language such as Chinese or English and is unprocessed professional text material corresponding to the current professional field.

[0022] External authoritative information refers to credible knowledge sources that are independent of the original source language data and recognized by the industry, such as official texts and regulatory announcements, introduced through online searches when generating source language question-answer pairs. These sources are used to supplement, correct, or enhance the accuracy of the answers, helping to avoid localization bias and ensuring that the source language answers have global consistency and credibility.

[0023] A source language question-answering pair is a structured unit consisting of the expected answer to an atomic question, where an atomic question corresponds to an indivisible sub-question, and the corresponding answer is also a single piece of knowledge.

[0024] Translation quality verification is a process of evaluating and screening machine translation results. It includes at least bidirectional semantic alignment verification to check whether a single semantic equivalence is preserved, but does not exclude the verification of other auxiliary indicators.

[0025] Based on the aforementioned background technology, for cross-language intelligent question-answering systems, when the user's native language (i.e., the target language) is a less common language with scarce language resources, such as Indonesian, and the knowledge domain applied by the cross-language intelligent question-answering system is highly specialized, such as the financial field, traditional cross-language understanding methods struggle to obtain a large amount of expert-annotated and terminologically accurate knowledge when building the knowledge base, resulting in a lack of assurance regarding the quality of the source data. Furthermore, directly using a general machine translation engine to translate professional documents in their entirety can easily lead to mistranslation of key terms and misunderstandings of logical relationships, reducing the credibility of the translated knowledge base. Moreover, existing methods lack semantic consistency verification of knowledge units during the translation process, resulting in the inability to establish an accurate semantic mapping between knowledge fragments in the source and target languages, thus rendering subsequent cross-language retrieval ineffective.

[0026] To address the aforementioned issues, step S102 involves acquiring the original source language data for the current professional field, then decomposing the original source language data and combining it with external authoritative information to obtain source language question-and-answer pairs.

[0027] In one approach, the decomposition method can be a semantic parsing process. For example, based on an existing text segmentation model, it identifies independent factual statements or definition sentences in a document, splits each statement into interrogative sentences, and uses these as source language questions. It then retrieves corresponding external authoritative information to assist in generating source language answers corresponding to the source language questions, thus obtaining source language question-answer pairs. This involves using atomic questions as query terms to search external authoritative information, obtaining corresponding source language answers, and then constructing source language question-answer pairs based on the atomic sub-questions and their corresponding source language answers. One approach may incorporate external authoritative information in ways including, but not limited to: comparing answers obtained from the original source language data with answers obtained from the external authoritative information, correcting the answers, and obtaining source language answers; supplementing and expanding answers obtained from the original source language data based on answers obtained from external authoritative information; and directly using external authoritative information sources to obtain source language answers when the original data does not fully cover the question. By integrating external authoritative information, it compensates for potential incomplete or delayed expressions in local minority language source languages, enabling source language answers to be real-time and professionally authoritative.

[0028] The obtained source language question-answer pairs are translated to obtain translation results, and translation quality verification is performed on the translation results. This translation quality verification includes at least bidirectional semantic alignment verification. Understandably, in one implementation, bidirectional semantic alignment verification involves translating the source language question-answer pairs into initial target language question-answer pairs, and then translating the initial target language question-answer pairs back into the source language to obtain back-translated question-answer pairs. A similarity score is obtained based on the cosine similarity between the original source language question-answer pairs and the back-translated question-answer pairs, and the verification pass / fail is determined based on the similarity score. This similarity score can be set based on expert experience or on statistical analysis results of historical data. The specific implementation of the above bidirectional semantic alignment verification is only an example; those skilled in the art can also use other semantic equivalence judgment methods, such as logical form alignment, for alignment verification. Through translation quality verification, the problem of translation error accumulation in traditional methods is avoided, and the problem of answer bias caused by translation errors is reduced.

[0029] After the initial target language question-answer pairs pass verification, the translated target language question-answer pairs are obtained. The verified source language question-answer pairs are then associated and stored with their corresponding target language question-answer pairs, forming a structured cross-language knowledge base. This addresses the problem of limited language resources for less commonly taught languages ​​and provides a stable semantic knowledge foundation for subsequent cross-language understanding. Based on this pre-built cross-language knowledge base, online question-answering services can retrieve corresponding answers without real-time translation, improving both response speed and accuracy.

[0030] In one alternative approach, the original source language data in the current professional field is decomposed and combined with external authoritative information to obtain source language question-answer pairs, specifically including: In addition to the semantic analysis methods mentioned above, this optional approach can also decompose the source language data of the current professional field into multiple atomic sub-problems. Taking cross-language understanding in the Indonesian financial field as an example, it requires using professional financial documents written in Indonesian as the source language data. Then, this source language data is decomposed into multiple atomic sub-problems.

[0031] The process of breaking down the original language data into atomic sub-problems can be approached in two ways. One approach is to use a rule-based dependency parser to decompose long paragraphs into atomic sub-problems. Another approach involves semantic role labeling of the source language data, identifying predicates and related semantic roles such as events, time, and conditions. Based on different combinations of semantic roles, corresponding question templates are matched to generate atomic sub-problems. Essentially, decomposing the source language data into atomic problems means dividing the source language problem into individual, independent sub-problems. For example, from a description of "loan classification," one could extract independent sub-problems such as "How should loans overdue for more than 90 days be classified?" and "Are restructured loans considered non-performing?". This process decomposes the source language data within the current professional domain, decoupling the problems so that each question and answer can retrieve and match a single knowledge point, improving the accuracy of retrieving answers to complex questions across different language domains.

[0032] For each atomic sub-question, external authoritative information is introduced to assist in generating accurate answers, thus generating source language question-answer pairs. That is, when the original source language data cannot cover the atomic sub-questions, external authoritative information corresponding to the atomic sub-questions is searched based on the large language model. This allows the large language model to generate corresponding source language answers based on the atomic sub-questions and the external authoritative information, thereby forming source language question-answer pairs. Specifically, the process of searching for external authoritative information corresponding to the atomic sub-questions based on the large language model, and then generating corresponding source language answers based on the atomic sub-questions and the external authoritative information, is as follows: For each atomic sub-problem, the large language model is forcibly invoked to enable its real-time web search capabilities, thereby acquiring relevant external authoritative information in real time. For example, the large language model generates search query terms based on the atomic sub-problem, performs a web search based on these terms, obtains corresponding URLs, website names, etc., and then filters the search results for officially reliable content, ultimately obtaining external authoritative information. Since the retrieved external authoritative information may contain content not only in the source language but also in other languages, if the retrieved external authoritative information is not in the source language, the terminology within the external authoritative information is identified, and the non-source language terms are translated to obtain the source language terms. This translation can be based on a language database or machine translation, and will not be elaborated further. The terminology refers to the standardized definitions of professional terms, regulatory codes, organization names, etc., included in the external authoritative information.

[0033] After obtaining the source language terms, a source language answer corresponding to the atomic sub-question is generated based on the acquired source language terms and their contextual information. For example, in one approach, the atomic sub-question, the translated source language terms, and the contextual information of those source language terms in external authoritative information are input into a large language model. Structured prompts instruct the model to output based on the source language, generating a source language answer corresponding to the atomic sub-question. In another approach, the question type of the atomic sub-question is identified, and a corresponding template is matched from a pre-stored standardized answer template library. Template types include definitional, judgmental, and categorical. After selecting the categorization template corresponding to the question type, the input source language terms and their contextual information are filled into predefined slots in the template to obtain the source language answer.

[0034] This process forcibly invokes the real-time network search capabilities of a large language model, ensuring that the generation of answers for each question is based on the latest publicly available information. This addresses the lack of timeliness in static knowledge bases, enabling subsequent knowledge base construction to be updated and evolved, ensuring the professionalism and timeliness of knowledge during cross-language understanding. By decoupling questions to obtain atomic questions, when users pose complex questions, these questions can be broken down and accurately matched and reasoned with atomic sub-questions in the knowledge base, thereby combining to generate accurate composite answers and effectively improving the overall performance of complex cross-language question answering.

[0035] Traditional automated evaluation metrics such as BLEU and CHRF primarily rely on n-gram matching, making it difficult to effectively capture translation quality indicators such as semantic consistency, logical completeness, and domain specialization. This is especially true in specialized domains like finance, where subtle deviations in key terms or sentence structure can lead to severe semantic distortion, consequently affecting the reliability of downstream tasks. To address this, one feasible approach is... Figure 2 As shown, based on translation quality verification, the target language question-answer pairs are obtained after translating the source language question-answer pairs, specifically including: The source language question-answer pairs are input into an existing multilingual translation model that supports professional domain translation, so as to translate the source language question-answer pairs into the initial target language question-answer pairs.

[0036] The initial target language question-answer pairs are back-translated, meaning the source language question-answer pairs are translated into the same way as the initial target language question-answer pairs, and then the initial target language question-answer pairs are re-translated back into the source language to obtain back-translated source language question-answer pairs. Semantic confidence is obtained based on the semantic similarity between the source language question-answer pairs and the back-translated source language question-answer pairs. Specifically, in a certain application scenario, a pre-trained multilingual semantic embedding model can be used to vectorize the questions and answers of the two sets of question-answer pairs respectively, and their weighted cosine similarity is calculated. If the overall similarity score reaches or exceeds the preset semantic confidence threshold, the semantic confidence is validated.

[0037] Furthermore, to improve verification quality, this embodiment can introduce one or more pre-set auxiliary quality indicators to determine whether the initial target language question-answer pair passes translation quality verification based on the verification results of semantic confidence and / or the pre-set auxiliary quality indicators. It should be noted that the pre-set auxiliary quality indicators include, but are not limited to, semantic reward scores and perplexity. If the verification is deemed successful based on the verification results of semantic confidence and / or the pre-set auxiliary quality indicators, the initial target language question-answer pair is used as the target language question-answer pair translated from the source language question-answer pair.

[0038] Specifically, determining whether the initial target language question-answer pair passes translation quality verification based on the verification results of semantic confidence and / or pre-set auxiliary quality indicators includes: When the semantic confidence score and all auxiliary quality metrics pass the verification, the initial target language question-answer pair is deemed to have passed the translation quality verification. That is, as follows... Figure 2 As shown in the example of Indonesian in a certain application scenario, to verify the semantic consistency of the translation, the Indonesian translation result, i.e., the initial target language, is first translated back into the source language and compared with the original source language question-and-answer pair. Based on the semantic similarity between the source language question-and-answer pair and the back-translated source language question-and-answer pair, a semantic confidence score is obtained. Then, based on the comparison between the semantic confidence score and a preset semantic confidence threshold, highly aligned initial target language question-and-answer pairs are retained. Next, based on a dedicated reward model, the semantic reasonableness and professionalism of the initial target language question-and-answer pairs selected based on semantic consistency are scored to obtain a semantic reward score. This dedicated reward model can be Seed-X-RM-7B, which can be used to further filter the initial target language question-and-answer pairs after semantic consistency screening. This process comprehensively considers the semantic reward score and semantic confidence score. If the semantic confidence scores of multiple initial target language question-and-answer pairs are all greater than the threshold, the initial target language question-and-answer pair with the higher semantic reward score is prioritized.

[0039] Furthermore, even if semantic rationality is ensured based on the above two processes, problems such as grammatical awkwardness, unnatural expression, and lack of cultural semantics may still exist. Therefore, perplexity is further introduced as a pre-set auxiliary quality indicator to measure language fluency, naturalness, and cultural adaptability. The perplexity of the initial target language question-and-answer pair is calculated by performing an autoregressive scoring on the pair. Here, the autoregressive scoring model is a specialized model for the target language, such as using the Sailor2-8B model to autoregressively score Indonesian translation results. The upper limit of perplexity further filters out translations that are semantically similar but awkwardly expressed or do not conform to native language habits. It can be understood that the lower the perplexity, the more natural and reliable the initial target language question-and-answer pair is. Only when all the above indicators meet the standards is the initial target language question-and-answer pair accepted. This process, through a bidirectional translation alignment mechanism, integrates back-translation consistency verification, semantic reward scoring, and perplexity filtering to achieve translation consistency in the current professional field, ensuring the reliability of cross-language tasks.

[0040] In one feasible approach, the method further includes: if the translation quality verification fails, retranslating the source language question-and-answer pair to obtain at least one retranslated target language question-and-answer pair. The translation quality verification is then performed again on the retranslated target language question-and-answer pair until it passes, thus obtaining a target language question-and-answer pair. If the retranslation still fails to meet all the indicators, the initial target language question-and-answer pair is discarded and manually verified.

[0041] In one feasible approach, to further verify semantic bias, terminological mismatch, or weakened logical relationships, and to prevent users from retrieving biased or incorrect knowledge when asking questions, thus generating incorrect answers, the method further includes: Based on the translation quality verification scores of the translated target language question-and-answer pairs, a subset of translated target language question-and-answer pairs with higher scores is selected as the evaluation subset. The semantic consistency of each target language question-and-answer pair in the evaluation subset when translated back to the source language is verified. If inconsistency is found, root cause analysis is performed on the translated source language answer pair to determine if there is a translation semantic bias. If a translation semantic bias exists, meaning it is confirmed that the model's answer is incorrect due to a translation problem, the target language question-and-answer pair is removed from the evaluation subset. Removing the target language question-and-answer pair ensures that the task logic has not changed due to translation, achieving end-to-end semantic consistency.

[0042] The verification of semantic consistency between the target language question-and-answer pairs in the evaluation subset and their translation back to the source language includes the following steps: The answer texts of each target language question-and-answer pair in the evaluation subset are translated back to obtain the translated source language answers. The translated source language answers are then compared with the answer texts in the source language question-and-answer pairs to determine semantic consistency.

[0043] By selecting and evaluating itself and then performing semantic consistency verification, the translation quality verification was further sampled, audited, and dynamically cleaned. Combined with the translation quality verification, this constituted a multi-level translation quality control, which eliminated hidden semantic deviations and ensured that the end-to-end task logic remained unchanged.

[0044] In one feasible approach, a cross-language knowledge base is constructed based on source language question-answer pairs and target language question-answer pairs, specifically including: The question text in the target language question-answer pair is vectorized to obtain target language question vectors. This vectorization process is accomplished by calling a text embedding model optimized specifically for the target language. For example, if the target language is Indonesian, a sentence embedding model that has been fully pre-trained on Indonesian corpora and may have been fine-tuned in the financial domain is used to convert each target language question into a high-dimensional target language question vector representing its semantics.

[0045] The target language question vector, source language question-answer pairs, and target language question-answer pairs are associated and stored to obtain a cross-language knowledge base. In one approach, this associated storage involves assigning a unique identifier to each knowledge unit and storing the target language question vector, source language question-answer pairs, and target language question-answer pairs in a unified data structure using this identifier as the key. This allows for direct return of the corresponding question and answer content after a vector-based retrieval.

[0046] By associating and storing target language question vectors, source language question-answer pairs, and target language question-answer pairs, a cross-language knowledge base is created. This allows users to ask questions in their native language (target language) by simply vectorizing the question using the same embedding model, then searching the cross-language knowledge base to directly find semantically matching pre-stored target language question vectors, and subsequently retrieving the corresponding bilingual question-answer pairs. This eliminates the need for real-time translation services, improving retrieval speed and system responsiveness. Furthermore, by storing bilingual question-answer pairs in a linked structure, the semantically equivalent source language text can be traced while returning the target language answer, enhancing the interpretability of the response. Moreover, by prioritizing the computationally intensive vectorization and indexing processes during the knowledge base construction phase, the online query service becomes lightweight, requiring only efficient vector retrieval. This decouples offline processing from online services, ensuring high performance and stability for the online service.

[0047] In one feasible approach, the target language question vector, source language question-answer pairs, and target language question-answer pairs are associated and stored to obtain a cross-language knowledge base, including the following processes: While traditional cross-language knowledge bases in the form of question-answer pairs can solve the problem of querying single knowledge points, they lack semantic connections between the queries, making them unsuitable for complex problems requiring multi-hop reasoning. Furthermore, current cross-language question-answering systems often present the same concept in different languages, failing to establish a unified semantic framework. To address this, this feasible approach binds the obtained target language question vector, the associated source language question-answer pairs, and the target language question-answer pairs to construct structured knowledge units. This means assigning a unique identifier to each structured knowledge unit and storing the aforementioned three components as attribute fields, thereby ensuring consistency between the target language question vector, the associated source language question-answer pairs, and the target language question-answer pairs.

[0048] By performing semantic analysis on the source language question-answer pairs within structured knowledge units, key terms in each source language question-answer pair within the structured knowledge unit are identified. These key terms are then associated with corresponding nodes in a pre-defined multilingual knowledge graph of the current professional domain, establishing knowledge graph fragments. These knowledge graph fragments represent the semantic position of the structured knowledge unit within the pre-defined knowledge graph, as well as the relationships between the structured knowledge units. The references between the structured knowledge units and the knowledge graph fragments are then further associated and stored to obtain a cross-language knowledge base.

[0049] This process connects structured knowledge units to corresponding nodes in a multilingual knowledge graph by associating key terms within the structured knowledge units. This enables problem understanding and identification of the necessary knowledge content for association and reasoning when processing user questions. Furthermore, by associating with corresponding nodes in the multilingual knowledge graph, the same technical terms expressed in multiple languages ​​correspond to the same node in the cross-language knowledge base, resolving semantic confusion caused by inconsistent terminology translations and achieving semantic consistency.

[0050] like Figure 3As shown, in a certain application scenario, the construction and optimization process of a cross-language knowledge base can be achieved in the following way: Starting with a question in a professional field, the question is decomposed into multiple atomic sub-questions, and the corresponding answers are extracted to form structured atomic sub-question-answer pairs. These atomic sub-question-answer pairs are then input into a balanced large language model for preliminary understanding and logical organization to obtain standardized atomic sub-question-answer pairs. During the knowledge processing stage, the deep learning large language model is invoked to optimize the standardized atomic sub-question-answer pairs output by the balanced large language model. This optimization involves two aspects: first, assessing the relevance of sub-questions to the content to ensure that each sub-question truly reflects the core intent of the original question; and second, optimizing the expression of the answers to improve their professionalism, accuracy, and fluency. Simultaneously, the multilingual task large language model supports cross-language adaptation, especially when processing data in less commonly spoken professional fields such as Indonesian financial assessment data, achieving unified decomposition and semantic mapping of bilingual questions. Meanwhile, the professional domain evaluation dataset and the target language evaluation dataset are decomposed into problems and then aggregated as benchmark resources for training and validation. This supports model evaluation and iterative optimization in multiple languages ​​and scenarios. Through the division of labor and collaboration among multiple models, the efficient transformation from the original problem to structured knowledge units is achieved, which helps to build a cross-language financial intelligent question answering system with high accuracy and strong robustness.

[0051] Among them, the aforementioned balanced large language model achieves an optimization between reasoning ability, response speed, and cost, and is suitable for medium-complexity tasks such as question answering, text generation, and logical analysis. The deep exploration large language model possesses powerful general language understanding and generation capabilities, performs excellently in mathematical, code, and domain-specific reasoning, supports long context processing, and is widely used in knowledge-intensive tasks. The multilingual task large language model features a parameter-intensive architecture that supports long context lengths, exhibits outstanding performance in programming, mathematical, and multilingual tasks, and is suitable for edge deployment and agent invocation.

[0052] S104: Receive user questions to be processed in the target language, decompose the user questions to be processed to obtain a set of sub-questions, and retrieve knowledge that semantically matches the set of sub-questions from the cross-language knowledge base.

[0053] The system receives user questions in the target language and decomposes them into a set of sub-questions. Knowledge that semantically matches these sub-questions is then retrieved from a specific cross-language knowledge base. In one feasible approach, the user question is broken down into one or more atomic sub-questions, forming a subset. This decomposition process identifies multiple implicit knowledge requirements or logical branches within the user question, aiding in the analysis and understanding of complex questions such as compound, multi-hop, or nested questions. After obtaining the subset, each atomic sub-question is encoded and retrieved from the constructed cross-language knowledge base. Since the target language questions retrieved from the cross-language knowledge base undergo vector processing, semantically matching target language question vectors are obtained by determining the similarity between the vectors of each atomic sub-question and the target language question vectors in the cross-language knowledge base. The source language question-answer pairs associated with these semantically matching target language question vectors are then returned as knowledge matching the sub-question set. By decoupling complex user questions into atomic sub-questions for allocation, retrieval failures caused by complex question descriptions or missing keywords are effectively avoided, significantly improving the accuracy and robustness of cross-language question answering in real business scenarios.

[0054] S106: Generate a response in the current target language based on the knowledge, thus completing cross-language understanding.

[0055] After acquiring knowledge that semantically matches the set of sub-questions, an answer in the current target language is generated based on this knowledge, thus completing cross-language understanding. Specifically, in one feasible approach, after acquiring knowledge that semantically matches the set of sub-questions, this semantically matched knowledge is integrated. If the user question to be processed is broken down into multiple atomic sub-questions, each sub-question corresponds to one or more structured knowledge units. At this point, the corresponding structured knowledge units are sorted according to the logical relationships between the sub-questions to obtain integrated knowledge content. The large language model is then invoked to provide an answer based on this integrated knowledge content, thereby guiding the large language model to answer based on an expression format consistent with the target language through prompting engineering. If no high-confidence knowledge is retrieved or matched for a sub-question, the context of the sub-question is considered to determine whether to omit the sub-question, or to generate supplementary explanations with low-confidence markers to generate an answer in the current target language, thus completing cross-language understanding.

[0056] In another feasible approach, after obtaining knowledge that semantically matches the set of sub-problems, the associated knowledge graph fragments can be invoked simultaneously. By traversing the semantic relationships in the preset multilingual knowledge graph, the logical relationships between each sub-problem can be deduced. Based on the logical relationships, the causal path corresponding to each sub-problem can be determined. Based on the causal path, a standardized answer template is extracted from the preset multilingual knowledge graph, and the obtained knowledge is filled into the slots of the answer template to generate the answer in the current target language, thus completing cross-language understanding.

[0057] In practice, users in certain regions often mix their local language with technical terms from English or other languages. If the user's question is forcibly treated as a single language (the local language), it may lead to the loss of terminology or failed searches. Therefore, to respect users' language habits and ensure that technical terms are not misunderstood during searches, avoiding incorrect answers due to ambiguity, a feasible approach includes the following process: When the user problem to be processed is a mixed language input, the mixed language input is decomposed to obtain a set of sub-problems including mixed language sub-problems. The problem decomposition process can be carried out based on the feasible methods proposed above, and the problem decomposition process will not be described here.

[0058] It should be noted that the mixed-language input is based on at least two languages. The target language of this mixed-language input is the principal component language among the at least two languages, determined based on their respective language proportions. This principal component language can be determined after language recognition of the mixed-language input to separate language segments from at least two languages, based on the length of each segment, and then the principal component language in the mixed-language input is determined according to these proportions. If the proportions are the same, the user system language or a preset priority language will prevail. Furthermore, the aforementioned mixed-language sub-problem is a sub-problem based on at least two languages.

[0059] Retrieve non-target language terms from any mixed-language subproblem. These non-target language terms have standardized definitions within the current professional domain. For example, in a scenario where the template language for the mixed-language input is Indonesian, certain Indonesian statements contain non-target language terms from current professional domains, such as the financial field's "CAR (Capital Adequacy Ratio)." Although these non-target terms are not in the target language, they have a standardized definition of "capital adequacy ratio" in the financial field.

[0060] Terminological constraints are determined based on non-target language terms to ensure that the question text of the target language question-answer pair associated with the knowledge contains the same terms as the non-target language terms. These same terms refer to terms that are consistent or semantically equivalent. For example, if the user's mixed-language input contains "CAR," then the terminology constraint requires the target language question to contain the same terms such as "capital adequacy ratio," "CAR," or "Capital Adequacy Ratio." By incorporating terminology constraints, the retrieval drift problem caused by cluttered terminology is effectively avoided. After determining the corresponding terminology constraints, the cross-language knowledge base is filtered according to these constraints to obtain matching knowledge.

[0061] Based on the same idea, one or more embodiments of this specification also provide apparatus and devices corresponding to the above methods, such as... Figure 4 , Figure 5 As shown.

[0062] Figure 4 This specification provides a schematic diagram of the structure of a cross-language understanding device according to one or more embodiments, the device comprising: A cross-language knowledge base determination unit 402 is used to determine a pre-constructed cross-language knowledge base; the construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, combining it with external authoritative information to obtain source language question-answer pairs; obtaining target language question-answer pairs translated from the source language question-answer pairs based on translation quality verification; the translation quality verification includes at least: bidirectional semantic alignment verification; and constructing a cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs. The question retrieval unit 404 is used to receive user questions to be processed in the target language, decompose the user questions to be processed to obtain a set of sub-questions, and retrieve knowledge that semantically matches the set of sub-questions from the cross-language knowledge base. The generation unit 406 is used to generate a response in the current target language based on the knowledge, thereby completing cross-language understanding.

[0063] Optionally, the cross-language knowledge base determination unit 402 includes: a question-answer pair construction unit 4021; The question-and-answer pair construction unit 4021 is used to decompose the original source language data of the current professional field into multiple atomic sub-questions; The large language model searches for external authoritative information corresponding to the atomic sub-problem, so that the large language model generates the corresponding source language answer based on the atomic sub-problem and the external authoritative information; The atomic sub-questions and their corresponding source language answers constitute source language question-answer pairs.

[0064] Optionally, the question-and-answer pair construction unit 4021 specifically includes: Based on the atomic subproblem, the large language model is invoked to obtain external authoritative information related to the atomic subproblem in real time; If the external authoritative information is not in the source language, then the non-source language terms in the external authoritative information are translated to obtain source language terms; Based on the source language terms and their contextual information, a source language answer corresponding to the atomic sub-problem is generated.

[0065] Optionally, the cross-language knowledge base determination unit 402 includes: a translation verification unit 4022; The translation verification unit 4022 is used to translate the source language question-answer pair into an initial target language question-answer pair. The initial target language question-answer pair is back-translated to obtain the back-translated source language question-answer pair, and the semantic confidence is obtained based on the semantic similarity between the source language question-answer pair and the back-translated source language question-answer pair. Based on the verification results of the semantic confidence and / or the preset auxiliary quality indicators, it is determined whether the initial target language question-answer pair passes the translation quality verification; If successful, the initial target language question-answer pair will be used as the target language question-answer pair translated from the source language question-answer pair.

[0066] Optionally, the translation verification unit 4022 specifically includes: When the semantic confidence score and the verification results of each of the auxiliary quality indicators are both passed, the initial target language question-answer pair is determined to have passed the translation quality verification.

[0067] Optionally, the apparatus further includes: an iterative retranslation unit 4023; The iterative retranslation unit 4023 is used to retranslate the source language question-answer pair if the translation quality verification fails, to obtain at least one retranslated target language question-answer pair. The translation quality of the retranslated target language question-and-answer pairs is verified until a target language question-and-answer pair that passes the translation quality verification is obtained.

[0068] Optionally, the cross-language knowledge base determination unit 402 includes: a construction unit 4024; The construction unit 4024 is used to vectorize the question text in the target language question-answer pair to obtain the target language question vector. The target language question vector, the source language question-answer pair, and the target language question-answer pair are associated and stored to obtain a cross-language knowledge base.

[0069] Optionally, the building unit 4024 specifically includes: Based on the target language question vector, the source language question-answer pair associated with the target language question vector, and the target language question-answer pair, a structured knowledge unit is constructed; Identify key terms in each source language question-answer pair within the structured knowledge unit, associate the key terms with corresponding nodes in a pre-defined multilingual knowledge graph of the current professional field, and establish a knowledge graph fragment; the knowledge graph fragment is used to represent the semantic position of the structured knowledge unit in the pre-defined knowledge graph, and the relationships between the structured knowledge units; The structured knowledge units are associated with the knowledge graph fragments and stored together to obtain a cross-language knowledge base.

[0070] Optionally, the question retrieval unit 404 specifically includes: Obtain the sub-question vectors from the sub-question set, and retrieve the target language question vectors that semantically match the sub-question vectors in the cross-language knowledge base; Obtain the source language question-answer pairs associated with the semantically matched target language question vector, and return the associated source language question-answer pairs as knowledge.

[0071] Optionally, the device further includes: a screening unit 4025; The filtering unit 4025 is used to select an evaluation subset based on the translated target language question-answer pairs, and to verify the semantic consistency of each target language question-answer pair in the evaluation subset when translated back to the source language. If there is a discrepancy, a root cause analysis is performed on the back-translated source language response pair to determine whether there is a semantic bias in the translation. If so, then delete the target language question-answer pair from the evaluation subset.

[0072] Optionally, the filtering unit 4025 specifically includes: The answer text of each target language question-answer pair in the evaluation subset is back-translated to obtain the back-translated source language answer; By comparing the translated source language answer with the answer text in the source language question-answer pair, it is determined whether the semantics are consistent.

[0073] Optionally, the device further includes: a mixed language processing unit 408; The mixed language processing unit 408 is configured to, if the user's question to be processed is a mixed language input, decompose the mixed language input to obtain a set of sub-questions including mixed language sub-questions; wherein the mixed language input is based on at least two languages; and the target language of the mixed language input is the principal component language among the at least two languages ​​determined based on the language proportion. Obtain non-target language terms from any of the aforementioned mixed language sub-problems; wherein, the non-target language terms have standardized definitions in the current professional field; Based on the non-target language terms, corresponding term constraints are determined to filter the cross-language knowledge base according to the term constraints and obtain matching knowledge; the term constraints are used to constrain the question text of the target language question-answer pair associated with the knowledge to contain the same terms as the non-target language terms.

[0074] Figure 5 A schematic diagram of a cross-language understanding device provided for one or more embodiments of this specification, the device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: A pre-constructed cross-language knowledge base is determined; the construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, combining it with external authoritative information to obtain source language question-answer pairs; based on translation quality verification, obtaining target language question-answer pairs translated from the source language question-answer pairs; the translation quality verification includes at least: bidirectional semantic alignment verification; and constructing the cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs. Receive user questions to be processed in the target language, break down the user questions to be processed to obtain a set of sub-questions, and retrieve knowledge that semantically matches the set of sub-questions from the cross-language knowledge base; Based on the knowledge provided, a response in the target language is generated, thus achieving cross-language understanding.

[0075] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog are commonly used. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0076] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0077] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0078] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0079] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0084] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0085] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0086] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0087] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0088] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, and therefore described more simply; relevant parts can be referred to the descriptions of the method embodiments.

[0089] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0090] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A cross-language understanding method, the method comprising: Identify a pre-built cross-language knowledge base; The construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, and combining it with external authoritative information to obtain source language question-answer pairs; Based on translation quality verification, the target language question-answer pairs translated from the source language question-answer pairs are obtained; The translation quality verification includes at least: bidirectional semantic alignment verification; and constructing a cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs. Receive user questions to be processed in the target language, break down the user questions to be processed to obtain a set of sub-questions, and retrieve knowledge that semantically matches the set of sub-questions from the cross-language knowledge base; Based on the knowledge provided, a response in the target language is generated, thus achieving cross-language understanding.

2. The method as described in claim 1, which decomposes the original source language data in the current professional field to obtain source language question-answer pairs by combining it with external authoritative information, specifically includes: The original source language data in the current professional field is decomposed into multiple atomic sub-problems; The large language model searches for external authoritative information corresponding to the atomic sub-problem, so that the large language model generates the corresponding source language answer based on the atomic sub-problem and the external authoritative information. The atomic sub-questions and their corresponding source language answers constitute source language question-answer pairs.

3. The method as described in claim 2, wherein the external authoritative information corresponding to the atomic sub-problem is searched based on a large language model, so that the large language model generates a corresponding source language answer based on the atomic sub-problem and the external authoritative information, specifically includes: Based on the atomic subproblem, the large language model is invoked to obtain external authoritative information related to the atomic subproblem in real time; If the external authoritative information is not in the source language, then the non-source language terms in the external authoritative information are translated to obtain source language terms; Based on the source language terms and their contextual information, a source language answer corresponding to the atomic sub-problem is generated.

4. The method as described in claim 1, based on translation quality verification, obtains the target language question-answer pair translated from the source language question-answer pair, specifically including: Translate the source language question-answer pairs into initial target language question-answer pairs; The initial target language question-answer pair is back-translated to obtain the back-translated source language question-answer pair, and the semantic confidence is obtained based on the semantic similarity between the source language question-answer pair and the back-translated source language question-answer pair; Based on the verification results of the semantic confidence and / or the preset auxiliary quality indicators, determine whether the initial target language question-answer pair passes the translation quality verification; If successful, the initial target language question-answer pair will be used as the target language question-answer pair translated from the source language question-answer pair.

5. The method as described in claim 4, wherein the initial target language question-answer pair is determined to pass the translation quality verification based on the verification results of the semantic confidence and / or preset auxiliary quality indicators, specifically including: When the semantic confidence score and the verification results of each of the auxiliary quality indicators are both passed, the initial target language question-answer pair is determined to have passed the translation quality verification.

6. The method according to claim 4, further comprising: If the translation quality verification fails, the source language question-answer pair is retranslated to obtain at least one retranslated target language question-answer pair. The translation quality of the retranslated target language question-and-answer pairs is verified until a target language question-and-answer pair that passes the translation quality verification is obtained.

7. The method as described in claim 1, wherein a cross-language knowledge base is constructed based on the source language question-answer pair and the target language question-answer pair, specifically including: The question text in the target language question-answer pair is vectorized to obtain the target language question vector; The target language question vector, the source language question-answer pair, and the target language question-answer pair are associated and stored to obtain a cross-language knowledge base.

8. The method as described in claim 7, wherein the target language question vector, the source language question-answer pair, and the target language question-answer pair are associated and stored to obtain a cross-language knowledge base, specifically including: Based on the target language question vector, the source language question-answer pair associated with the target language question vector, and the target language question-answer pair, a structured knowledge unit is constructed; Identify key terms in each source language question-answer pair within the structured knowledge unit, associate the key terms with corresponding nodes in a pre-defined multilingual knowledge graph of the current professional field, and establish a knowledge graph fragment; the knowledge graph fragment is used to represent the semantic position of the structured knowledge unit in the pre-defined knowledge graph, and the relationships between the structured knowledge units; The structured knowledge units are associated with the knowledge graph fragments and stored together to obtain a cross-language knowledge base.

9. The method of claim 7, wherein retrieving knowledge semantically matching the sub-question set from the cross-language knowledge base specifically includes: Obtain the sub-question vectors from the sub-question set, and retrieve the target language question vectors that semantically match the sub-question vectors in the cross-language knowledge base; Obtain the source language question-answer pairs associated with the semantically matched target language question vector, and return the associated source language question-answer pairs as knowledge.

10. The method of claim 1, further comprising: An evaluation subset is selected based on the translated target language question-answer pairs, and the semantic consistency of each target language question-answer pair in the evaluation subset is verified by back-translating it into the source language. If there is a discrepancy, a root cause analysis is performed on the back-translated source language response pair to determine whether there is a semantic bias in the translation. If so, then delete the target language question-answer pair from the evaluation subset.

11. The method of claim 10, wherein verifying the semantic consistency of each target language question-answer pair in the evaluation subset when translated back to the source language, specifically includes: The answer text of each target language question-answer pair in the evaluation subset is back-translated to obtain the back-translated source language answer; By comparing the translated source language answer with the answer text in the source language question-answer pair, it is determined whether the semantics are consistent.

12. The method of claim 1, further comprising: If the user problem to be processed is a mixed-language input, then the mixed-language input is decomposed to obtain a set of sub-problems including mixed-language sub-problems; wherein, the mixed-language input is based on at least two languages; the target language of the mixed-language input is the principal component language among the at least two languages ​​determined based on the language proportion. Obtain non-target language terms from any of the aforementioned mixed language sub-problems; wherein, the non-target language terms have standardized definitions in the current professional field; Based on the non-target language terms, corresponding term constraints are determined to filter the cross-language knowledge base according to the term constraints and obtain matching knowledge; the term constraints are used to constrain the question text of the target language question-answer pair associated with the knowledge to contain the same terms as the non-target language terms.

13. A cross-language understanding device, comprising: Cross-language knowledge base determination unit, used to determine a pre-built cross-language knowledge base; The construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, and combining it with external authoritative information to obtain source language question-answer pairs; Based on translation quality verification, the target language question-answer pairs translated from the source language question-answer pairs are obtained; The translation quality verification includes at least: bidirectional semantic alignment verification; and constructing a cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs. The problem retrieval unit is used to receive user problems to be processed in the target language, decompose the user problems to be processed to obtain a set of sub-problems, and retrieve knowledge that semantically matches the set of sub-problems from the cross-language knowledge base; The generation unit is used to generate a response in the current target language based on the knowledge, thereby completing cross-language understanding.

14. The apparatus of claim 13, wherein the cross-language knowledge base determination unit comprises: Question-answer pair building blocks; The question-and-answer pair construction unit is used to decompose the original source language data of the current professional field into multiple atomic sub-questions; The large language model searches for external authoritative information corresponding to the atomic sub-problem, so that the large language model generates the corresponding source language answer based on the atomic sub-problem and the external authoritative information. The atomic sub-questions and their corresponding source language answers constitute source language question-answer pairs.

15. The apparatus of claim 14, wherein the question-answer pair construction unit specifically comprises: Based on the atomic subproblem, the large language model is invoked to obtain external authoritative information related to the atomic subproblem in real time; If the external authoritative information is not in the source language, then the non-source language terms in the external authoritative information are translated to obtain source language terms; Based on the source language terms and their contextual information, a source language answer corresponding to the atomic sub-problem is generated.

16. The apparatus of claim 13, wherein the cross-language knowledge base determination unit comprises: Translation verification unit; The translation verification unit is used to translate the source language question-answer pair into an initial target language question-answer pair. The initial target language question-answer pair is back-translated to obtain the back-translated source language question-answer pair, and the semantic confidence is obtained based on the semantic similarity between the source language question-answer pair and the back-translated source language question-answer pair; Based on the verification results of the semantic confidence and / or the preset auxiliary quality indicators, determine whether the initial target language question-answer pair passes the translation quality verification; If successful, the initial target language question-answer pair will be used as the target language question-answer pair translated from the source language question-answer pair.

17. The apparatus of claim 16, wherein the translation verification unit specifically comprises: When the semantic confidence score and the verification results of each of the auxiliary quality indicators are both passed, the initial target language question-answer pair is determined to have passed the translation quality verification.

18. The apparatus of claim 16, further comprising: Iterative retranslation unit; The iterative retranslation unit is used to retranslate the source language question-answer pair if the translation quality verification fails, thereby obtaining at least one retranslated target language question-answer pair. The translation quality of the retranslated target language question-and-answer pairs is verified until a target language question-and-answer pair that passes the translation quality verification is obtained.

19. The apparatus of claim 13, wherein the cross-language knowledge base determination unit comprises: Building blocks; The construction unit is used to vectorize the question text in the target language question-answer pair to obtain the target language question vector. The target language question vector, the source language question-answer pair, and the target language question-answer pair are associated and stored to obtain a cross-language knowledge base.

20. The apparatus of claim 19, wherein the building unit specifically comprises: Based on the target language question vector, the source language question-answer pair associated with the target language question vector, and the target language question-answer pair, a structured knowledge unit is constructed; Identify key terms in each source language question-answer pair within the structured knowledge unit, associate the key terms with corresponding nodes in a pre-defined multilingual knowledge graph of the current professional field, and establish a knowledge graph fragment; the knowledge graph fragment is used to represent the semantic position of the structured knowledge unit in the pre-defined knowledge graph, and the relationships between the structured knowledge units; The structured knowledge units are associated with the knowledge graph fragments and stored together to obtain a cross-language knowledge base.

21. The apparatus of claim 19, wherein the question retrieval unit specifically comprises: Obtain the sub-question vectors from the sub-question set, and retrieve the target language question vectors that semantically match the sub-question vectors in the cross-language knowledge base; Obtain the source language question-answer pairs associated with the semantically matched target language question vector, and return the associated source language question-answer pairs as knowledge.

22. The apparatus of claim 13, further comprising: Filtering unit; The filtering unit is used to select an evaluation subset based on the translated target language question-answer pairs, and to verify the semantic consistency of each target language question-answer pair in the evaluation subset when translated back to the source language. If there is a discrepancy, a root cause analysis is performed on the back-translated source language response pair to determine whether there is a semantic bias in the translation. If so, then delete the target language question-answer pair from the evaluation subset.

23. The apparatus of claim 22, wherein the screening unit specifically comprises: The answer text of each target language question-answer pair in the evaluation subset is back-translated to obtain the back-translated source language answer; By comparing the translated source language answer with the answer text in the source language question-answer pair, it is determined whether the semantics are consistent.

24. The apparatus of claim 13, further comprising: Mixed language processing unit; The mixed language processing unit is configured to, if the user's question to be processed is a mixed language input, decompose the mixed language input to obtain a set of sub-questions including mixed language sub-questions; wherein the mixed language input is based on at least two languages; and the target language of the mixed language input is the principal component language among the at least two languages ​​determined based on the language proportion. Obtain non-target language terms from any of the aforementioned mixed language sub-problems; wherein, the non-target language terms have standardized definitions in the current professional field; Based on the non-target language terms, corresponding term constraints are determined to filter the cross-language knowledge base according to the term constraints and obtain matching knowledge; the term constraints are used to constrain the question text of the target language question-answer pair associated with the knowledge to contain the same terms as the non-target language terms.

25. A cross-language understanding device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: A pre-constructed cross-language knowledge base is determined; the construction of the cross-language knowledge base involves: decomposing the original source language data of the current professional field, combining it with external authoritative information to obtain source language question-answer pairs; based on translation quality verification, obtaining target language question-answer pairs translated from the source language question-answer pairs; the translation quality verification includes at least: bidirectional semantic alignment verification; and constructing the cross-language knowledge base based on the source language question-answer pairs and the target language question-answer pairs. Receive user questions to be processed in the target language, break down the user questions to be processed to obtain a set of sub-questions, and retrieve knowledge that semantically matches the set of sub-questions from the cross-language knowledge base; Based on the knowledge provided, a response in the target language is generated, thus achieving cross-language understanding.

Images