A multilingual corpus alignment method, device, equipment and medium
Through the selection of standard language in multi -language text processing and translation and alignment of other language texts, the problems that are difficult to achieve multi -language alignment in existing technologies have been solved. Align.
Patent Information
- Application Number
- CN202111612435.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2025-01-28
- Estimated Expiration
- 2041-12-27
AI Technical Summary
It is difficult to achieve corpus alignment between three and more languages in the prior art.
By obtaining the original text and the identification of language, select a language as a standard language, translate the text of other types of text into standard language, and align all the standard texts to obtain the results of corpus alignment.
The corpus alignment between three and more than three types of languages is achieved, and the efficiency and accuracy of multi -language processing is improved.
Smart Images

Figure CN114282554B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a multilingual corpus alignment method, device, equipment and medium. Background Art
[0002] In recent years, corpus alignment has been widely used in AI language processing technology. Traditional corpus alignment refers to achieving one-to-one correspondence between sentences or paragraphs in documents of two languages to form standardized language pairs. However, there is a lack of research on corpus alignment of three or more languages. Summary of the invention
[0003] The technical problem to be solved by the present invention is how to achieve corpus alignment between three or more languages. Therefore, the present invention provides a multilingual corpus alignment method, device, equipment and medium to achieve corpus alignment between three or more languages.
[0004] The present invention is achieved through the following technical solutions:
[0005] A multilingual corpus alignment method, comprising:
[0006] Obtaining the original text, and performing language recognition on the original text through language recognition technology, and obtaining the language of the original text as the target language;
[0007] Selecting one language from the target languages as a standard language, and selecting the remaining languages from the target languages as languages to be translated;
[0008] The original text corresponding to the language to be translated is translated into the standard text of the standard language, and all the standard texts are aligned and recognized to obtain the corpus alignment result.
[0009] Furthermore, the selecting a language from the target languages as the standard language includes:
[0010] The language that appears most frequently in the target language is determined as the standard language; or
[0011] Select a language from the target languages that matches your translation needs as the standard language.
[0012] Furthermore, the aligning and identifying all standard texts to obtain corpus alignment results includes:
[0013] Preprocess all standard texts to obtain the words to be processed of each standard text;
[0014] Determine the meaning relationship and position relationship of the to-be-processed words in each standard text. If the meaning relationship of the to-be-processed words in different standard texts is consistent and they are in the same position, then it is considered that the to-be-processed words are aligned.
[0015] The number of aligned words to be processed is counted. When the number of aligned words to be processed meets the preset condition, the corresponding corpus alignment result is corpus matching.
[0016] When the number of aligned words to be processed does not meet the preset condition, the corresponding corpus alignment result is corpus mismatch.
[0017] Furthermore, the preprocessing of all standard texts to obtain the words to be processed of each standard text includes:
[0018] Segment each of the standard texts according to the segmentation marks to obtain sentences of each standard text;
[0019] Sentences in different standard texts are segmented and stop words are removed to obtain the words to be processed for each standard text.
[0020] Furthermore, each sentence carries a sentence sequence number;
[0021] Determining the meaning relationship and position relationship of the words to be processed in each standard text includes:
[0022] Confirm the meaning relationship of all the words to be processed with the same sentence sequence number in different standard texts. When the meaning relationship is consistent, determine the position relationship of the corresponding words to be processed.
[0023] When the positions of the words to be processed with the same meaning relationship in their respective sentences are consistent, it means that the words to be processed with the same meaning relationship are in the same position in their respective sentences;
[0024] When the positions of the to-be-processed words with the same meaning relationship in their respective sentences are inconsistent, it means that the to-be-processed words with the same meaning relationship are in different positions in their respective sentences.
[0025] Furthermore, the confirmation of the meaning relationship of all the to-be-processed words with the same sentence sequence number in different standard texts includes:
[0026] When the sequence numbers of the words to be processed in the same sentence in different standard texts are consistent or are synonyms or near synonyms, it means that the meanings of the words to be processed are consistent;
[0027] When the pending words with the same sentence sequence number in different standard texts are not synonyms or synonyms of each other, it means that the meaning relationship of the pending words is inconsistent.
[0028] A multilingual corpus alignment device, comprising:
[0029] The text language determination module is used to obtain the original text and identify the language of the original text through language recognition technology to obtain the language of the original text as the target language;
[0030] A text language classification module, used to select a language from the target language as a standard language, and select the remaining languages in the target language as languages to be translated;
[0031] The corpus alignment judgment module is used to translate the original text corresponding to the language to be translated into the standard text of the standard language, and align and identify all the standard texts to obtain the corpus alignment result.
[0032] Furthermore, the corpus alignment judgment module includes:
[0033] A text preprocessing unit is used to preprocess all standard texts to obtain the words to be processed of each standard text;
[0034] The to-be-processed word alignment judgment unit is used to determine the meaning relationship and position relationship of the to-be-processed words in each standard text. If the meaning relationship of the to-be-processed words in different standard texts is consistent and they are in the same position, then it is considered that the to-be-processed words are aligned;
[0035] The first corpus alignment judgment unit is used to count the number of aligned words to be processed. When the number of aligned words to be processed meets a preset condition, the corresponding corpus alignment result is corpus matching;
[0036] The second corpus alignment judgment unit is used to determine that when the number of aligned to-be-processed words does not meet a preset condition, the corresponding corpus alignment result is that the corpus does not match.
[0037] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the multilingual corpus alignment method when executing the computer program.
[0038] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the multilingual corpus alignment method is implemented.
[0039] The present invention provides a multilingual corpus alignment method, device, equipment and medium, which obtains original text and performs language recognition on the original text through language recognition technology to obtain the language of the original text as the target language; selects a language from the target language as the standard language, and uses the remaining languages in the target language as the languages to be translated; translates the original text corresponding to the language to be translated into a standard text of the standard language, and performs alignment recognition on all standard texts to obtain a corpus alignment result, so as to achieve corpus alignment between three or more languages. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:
[0041] Figure 1 The present invention is a flowchart of a multilingual corpus alignment method.
[0042] Figure 2 for Figure 1 A specific flow chart of step S30 in FIG.
[0043] Figure 3 for Figure 3 A specific flow chart of step S31 in FIG.
[0044] Figure 4 for Figure 3 A specific flow chart of step S32 in FIG.
[0045] Figure 5 The figure is a schematic diagram of the structure of a multilingual corpus alignment device of the present invention.
[0046] Figure 6 A schematic diagram of a computer device of the present invention. DETAILED DESCRIPTION
[0047] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.
[0048] The present invention provides a multilingual corpus alignment method, which is not only applicable to the alignment of corpora of three or more languages, but also applicable to the alignment of corpora of two languages. The method can be applied to different electronic devices, including but not limited to various personal computers, laptops, smart phones and tablet computers.
[0049] In one embodiment, if Figure 1 As shown, the present invention provides a multilingual corpus alignment method, comprising:
[0050] S10: Obtaining the original text, and performing language recognition on the original text through language recognition technology to obtain the language of the original text as the target language.
[0051] S20: Selecting a language from the target languages as the standard language, and selecting the remaining languages in the target languages as the languages to be translated.
[0052] S30: Translate the original text corresponding to the language to be translated into a standard text of the standard language, and perform alignment recognition on all the standard texts to obtain a corpus alignment result.
[0053] The original text refers to text data that expresses the same text content in different languages. In this embodiment, the original text is obtained by, but is not limited to, obtaining it from historically stored multilingual texts, crawling it using a crawler tool, and inputting it through a client.
[0054] As an example, in step S10, after obtaining the original text, the original text is recognized by language recognition technology, and the language of the original text is obtained as the target language. Since the data of the original text contains different languages, this embodiment uses language recognition technology to recognize the original text, and text data in at least two languages can be obtained.
[0055] As an example, in step S20, after determining the language of the original text, that is, the target language, a language is selected from the target language as the standard language, and the remaining languages in the target language are used as the languages to be translated, so that text data in different languages can be translated into the same language, which is convenient for subsequent alignment recognition. In this embodiment, selecting a language from the target language as the standard language can determine the language that appears most frequently in the target language as the standard language; or selecting a language from the target language that matches the translation requirements as the standard language.
[0056] Furthermore, if the language in actual demand does not exist in the target language, the language in actual demand is used as the standard language.
[0057] As an example, in step S30, although the original text includes text data in different languages, the text content is the same, but translated into different languages. Therefore, in order to align and identify the text data in different languages, the text data in different languages need to be uniformly translated into a standard language to form a standard text. After obtaining the standard text, all standard texts are aligned and identified to obtain the corpus alignment result.
[0058] The present embodiment provides a multilingual corpus alignment method, which achieves corpus alignment between two languages, three languages, or more than three languages by translating text data in different languages in the original text into text corresponding to a standard language.
[0059] In one embodiment, if Figure 2 As shown, in step S30, all standard texts are aligned and recognized to obtain corpus alignment results, which specifically includes the following steps:
[0060] S31: Preprocess all standard texts to obtain the words to be processed of each standard text.
[0061] S32: Determine the meaning relationship and position relationship of the to-be-processed words in each standard text. If the meaning relationship of the to-be-processed words in different standard texts is consistent and is in the same position, it is considered that the to-be-processed words are aligned.
[0062] S33: Counting the number of aligned words to be processed. When the number of aligned words to be processed meets a preset condition, the corresponding corpus alignment result is corpus matching.
[0063] S34: When the number of aligned words to be processed does not meet the preset condition, the corresponding corpus alignment result is corpus mismatch.
[0064] The words to be processed refer to the words that need to be aligned after preprocessing the standard text, and the words to be processed do not include stop words.
[0065] The preset condition refers to the condition for judging whether the number of aligned pending words meets the requirement. The preset condition in this embodiment can be limited according to the actual situation and is not limited here. For example, it can be set to the number of aligned pending words (10), or the proportion of the number of aligned pending words to all pending words in the sentence (more than 70%).
[0066] As an example, in step S31, in order to achieve corpus alignment between different languages, it is first necessary to preprocess all the obtained standard texts, divide all the standard texts into texts with words as units, and obtain the to-be-processed words of each standard text.
[0067] As an example, in step S32, since the original text only expresses the same text content in different languages, after the original text is translated into the standard text, in theory, the order of paragraphs in the text, the order of sentences in each paragraph, and the order of words in each sentence should be the same. Therefore, after obtaining the to-be-processed words corresponding to each standard text, this embodiment determines the corpus alignment result of the original text by determining whether the meaning relationship and position relationship of the to-be-processed words in each standard text are consistent. If the meaning relationship of the to-be-processed words in different standard texts is consistent and is in the same position, it is considered that the to-be-processed words are aligned. If the meaning relationship of the to-be-processed words in different standard texts is inconsistent and / or the positions are different, it is considered that the to-be-processed words are not aligned.
[0068] Because a word may appear multiple times in a sentence, and the meaning expressed each time it appears is not the same. For example, although the word "meaning" is the "meaning" in the sentence "I mean that this is not what I meant", the meanings of "meaning" in different positions are not the same. Therefore, in determining whether the pending words in different standard texts are aligned, not only the meaning relationship needs to be consistent, but also the position in the sentence needs to be the same in order to more accurately determine whether the pending words in different standard texts are aligned.
[0069] Through step S32, the accuracy of determining whether the words to be processed are aligned can be further improved.
[0070] As an example, in steps S33-S34, if the number of aligned words to be processed is small (such as only one or two), or the proportion is small (such as only 20%), it is impossible to consider that the corpora in different standard texts match. Therefore, after determining the alignment of the words to be processed in different standard texts, it is also necessary to count the number of aligned words to be processed. Only when the number of aligned words to be processed meets the preset conditions, the corpora in different standard texts are considered to match. If the number of aligned words to be processed does not meet the preset conditions, it means that the corpora in different standard texts do not match.
[0071] In one embodiment, if Figure 3 As shown, step S31 pre-processes all standard texts to obtain the words to be processed of each standard text, which specifically includes the following steps:
[0072] S311: Segment each standard text according to the segmentation marks to obtain sentences of each standard text.
[0073] S312: Segment the sentences in different standard texts and remove stop words to obtain the words to be processed for each standard text.
[0074] The segmentation mark refers to a mark that can divide the text content into sentences. The segmentation mark in this embodiment is an identifier that can segment the text content in any language, and is not limited here. Taking Chinese as an example, the segmentation mark can be a punctuation mark ".", "!", "?", ":", or other forms of identifiers.
[0075] As an example, in steps S311-S312, each standard text is first segmented according to the segmentation marks, and the text content in each standard text is processed into sentences. Then, a word segmentation tool is used to segment the sentences in different standard texts and remove stop words to obtain the words to be processed for each standard text.
[0076] In one embodiment, since the text content of the original text is the same, after being translated into the standard text, the sentence order in the text content corresponding to each language should be consistent in theory. Therefore, by setting a sentence sequence number for each sentence, the position of the sentence in the text content can be determined. After determining the position of the sentence, since a word to be processed may appear in multiple sentences, in order to improve the accuracy of the alignment of the word to be processed, this embodiment sets a sentence sequence number for each sentence to facilitate the identification of the order of the sentences in which the word to be processed in the text content is located, such as Figure 4 As shown, in step S32, determining the meaning relationship and position relationship of the words to be processed in each standard text specifically includes the following steps:
[0077] S321: Confirm the meaning relationship of all the words to be processed with the same sentence sequence number in different standard texts. When the meaning relationship is consistent, determine the position relationship of the corresponding words to be processed.
[0078] S322: When the positions of the to-be-processed words with the same meaning relationship in their respective sentences are consistent, it means that the to-be-processed words with the same meaning relationship are in the same position in their respective sentences.
[0079] S323: When the positions of the to-be-processed words with the same meaning relationship in their respective sentences are inconsistent, it means that the to-be-processed words with the same meaning relationship are in different positions in their respective sentences.
[0080] As an example, in step S321, when the to-be-processed words with the same sentence sequence numbers in different standard texts are consistent or are synonyms or near synonyms, it means that the meanings of the to-be-processed words are consistent; when the to-be-processed words with the same sentence sequence numbers in different standard texts are not synonyms or near synonyms, it means that the meanings of the to-be-processed words are inconsistent.
[0081] As an example, in step S322, when the sentences where the to-be-processed words with consistent semantic relationship are located have the same sentence sequence numbers and are located in the same positions in their respective sentences, it means that the to-be-processed words with consistent semantic relationship are located in the same positions in their respective sentences.
[0082] As an example, in step S323, when the sentences containing the to-be-processed words with consistent semantic relationships have different sentence sequence numbers, or the sentences containing the to-be-processed words with consistent semantic relationships have the same sentence sequence numbers but their positions in their respective sentences are inconsistent, it means that the to-be-processed words with consistent semantic relationships are in different positions in their respective sentences.
[0083] Step S321 to step S323, based on the meaning relationship and position relationship of the to-be-processed words in different standard texts, the accuracy of judging whether the to-be-processed words are aligned can be improved.
[0084] The present invention provides a multilingual corpus alignment method, which determines a standard language, translates an original text corresponding to a language to be translated into a standard text, and then determines the meaning relationship and position relationship of the to-be-processed words in each standard text. When the to-be-processed words in each standard text are consistent or are synonyms or near synonyms and have the same sentence sequence number and consistent position in the sentence, it indicates that the to-be-processed words are aligned. Finally, the number of the to-be-processed words is counted. When the number of the to-be-processed words meets a preset condition, it is considered that the corpus in the original text is matched, so as to achieve corpus alignment between three or more languages.
[0085] It should be noted that the present invention can achieve corpus alignment between three or more languages through the above steps, but is not limited to achieving corpus alignment between three or more languages, and can also achieve corpus alignment between two languages.
[0086] In one embodiment, a multilingual corpus alignment device is provided, and the multilingual corpus alignment device corresponds one-to-one to a multilingual corpus alignment method in the above embodiment. Figure 5 As shown, the multilingual corpus alignment device includes a text language determination module 10, a text language division module 20 and a corpus alignment determination module 30. The functional modules are described in detail as follows:
[0087] The text language determination module 10 is used to obtain the original text and perform language recognition on the original text through language recognition technology to obtain the language of the original text as the target language.
[0088] The text language classification module 20 is used to select a language from the target language as the standard language and select the remaining languages in the target language as the languages to be translated.
[0089] The corpus alignment judgment module 30 is used to translate the original text corresponding to the language to be translated into a standard text of a standard language, and perform alignment recognition on all the standard texts to obtain a corpus alignment result.
[0090] Furthermore, the corpus alignment judgment module 30 includes a text preprocessing unit, a to-be-processed word alignment judgment unit, a first corpus alignment judgment unit and a second corpus alignment judgment unit.
[0091] The text preprocessing unit is used to preprocess all standard texts to obtain the words to be processed of each standard text.
[0092] The to-be-processed word alignment judgment unit is used to determine the meaning relationship and position relationship of the to-be-processed words in each standard text. If the meaning relationship of the to-be-processed words in different standard texts is consistent and is in the same position, the to-be-processed words are considered to be aligned.
[0093] The first corpus alignment judgment unit is used to count the number of aligned words to be processed. When the number of aligned words to be processed meets a preset condition, the corresponding corpus alignment result is corpus matching.
[0094] The second corpus alignment judgment unit is used to determine that when the number of aligned to-be-processed words does not meet a preset condition, the corresponding corpus alignment result is that the corpus does not match.
[0095] For the specific definition of the multilingual corpus alignment device, please refer to the definition of the multilingual corpus alignment method above, which will not be repeated here. Each module in the above-mentioned multilingual corpus alignment device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0096] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a computer-readable storage medium and an internal memory. The computer-readable storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the computer-readable storage medium. The database of the computer device is used to store data involved in the multilingual corpus alignment method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multilingual corpus alignment method is implemented.
[0097] A computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the multilingual corpus alignment method in the above embodiment are implemented, such as Figure 1 Steps S10-S30 shown, or Figures 2 to 4 Alternatively, when the processor executes the computer program, the functions of each module / unit of the multilingual corpus alignment device in the above embodiment are realized, for example Figure 5 The functions of modules 10 to 30 are shown in detail and will not be described again to avoid repetition.
[0098] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the multilingual corpus alignment method in the above embodiment are implemented, for example: Figure 1 Steps S10-S30 shown, or Figures 2 to 4 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the multilingual corpus alignment device are realized, for example Figure 5 The functions of the modules 10 to 30 are shown in detail and will not be described again to avoid repetition.
[0099] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0100] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0101] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A multilingual corpus alignment method, characterized in that: include: Obtaining the original text, and performing language recognition on the original text through language recognition technology, and obtaining the language of the original text as the target language; Selecting one language from the target languages as a standard language, and selecting the remaining languages from the target languages as languages to be translated; Translate the original text corresponding to the language to be translated into the standard text of the standard language, and align and identify all the standard texts to obtain the corpus alignment result; The step of selecting a language from the target languages as a standard language includes: The language that appears most frequently in the target language is determined as the standard language; or Select a language that matches the translation requirements from the target language as the standard language; The step of aligning and identifying all standard texts to obtain corpus alignment results includes: Preprocess all standard texts to obtain the words to be processed of each standard text; Determine the meaning relationship and position relationship of the to-be-processed words in each standard text. If the meaning relationship of the to-be-processed words in different standard texts is consistent and they are in the same position, then it is considered that the to-be-processed words are aligned. The number of aligned words to be processed is counted. When the number of aligned words to be processed meets the preset condition, the corresponding corpus alignment result is corpus matching. When the number of aligned words to be processed does not meet the preset conditions, the corresponding corpus alignment result is corpus mismatch; each sentence carries a sentence sequence number; Determining the meaning relationship and position relationship of the words to be processed in each standard text includes: Confirm the meaning relationship of all the words to be processed with the same sentence sequence number in different standard texts. When the meaning relationship is consistent, determine the position relationship of the corresponding words to be processed. When the positions of the words to be processed with the same meaning relationship in their respective sentences are consistent, it means that the words to be processed with the same meaning relationship are in the same position in their respective sentences; When the positions of the to-be-processed words with the same meaning relationship in their respective sentences are inconsistent, it means that the to-be-processed words with the same meaning relationship are in different positions in their respective sentences.
2. A multilingual corpus alignment method according to claim 1, characterized in that: The preprocessing of all standard texts to obtain the words to be processed of each standard text includes: Segment each of the standard texts according to the segmentation marks to obtain sentences of each standard text; Sentences in different standard texts are segmented and stop words are removed to obtain the words to be processed for each standard text.
3. A multilingual corpus alignment method according to claim 1, characterized in that: The confirmation of the meaning relationship of all the to-be-processed words with the same sentence sequence number in different standard texts includes: When the sequence numbers of the words to be processed in the same sentence in different standard texts are consistent or are synonyms or near synonyms, it means that the meanings of the words to be processed are consistent; When the pending words with the same sentence sequence number in different standard texts are not synonyms or synonyms of each other, it means that the meaning relationship of the pending words is inconsistent.
4. A multilingual corpus alignment device, characterized in that: include: The text language determination module is used to obtain the original text and identify the language of the original text through language recognition technology to obtain the language of the original text as the target language; A text language classification module, used to select a language from the target language as a standard language, and select the remaining languages in the target language as languages to be translated; The corpus alignment judgment module is used to translate the original text corresponding to the language to be translated into the standard text of the standard language, and align and identify all the standard texts to obtain the corpus alignment result; The step of selecting a language from the target languages as a standard language includes: The language that appears most frequently in the target language is determined as the standard language; or Select a language that matches the translation requirements from the target language as the standard language; The corpus alignment judgment module comprises: The text preprocessing unit is used to preprocess all standard texts to obtain the to-be-processed words of each standard text; the to-be-processed word alignment judgment unit is used to determine the meaning relationship and position relationship of the to-be-processed words in each standard text. If the meaning relationship of the to-be-processed words in different standard texts is consistent and they are in the same position, then it is considered that the to-be-processed words are aligned; The first corpus alignment judgment unit is used to count the number of aligned words to be processed. When the number of aligned words to be processed meets a preset condition, the corresponding corpus alignment result is corpus matching; The second corpus alignment judgment unit is used for determining that the corresponding corpus alignment result is a corpus mismatch when the number of aligned to-be-processed words does not meet a preset condition; Each sentence carries a sentence sequence number; Determining the meaning relationship and position relationship of the words to be processed in each standard text includes: Confirm the meaning relationship of all the words to be processed with the same sentence sequence number in different standard texts. When the meaning relationship is consistent, determine the position relationship of the corresponding words to be processed. When the positions of the words to be processed with the same meaning relationship in their respective sentences are consistent, it means that the words to be processed with the same meaning relationship are in the same position in their respective sentences; When the positions of the to-be-processed words with the same meaning relationship in their respective sentences are inconsistent, it means that the to-be-processed words with the same meaning relationship are in different positions in their respective sentences.
5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the multilingual corpus alignment method according to any one of claims 1 to 3 is implemented.
6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multilingual corpus alignment method according to any one of claims 1 to 3 is implemented.
Citation Information
Patent Citations
Bilingual corpus resource acquisition method and bilingual corpus resource acquisition system
CN102591857A