Multi-language text alignment method and device

Abstract

The embodiment of the invention provides a multi-language text alignment method and device, computer equipment, a computer readable storage medium and a computer program product, and belongs to the technical field of text alignment. The method comprises the following steps: respectively preprocessing a source language document and a target language document to obtain a plurality of source language text blocks and a plurality of target language text blocks; for the ith source language text block of the source language document, selecting a plurality of target language text blocks from the target language document based on a preset sliding window by taking the ith target language text block of the target language document as an anchor point; and matching the ith source language text block with the selected target language text blocks one by one, and selecting the target language text block with the highest matching degree as a target matching text block of the ith source language text block. According to the technical scheme provided by the embodiment of the invention, the calculation amount of multi-language text alignment can be reduced, and the alignment precision is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of text alignment, and particularly relate to a multilingual text alignment method and device, computer equipment, computer readable storage medium, and computer program product. BACKGROUND

[0002] Multilingual text alignment technology is a core supporting technology in the fields of machine translation, bilingual corpus construction, and translation memory library.

[0003] At present, conventional multilingual text alignment methods are mainly based on global semantic search, and alignment is achieved by performing semantic retrieval on each source language text in the entire target language document.

[0004] However, the above method produces a large amount of invalid calculation, resulting in a large amount of calculation and low alignment efficiency.

[0005] It should be noted that the above content is not necessarily prior art, and is not used to limit the patent protection scope of the present application. SUMMARY

[0006] Embodiments of the present application provide a multilingual text alignment method, device, computer equipment, computer readable storage medium, and computer program product to solve or alleviate one or more technical problems proposed above.

[0007] One aspect of an embodiment of the present application provides a multilingual text alignment method, the method comprising: preprocessing a source language document and a target language document to obtain a plurality of source language text blocks and a plurality of target language text blocks; for an i-th source language text block of the source language document, taking an i-th target language text block of the target language document as an anchor point, and selecting a plurality of target language text blocks from the target language document based on a preset sliding window, i = 1, 2, … n, n being the number of source language text blocks contained in the source language document; matching the i-th source language text block with the selected plurality of target language text blocks one by one, and selecting a target language text block with the highest matching degree as a target matching text block of the i-th source language text block.

[0008] Optionally, the matching the i-th source language text block with the selected plurality of target language text blocks one by one, and selecting a target language text block with the highest matching degree as a target matching text block of the i-th source language text block comprises: obtaining a first semantic vector corresponding to the i-th source language text block and a second semantic vector corresponding to the selected plurality of target language text blocks; respectively calculate semantic similarity of the first semantic vector and a plurality of second semantic vectors; select a target language text block with the highest semantic similarity as the target matching text block of the i-th source language text block.

[0009] Optionally, the similarity values of the first semantic vector and a plurality of second semantic vectors are respectively calculated, and the similarity values are taken as the semantic similarity; The method further includes: The similarity values of the first semantic vector and a plurality of second semantic vectors are respectively calculated by using a plurality of similarity algorithms; The semantic similarity is determined based on the similarity values calculated by using the plurality of similarity algorithms.

[0010] Optionally, the method of respectively calculating the semantic similarity of the first semantic vector and a plurality of second semantic vectors includes: The similarity values of the first semantic vector and a plurality of second semantic vectors are respectively calculated; determine the term matching weighted values of the i-th source language text block and a plurality of selected target language text blocks; The semantic similarity is determined based on the similarity values and the term matching weighted values corresponding to the similarity values.

[0011] Optionally, the size of the sliding window is dynamically adjusted based on the matching degree of the source language text block and the target language text block.

[0012] Optionally, the size of the sliding window is dynamically adjusted based on the position of the source language text block.

[0013] Optionally, the method of matching the i-th source language text block with a plurality of selected target language text blocks one by one and selecting a target language text block with the highest matching degree as the target matching text block of the i-th source language text block includes: The i-th source language text block is matched with a plurality of selected target language text blocks one by one to obtain a plurality of matching degree values; The target language text block with the highest matching degree value is taken as a candidate language text block; In the case that the matching degree value of the candidate language text block exceeds a preset matching degree threshold, the candidate language text block is taken as the target matching text block of the i-th source language text block.

[0014] Optionally, the matching degree threshold is determined by the following method: determine the language pair to which the languages in the source language document and the target language document belong; determine the matching degree threshold value based on the language pair.

[0015] Optionally, a text type to which the ith source language text block belongs is determined. The matching degree threshold value is determined based on the text type.

[0016] Optionally, the method further comprises: After the matching processing on all source language text blocks in the source language document is completed, it is detected whether there is a case that multiple source language text blocks correspond to a same target matching text block. If there is a case that multiple source language text blocks correspond to a same target matching text block, a pair of source language text block and target matching text block with the highest matching degree are reserved.

[0017] Optionally, the method further comprises: After the matching processing on all source language text blocks in the source language document is completed, quality inspection processing is performed on all language text block pairs obtained, the language text block pair including a source language text block and a corresponding target matching text block. After the language text block pair fails to pass the quality inspection processing, the language text block pair is marked as abnormal. The quality inspection processing includes at least one of length proportion inspection processing, term consistency inspection processing, and back translation inspection processing.

[0018] Another aspect of the embodiment of the present application provides a multilingual text alignment device, the device comprising: A preprocessing module is configured to perform preprocessing on a source language document and a target language document respectively, to obtain a plurality of source language text blocks and a plurality of target language text blocks. A selection module is configured to, for an ith source language text block of the source language document, select a plurality of target language text blocks from the target language document based on a preset sliding window, with an ith target language text block of the target language document as an anchor point, i=1, 2, …n, n being a number of source language text blocks contained in the source language document. A matching module is configured to perform matching processing on the ith source language text block and the selected plurality of target language text blocks one by one, and select a target matching text block with the highest matching degree as a target matching text block of the ith source language text block.

[0019] Another aspect of the embodiment of the present application provides a computer device, comprising: at least one processor; and a memory in communication connection with the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0020] Another aspect of the embodiments of the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are executed by a processor to implement the method described above.

[0021] Another aspect of the embodiments of the present application provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the method described above.

[0022] The embodiments of the present application can include the following advantages by adopting the above technical solutions: Firstly, the source language document and the target language document are preprocessed to obtain a plurality of source language text blocks and a plurality of target language text blocks, so that the text block processing is realized, and subsequent alignment processing is facilitated; secondly, for the i th source language text block, the i th target language text block is taken as an anchor point, and a plurality of target language text blocks are selected from the target language document as candidate matching objects by using a preset sliding window, compared with a global traversal mode, the sliding window mechanism can limit the matching range and reduce invalid calculation amount, thereby greatly improving the efficiency of text alignment; finally, by matching the i th source language text block with the candidate target language text blocks one by one and selecting the text block with the highest matching degree as the target matching text block, the semantic consistency of the source language text block and the target language text block can be guaranteed to the greatest extent, the alignment deviation problem caused by the dependence on length features or limited dictionary in the traditional method is effectively solved, and the accuracy of multilingual text alignment is significantly improved. BRIEF DESCRIPTION OF DRAWINGS The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain exemplary implementations of the application. The illustrated embodiments are illustrative only and not restrictive of the scope of the claims. In all the drawings, like reference numerals refer to like parts throughout the several views.

[0023] Figure 1 An operating environment diagram of a multilingual text alignment method according to Embodiment One of the present application is schematically shown; Figure 2 A flowchart of a multilingual text alignment method according to Embodiment One of the present application is schematically shown; Figure 3 A step refinement flowchart of matching the i th source language text block with the selected plurality of target language text blocks one by one and selecting the target language text block with the highest matching degree as the target matching text block of the i th source language text block is schematically shown; Figure 4 The step of calculating the similarity value of the first semantic vector and the plurality of second semantic vectors respectively and taking the similarity value as the semantic similarity is shown in a detailed flow chart; Figure 5 The step of calculating the similarity value of the first semantic vector and the plurality of second semantic vectors respectively and taking the similarity value as the semantic similarity is shown in a detailed flow chart; Figure 6 The step of matching the i th source language text block with the selected plurality of target language text blocks one by one and selecting the target language text block with the highest matching degree as the target matching text block of the i th source language text block is shown in a detailed flow chart; Figure 7 The newly added step flow chart of the multilingual text alignment method according to Embodiment One of the present application is shown; Figure 8 The newly added step flow chart of the multilingual text alignment method according to Embodiment One of the present application is shown; Figure 9 The exemplary application flow chart of the multilingual text alignment method of the present application is shown; Figure 10 The block diagram of the multilingual text alignment device according to Embodiment Two of the present application is shown; Figure 11 The hardware architecture schematic diagram of the computer device in Embodiment Three of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0025] It should be noted that the description of "first", "second" and the like in the embodiments of the present application is only for the purpose of description and should not be understood as indicating or implying the relative importance of the technical features indicated or implicitly indicating the number of technical features. Therefore, the features with "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can realize it, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist and is not within the scope of protection claimed by the present application.

[0026] It should be noted that, in the present application, if the collection, storage, use, transmission and processing of data are involved, each link of the data strictly follows the laws and regulations, industry standards and regulatory requirements of the data source, use location and relevant countries and regions to ensure the legality and compliance of data activities. In the collection link, the data subject is clearly informed of the collection purpose, method and scope in a conspicuous manner, and the collection is carried out after obtaining the legal authorization of the data subject, ensuring that the collection process follows the "minimum necessary" principle and does not collect data beyond the scope. In the storage link, the storage period is limited, and the data is deleted or anonymized and encrypted in a timely manner after achieving the storage purpose. In the use link, strict data security protection mechanisms will be implemented, and field-level desensitization technology will be used to process the original data according to the preset desensitization rules. For different types of data, various desensitization strategies such as data generalization, data anonymization and data encryption are adopted to effectively avoid the risk of sensitive information leakage and ensure that the data used is desensitized data that has been safely processed, thereby fully protecting the rights and interests of data subjects and data security. In the transmission and processing link, the confidentiality and security of data in the transmission and processing process are ensured.

[0027] In the description of the present application, it should be understood that the numerical reference before the step does not indicate the order of execution of the steps before and after, but is only used to facilitate the description of the present application and to distinguish each step, and therefore cannot be understood as a limitation on the present application.

[0028] First, the terms involved in the present application are explained: Terminology library: a database that saves the mapping relationship between source terms and target terms in the form of key-value pairs. The database can also record the metadata of each term, which can include domain tags, use case examples, priority, creation time, source, etc.

[0029] Translation memory: a database that stores previously translated sentences or paragraphs (called "translation units") in pairs. Translation memory can be reused in future translation projects, thereby improving translation efficiency and consistency. The translation memory can also save the semantic vectors of each sentence and metadata, including source information, domain tags, timestamps, and artificial confirmation markers.

[0030] Secondly, the technical solutions provided by the present application for the convenience of those skilled in the art to understand the embodiments of the present application are described as follows: Multilingual text alignment technology is a core supporting technology in the fields of machine translation, bilingual corpus construction, and translation memory.

[0031] Currently, conventional multilingual text alignment methods include the following: 1. Simple alignment method based on order, which assumes that the source language document and the target language document have the same number of blocks after segmentation and are one-to-one corresponding in order.

[0032] 2. Alignment method based on length or statistical characteristics (such as Gale-Church statistical method), which estimates alignment probability according to character / word length ratio or position characteristics.

[0033] 3. Back translation verification method based on bidirectional translation or machine translation, which translates the source language text to the target translation text by machine translation, and then matches the target translation text with the target language text to realize alignment.

[0034] 4. Global semantic search method, which realizes alignment by performing semantic search on each source language text in the entire target language document.

[0035] However, when aligning multilingual texts based on simple alignment methods, a large number of incorrect pairs are usually generated, and about 25% of dirty data is measured; when aligning multilingual texts based on length or statistical characteristics, semantic variants are helpless and misjudgment is easy; when aligning multilingual texts based on back translation verification method of bidirectional translation or machine translation, the calculation amount is large, and it is not practical for large-scale texts; in addition, when aligning multilingual texts based on global semantic search method, the calculation amount is also large, and the efficiency is low.

[0036] Therefore, the embodiments of the present application provide a technical scheme for aligning multilingual texts. In the technical scheme, first, a plurality of source language text blocks and a plurality of target language text blocks are obtained by preprocessing the source language document and the target language document, realizing text block processing, and facilitating subsequent alignment processing; second, for the i th source language text block, the i th target language text block is taken as an anchor point, and a plurality of target language text blocks are selected as candidate matching objects from the target language document by using a preset sliding window, compared with the global traversal mode, the sliding window mechanism can limit the matching range, reduce the invalid calculation amount, and greatly improve the efficiency of text alignment; finally, by matching the i th source language text block with the candidate target language text block one by one and selecting the text block with the highest matching degree as the target matching text block, the semantic consistency of the source language text block and the target language text block can be guaranteed to the greatest extent, the alignment deviation problem caused by the dependence on length characteristics or limited dictionary in traditional methods can be effectively solved, and the accuracy of multilingual text alignment is significantly improved. In addition, compared with the method based on length, the semantic vector can tolerate free translation, word order change and syntax difference, which can improve the recognition ability of free translation and expression difference, thereby improving the accuracy. See the following.

[0037] Finally, in order to facilitate understanding, an example running environment is provided below.

[0038] As shown in Figure 1 The running environment diagram includes: a service platform 2, a network 4, a client 6, wherein: The service platform 2 can be composed of a single or multiple computing devices. The multiple computing devices can include virtualized computing instances. The virtualized computing instances can include virtual machines, such as emulations of computer systems, operating systems, servers, etc. The computing devices can load the virtual machines based on virtual images and / or other data defining specific software (e.g., operating systems, specialized applications, servers) for the emulation. As the demand for different types of processing services changes, different virtual machines can be loaded and / or terminated on one or more computing devices. A hypervisor can be implemented to manage the use of different virtual machines on the same computing device.

[0039] The service platform 2 can be configured to communicate with the client 6, etc. through the network 4. The network 4 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 4 can include physical links, such as coaxial cable links, twisted-pair cable links, fiber-optic links, combinations thereof, etc., or wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0040] The service platform 2 can provide storage, reading, writing, querying, deleting, etc. services, such as running a multi-language text alignment model or providing a multi-language text alignment service for the client.

[0041] The client 6 can be an electronic device running an operating system such as Windows, AndroidTM, or iOS, etc., such as a smartphone, a tablet device, a laptop computer, a virtual reality device, a game device, a set-top box, a vehicle terminal, a smart television. Based on the above operating system, various applications can be run, such as an application for multi-language text alignment.

[0042] The client 6 can provide / configure a user access page for manipulating the service platform 2 or uploading objects (such as source language documents and target language documents), etc.

[0043] It should be noted that the above devices are exemplary, and the number and types of devices can be adjusted in different scenarios or according to different needs.

[0044] The following describes the technical solutions of the present application through a plurality of embodiments, taking the service platform 2 or the client 6 as the execution subject. It should be understood that these embodiments can be implemented in a variety of different forms, and should not be interpreted as being limited to the embodiments described herein.

[0045] Embodiment one Figure 2A flowchart of a multilingual text alignment method according to Embodiment One of the present application is shown schematically.

[0046] As shown in Figure 2 , the multilingual text alignment method can include steps S200-S204, in which: In step S200, the source language document and the target language document are respectively preprocessed to obtain a plurality of source language text blocks and a plurality of target language text blocks.

[0047] In step S202, for the i-th source language text block of the source language document, the i-th target language text block of the target language document is taken as an anchor point, and a plurality of target language text blocks are selected from the target language document based on a preset sliding window, i = 1, 2, … n, n being the number of source language text blocks contained in the source language document. In step S204, the i-th source language text block and the selected plurality of target language text blocks are matched one by one, and the target language text block with the highest matching degree is selected as the target matching text block of the i-th source language text block.

[0048] The multilingual text alignment method provided in this embodiment first obtains a plurality of source language text blocks and a plurality of target language text blocks by preprocessing the source language document and the target language document, realizes text block processing, and facilitates subsequent alignment processing. Secondly, for the i-th source language text block, the i-th target language text block is taken as an anchor point, and a plurality of target language text blocks are selected from the target language document as candidate matching objects using a preset sliding window. Compared with the global traversal method, the sliding window mechanism can limit the matching range, reduce the invalid calculation amount, and greatly improve the efficiency of text alignment. Finally, by matching the i-th source language text block with the candidate target language text blocks one by one and selecting the text block with the highest matching degree as the target matching text block, the semantic consistency of the source language text block and the target language text block can be guaranteed to the greatest extent, effectively solving the alignment deviation problem caused by relying on length features or limited dictionaries in traditional methods, and significantly improving the accuracy of multilingual text alignment.

[0049] The following Figure 2 , each step in steps S200-S204 and optional other steps are described in detail.

[0050] Step S200 , the source language document and the target language document are respectively preprocessed to obtain a plurality of source language text blocks and a plurality of target language text blocks.

[0051] In the embodiment, the source language document and the target language document can be in pure text format (.txt), structured document format (.doc,.docx,.pdf), etc. The document content in the source language document and the target language document is written in different languages respectively. For example, the source language document is a Chinese document, and the target language document is an English document.

[0052] The preprocessing operation can include text block segmentation processing and text block cleaning processing.

[0053] The text block segmentation processing is used to segment the source language document and the target language document into multiple text blocks respectively.

[0054] In a specific implementation, the source language document D_src and the target language document D_tgt can be segmented by using a preset segmentation rule to obtain multiple source language text blocks {B s1 ,B s2 ,...,B sn} and multiple target language text blocks {B t1 ,B t2 ,...,B tm}, where n and m are the number of source language text blocks and target language text blocks respectively, and n and m can be equal or not equal.

[0055] The segmentation rule can select sentence-level segmentation. The sentence-level segmentation is based on punctuation marks (such as period, question mark, exclamation mark) as a segmentation symbol to segment the document into single sentences as text blocks.

[0056] The segmentation rule can also select paragraph-level segmentation. The paragraph-level segmentation is based on line feed, paragraph mark or dialogue turn as a segmentation symbol to segment the document into paragraphs as text blocks.

[0057] Text block cleaning is used to standardize the segmented text blocks to ensure the purity of the text blocks. Text block cleaning includes but is not limited to: removing special characters (such as tab character, garbled character), unifying character case (such as converting all English characters to lowercase), removing redundant spaces, and normalizing the format of special content such as numbers and dates.

[0058] In the embodiment, the source language document and the target language document are segmented into text blocks with relatively complete semantics, so that the subsequent processing is more stable.

[0059] Step S202 For the i-th source language text block of the source language document, the i-th target language text block of the target language document is used as an anchor point, and multiple target language text blocks are selected from the target language document based on a preset sliding window, i = 1, 2, … n, n is the number of source language text blocks contained in the source language document.

[0060] In this embodiment, for the i-th source language text block B of the source language document si (1≤i≤n), the i-th target language text block B of the target language document ti Set as the anchor text block. Here, the index correspondence of the text block is based on the natural order of the documents, that is, after the source language document and the target language document are divided into blocks, the resulting text blocks are arranged in order.

[0061] In this embodiment, the core parameters of the sliding window may include: Window size k: This refers to the number of target language text blocks contained in the window. The window size k can be a fixed value or a dynamic value.

[0062] Slide direction: with anchor text block B ti Centered on the target language text block sequence, slide the text blocks in both directions, where the number of text blocks slid in both directions can be set according to the actual situation.

[0063] Slide step size: The number of text blocks the window moves with each slide. In this embodiment, the step size is set to 1, meaning the window moves only one text block position each time.

[0064] As an example, suppose the sliding parameters of the sliding window are window size k=11 and sliding direction is anchor text block B. ti Centered on the target language text block sequence, slide 5 text blocks in each of the two directions before and after, with a sliding step size of 1. Then, when targeting the i-th source language text block of the source language document, the i-5 to i+5 target language text blocks in the target language document will be selected as the text blocks to be matched with the i-th source language text block.

[0065] It should be noted that, in order to ensure that multiple target language text blocks can be selected for the source language text block to be matched under any circumstances, the following boundary processing rules can be set for this sliding window: When the anchor text block is at the beginning of the target language text block sequence (i=1), the window can only truncate text blocks backward until the window size k is reached; when the anchor text block is at the end of the target language text block sequence (i=m), the window only truncates text blocks forward until the window size k is reached; when the anchor text block is in the middle, the window truncates text blocks in both directions.

[0066] In this embodiment, a set of candidate target text blocks (composed of multiple selected target language text blocks) is determined by a sliding window, which can narrow the matching range and improve alignment efficiency and accuracy.

[0067] In an optional embodiment, the size of the sliding window can be dynamically adjusted based on the matching degree of the source language text block and the target language text block.

[0068] In an embodiment, when the matching degree value of the i-th source language text block B si and the i-th target language text block B ti is large (higher than a pre-set high matching degree threshold T H ), it indicates that the matching degree of B si and the anchor text block B ti is high, and the anchor text block is probably the optimal matching text block. At this time, the size of the sliding window can be reduced to reduce the number of candidate text blocks and improve the matching efficiency.

[0069] In an embodiment, when the matching degree value of the i-th source language text block B si and the i-th target language text block B ti is moderate (the matching degree is higher than a pre-set low matching degree threshold T L and lower than a pre-set high matching degree threshold T H ), it indicates that the matching degree of B si and the anchor text block B ti is moderate, and the optimal matching text block is probably near the anchor. At this time, the size of the sliding window can be kept as the initial default size.

[0070] In another embodiment, when the matching degree value of the i-th source language text block B si and the i-th target language text block B ti is small (the matching degree is lower than a pre-set low matching degree threshold T L ), it indicates that the matching degree of B si and the anchor text block B ti is low, and the optimal matching text block is probably not in the position adjacent to the anchor. At this time, the size of the sliding window can be expanded to increase the coverage range of the candidate text blocks and avoid missing the optimal matching result.

[0071] In this embodiment, the size of the sliding window can be dynamically expanded or reduced based on the matching degree of the source language text block and the target language text block, so that the candidate target language text block set is more in line with the actual matching requirements.

[0072] In an optional embodiment, the size of the sliding window can also be dynamically adjusted based on the position of the source language text block.

[0073] In an embodiment, the sequence of text blocks of the source language document {B s1 , B s2 ,..., B snThe three position intervals are divided, and the division ratio can be set based on actual needs, preferably three equal division: 1. The beginning interval: the text block index i e [1, n / 3], the text block in this interval is usually the introduction of the document, the background introduction, the semantics is relatively broad, and the cross-language matching difficulty is higher.

[0074] 2. The middle interval: the text block index i e (n / 3, 2n / 3], the text block in this interval is usually the core content of the document, the semantics is clear, and the corresponding order of the text block and the target language document is stronger.

[0075] 3. The end interval: the text block index i e (2n / 3, n], the text block in this interval is usually the conclusion and summary of the document, the semantics is relatively concise, and the matching difficulty is medium.

[0076] After completing the division of the position interval, different window sizes can be preset based on the text blocks in different position intervals, and the mapping rule is as follows: The size of the sliding window corresponding to the beginning interval is 7, the size of the sliding window corresponding to the middle interval is 3, and the size of the sliding window corresponding to the end interval is 5.

[0077] Subsequently, when matching the source language text block, the size of the sliding window can be determined according to the mapping relationship between the position interval to which the source language text block belongs and the size of the sliding window.

[0078] In this embodiment, different window sizes are adapted according to the position of the source language text block in the document (such as beginning, middle, and end), so as to solve the problem that the alignment difficulty of the text block in different positions of the document is different.

[0079] Step S204 The i-th source language text block is matched with the selected multiple target language text blocks one by one, and the target language text block with the highest matching degree is selected as the target matching text block of the i-th source language text block.

[0080] In this embodiment, when the i-th source language text block is matched with the selected multiple target language text blocks one by one, the text features of the i-th source language text block B si and the selected multiple target language text blocks B tc are extracted respectively, then based on the extracted text features, a preset matching algorithm is used to calculate the matching degree M(B si , B tc ) of B si and each target language text block B tc .

[0081] The text features include but are not limited to: Semantic vector features: refer to the numerical "semantic representation" (semantic vector) of the text block obtained after the text block is vectorized by a pre-trained multilingual model (such as multilingual-e5, LASER, etc.). By vectorizing the text block through a multilingual model, different language text blocks can be mapped to the same semantic space.

[0082] Statistical features: the number of words, the number of characters, keyword frequency (such as TF-IDF value), part-of-speech distribution (such as the proportion of nouns and verbs), etc. in the text block; Semantic features: core semantic components (such as subject, predicate, and object) of the text block are extracted through semantic role labeling and dependency syntax analysis.

[0083] It should be noted that different types of extracted text features use different matching algorithms. For example, if the extracted text features are semantic vector features, the matching algorithm can be cosine similarity algorithm, Euclidean distance algorithm, etc. For example, if the extracted text features are statistical features, the matching algorithm can be weighted term frequency overlap algorithm. The weighted term frequency overlap algorithm calculates the overlap degree based on the frequency of words (term frequency) in the text block.

[0084] In this embodiment, after completing the alignment processing of the i-th source language text block, the above steps S202-S204 are repeatedly executed so that each text block in the source language document can be sequentially selected as a candidate text block, the matching degree is calculated, and the optimal matching text block is determined, and finally the alignment result set of each source language text block and target language text block is obtained. s1 ,B s2 ,...,B sn} in the source language document are selected as candidate text blocks, the matching degree is calculated, and the optimal matching text block is determined, and finally the alignment result set of each source language text block and target language text block is obtained.

[0085] It should be noted that there are various ways to determine the target matching text block of the i-th source language text block. Several exemplary determination methods are provided below.

[0086] In an optional implementation, referring to Figure 3 , the matching processing of the i-th source language text block and the selected multiple target language text blocks is performed one by one, and the target language text block with the highest matching degree is selected as the target matching text block of the i-th source language text block. Step S300: obtaining the first semantic vector corresponding to the i-th source language text block and the second semantic vector corresponding to the selected multiple target language text blocks.

[0087] In the embodiment, the first semantic vector corresponding to the ith source language text block and the second semantic vectors corresponding to the selected target language text blocks can be obtained by vectorizing the first semantic vector and the second semantic vectors by the multilingual model.

[0088] In step S302, the semantic similarity of the first semantic vector and the second semantic vectors is calculated respectively.

[0089] The semantic similarity is used to represent the similarity between the ith source language text block and each selected target language text block. For example, the semantic similarity is 0.9.

[0090] In the embodiment, the cosine similarity algorithm can be used to calculate the semantic similarity of the first semantic vector and the second semantic vectors.

[0091] In other embodiments, the Euclidean distance, Manhattan distance, or other algorithms can be selected according to actual needs to calculate the semantic similarity of the first semantic vector and the second semantic vectors.

[0092] It should be noted that if a distance algorithm is selected, the distance value needs to be converted into a similarity value.

[0093] In an optional embodiment, the calculation of the semantic similarity of the first semantic vector and the second semantic vectors includes: The similarity values of the first semantic vector and the second semantic vectors are calculated respectively, and the similarity values are used as the semantic similarity.

[0094] In the embodiment, one or more similarity algorithms can be used to calculate the similarity values of the first semantic vector and the second semantic vectors. After obtaining the similarity values, the semantic similarity can be determined based on one or more similarity algorithms.

[0095] In the embodiment, the similarity values output by the algorithm are directly used as the semantic similarity, which reduces the calculation process and improves the execution efficiency of the alignment task.

[0096] In an optional embodiment, referring to Figure 4 , the calculation of the similarity values of the first semantic vector and the second semantic vectors includes: In step S400, one or more similarity algorithms are used to calculate the similarity values of the first semantic vector and the second semantic vectors.

[0097] In step S402, the semantic similarity is determined based on the similarity values calculated by the one or more similarity algorithms.

[0098] In the embodiment, at least two different types of vector similarity algorithms can be selected to form an algorithm combination to calculate the similarity values of the first semantic vector and the plurality of second semantic vectors. For example, cosine similarity and Euclidean distance similarity algorithms can be selected as the algorithm combination. For another example, Pearson correlation coefficient and cosine similarity algorithms can be selected as the algorithm combination. Even, cosine similarity, Euclidean distance similarity, and Pearson correlation coefficient algorithms can be selected as the algorithm combination.

[0099] In the embodiment, after the similarity values of the first semantic vector and the plurality of second semantic vectors are calculated by using each type of similarity algorithm, the similarity values calculated by the plurality of algorithms can be weighted and summed to obtain a final similarity value, and the final similarity value is taken as the semantic similarity.

[0100] As an example, cosine similarity and Euclidean distance similarity are selected to form a combination, and the weights are set as w1=0.7 and w2=0.3. The cosine similarity value S1 is calculated as 0.91, and after normalization, S1' (similarity value) is 0.91. The Euclidean distance value S2 is calculated as 0.15, and after normalization, S2' (similarity value) is 0.85. The final semantic similarity is calculated by weighted average as Sim=0.7*0.91+0.3*0.85=0.892.

[0101] In the embodiment, the similarity values are calculated and fused by using the plurality of algorithm combinations, which can take into account the advantages of different algorithms and cover more types of text semantic features. Compared with a single algorithm, the semantic similarity calculation method of the multi-algorithm fusion can improve the text alignment accuracy and robustness.

[0102] In step S304, the target language text block with the highest semantic similarity is selected as the target matching text block of the ith source language text block.

[0103] In the embodiment, the target matching text block of the ith source language text block is selected based on the semantic similarity, which can improve the accuracy and robustness of the multi-language text block alignment.

[0104] In an optional implementation, referring to Figure 5 The calculating the semantic similarity of the first semantic vector and the plurality of second semantic vectors includes: In step S500, the similarity values of the first semantic vector and the plurality of second semantic vectors are calculated.

[0105] In the embodiment, the calculation method of the similarity values of the first semantic vector and the plurality of second semantic vectors has been described in detail in the above embodiments, and will not be described herein.

[0106] Step S502, determining a term matching weighting value of the i-th source language text block and the selected target language text blocks.

[0107] In this embodiment, when determining the term matching weighting value, the professional terms contained in the i-th source language text block can be extracted from the i-th source language text block according to the pre-created term library, to obtain a source language term subset T si si ={t s1 ,t s2 ,...,t sp}, where p is the number of professional terms contained in the i-th source language text block B si .

[0108] In addition, when determining the term matching weighting value, the professional terms contained in each selected target language text block B tc can also be extracted from the selected target language text block according to the pre-created term library, to obtain a target language term subset T tcj tcj ={t t1 ,t t2 ,...,t tq}, where q is the number of professional terms contained in the target language text block B tc .

[0109] After extracting the terms, the source language term subset T si and the target language term subset T tcj tcj ={t t1 ,t t2 ,...,t tq} can be matched to determine the number of matched terms, and then the term matching weighting value can be determined according to the number of matched terms.

[0110] In a specific embodiment, different term matching weighting values corresponding to different numbers of matched terms can be pre-set, for example, when the number of matched terms is 1-2, the corresponding term matching weighting value is 1.1, when the number of matched terms is 3-4, the corresponding term matching weighting value is 1.2, and when the number of matched terms is more than 4, the corresponding term matching weighting value is 1.3.

[0111] Through the above pre-set mapping relationship between the number of matched terms and the term matching weighting value, the term matching weighting value can be quickly determined.

[0112] Step S504, determining the semantic similarity based on the similarity value and the term matching weighting value corresponding to the similarity value.

[0113] ​​​In the embodiment, after the similarity value and the term matching weight value corresponding to the similarity value are determined, the product of the similarity value and the term matching weight value is taken as the semantic similarity, i.e., the semantic similarity = the similarity value x the term matching weight value.

[0114] In the embodiment, the vector similarity value is corrected by introducing the term matching weight value, the core role of professional terms in cross-language text matching is strengthened, and the accuracy of multi-language text alignment in a professional field is improved.

[0115] In an optional embodiment, the participating Figure 6 The matching processing of the i-th source language text block and the selected target language text blocks is performed one by one, and the target language text block with the highest matching degree is selected as the target matching text block of the i-th source language text block. In step S600, the matching processing of the i-th source language text block and the selected target language text blocks is performed one by one, and a plurality of matching degree values are obtained.

[0116] In step S602, the target language text block with the highest matching degree value is selected as the candidate language text block.

[0117] In step S604, when the matching degree value of the candidate language text block exceeds a preset matching degree threshold, the candidate language text block is taken as the target matching text block of the i-th source language text block.

[0118] The matching degree value is used to represent the matching degree of the i-th source language text block and each selected target language text block. For example, the matching degree value is 0.9.

[0119] The matching degree threshold is preset, and its specific value can be set and modified according to actual conditions. For example, the matching degree threshold is 0.68.

[0120] In an embodiment, the matching degree threshold is a fixed value. In another embodiment, the matching degree threshold can also be a dynamic value.

[0121] In the embodiment, the candidate language text block is taken as the target matching text block of the i-th source language text block only when the matching degree value of the candidate language text block exceeds the preset matching degree threshold, so that the candidate language text block corresponding to a too low matching degree value can be avoided to be taken as the target matching text block of the i-th source language text block, thereby effectively reducing the proportion of dirty data.

[0122] In an embodiment, when the matching degree value of the candidate language text block does not exceed the matching degree threshold, the candidate language text block can be discarded.

[0123] In another embodiment, the candidate language text block can also be marked when the matching degree value of the candidate language text block does not exceed the matching degree threshold.

[0124] In an optional embodiment, the matching degree threshold can be determined in the following manner: determining a language pair to which the languages in the source language document and the target language document belong; and determining the matching degree threshold based on the language pair.

[0125] In this embodiment, a mapping relationship table of different language pairs and matching degree thresholds can be established in advance, for example, the matching degree threshold corresponding to the Chinese-English language pair is 0.8, the matching degree threshold corresponding to the Chinese-Japanese language pair is 0.7, and the matching degree threshold corresponding to the Chinese-French language pair is 0.75. Subsequently, after determining the language pair to which the languages in the source language document and the target language document belong, the matching degree threshold corresponding to the determined language pair can be queried based on the mapping relationship table.

[0126] In this embodiment, by setting different matching degree thresholds based on different language pairs, the accuracy of text alignment in different scenarios can be optimized.

[0127] In an optional embodiment, the matching degree threshold can also be determined in the following manner: determining a text type to which the i-th source language text block belongs; and determining the matching degree threshold based on the text type.

[0128] The text type can include dialogue, plot, commentary / narration, monologue, and psychological description.

[0129] In this embodiment, a mapping relationship table of different text types and matching degree thresholds can be established in advance, for example, the matching degree threshold corresponding to the text type of dialogue is 0.8, the matching degree threshold corresponding to the text type of plot is 0.7, the matching degree threshold corresponding to the text type of monologue is 0.75, and the matching degree threshold corresponding to the text type of psychological description is 0.7. Subsequently, after determining the text type to which the i-th source language text block belongs, the corresponding matching degree threshold can be queried based on the mapping relationship table.

[0130] In this embodiment, by setting different matching degree thresholds based on different text types, the accuracy of text alignment in different scenarios can be optimized.

[0131] In an optional embodiment, referring to Figure 7 , the method further comprises: Step S700, after the matching process on all source language text blocks in the source language document is completed, it is detected whether there are multiple source language text blocks corresponding to the same target matching text block.

[0132] Step S702, if there are multiple source language text blocks corresponding to the same target matching text block, the pair of source language text block and target matching text block with the highest matching degree is reserved.

[0133] In the embodiment, since the matching process on the source language text blocks in the source language document is performed one by one, in the matching process, there can be a case that multiple source language text blocks correspond to the same target matching text block. When this case occurs, in order to avoid that the same target matching text block is occupied by multiple source language text blocks, the pair of source language text block and target matching text block with the highest matching degree can be reserved, and the pairs of source language text block and target matching text block composed of other source language text blocks and target matching text blocks can be discarded.

[0134] In the embodiment, by the above processing manner, the one-to-many ambiguous situation can be avoided, the one-to-one requirement of the translation memory library is ensured, and the artificial judgment burden in post-processing is reduced.

[0135] In optional embodiments, referring to Figure 8 , the method further includes: Step S800, after the matching process on all source language text blocks in the source language document is completed, quality inspection processing is performed on all pairs of language text blocks obtained, the pair of language text blocks including a source language text block and a target matching text block corresponding thereto.

[0136] Step S802, after the pair of language text blocks fails the quality inspection processing, the pair of language text blocks is marked as abnormal.

[0137] The quality inspection processing includes at least one of length proportion inspection processing, term consistency inspection processing, and back-translation inspection processing.

[0138] The length proportion inspection processing is used to check whether the source language text block and the target matching text block are too inconsistent in length.

[0139] The term consistency inspection processing is used to check whether the terms in the source language text block and the target matching text block all have reasonable correspondence.

[0140] The back-translation inspection processing is used to back-translate the target matching text block into a target source language text block, and then check whether the meaning of the back-translated target source language text block is consistent with that of the source language text block.

[0141] In this embodiment, after the language text block pair fails to pass the quality inspection processing, the language text block pair is marked as abnormal or submitted to a manual review team, so that subsequent manual correction can be facilitated.

[0142] In an embodiment, after the quality inspection processing is completed, all language text block pairs that pass the quality inspection processing can be stored in a translation memory library.

[0143] In order to make the present application easier to understand, the following will be combined with Figure 9 An exemplary application is provided.

[0144] In this exemplary application, the multi-language text alignment method includes: Step S900, input a source language document and a target language document.

[0145] Step S901, divide the source language document into m source language text blocks and divide the target language document into n target language text blocks.

[0146] Step S902, use a multi-language model to respectively perform vectorization processing on the m source language text blocks and the n target language text blocks, to generate 768-dimensional embedding vectors (semantic vectors).

[0147] Step S903, for the i-th (i = 1, 2, 3,..., m) source language text block of the source language document, take the i-th target language text block of the target language document as an anchor point, and select a plurality of target language text blocks from the target language document based on a preset sliding window.

[0148] Step S904, respectively calculate the cosine similarity between the i-th source language text block and the semantic vectors of each selected target language text block.

[0149] Step S905, select the target language text block with the largest cosine similarity value as a candidate language text block.

[0150] Step S906, perform confidence checking on the cosine similarity of the candidate language text block to determine whether the cosine similarity of the candidate language text block is greater than or equal to a preset threshold (for example, 0.68).

[0151] Step S907, in the case where the cosine similarity of the candidate language text block is greater than or equal to the preset threshold, the candidate language text block and the i-th source language text block form a text block pair, and the record is recorded.

[0152] Embodiment Two Figure 10Fig. 1 shows a block diagram of a multilingual text alignment device 1000 according to Embodiment Two of the present application. The device 1000 can be divided into one or more program modules stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program modules referred to in the embodiments of the present application are a series of computer program instruction segments capable of completing a specific function. The functions of the program modules in the embodiments will be described in detail below. As shown in Fig. 1, the device 1000 can include a preprocessing module 1010, a selection module 1020, and a matching module 1030, wherein: Figure 10 The preprocessing module 1010 is configured to preprocess the source language document and the target language document respectively to obtain a plurality of source language text blocks and a plurality of target language text blocks. The preprocessing module 1010 is configured to preprocess the source language document and the target language document respectively to obtain a plurality of source language text blocks and a plurality of target language text blocks. The selection module 1020 is configured to, for an i-th source language text block of the source language document, select a plurality of target language text blocks from the target language document based on a preset sliding window with the i-th target language text block of the target language document as an anchor point, i = 1, 2, … n, n being a number of source language text blocks contained in the source language document. The matching module 1030 is configured to perform matching processing on the i-th source language text block and the plurality of selected target language text blocks one by one, and select a target language text block with the highest matching degree as a target matching text block of the i-th source language text block.

[0153] In an optional embodiment, the matching module 1030 is further configured to: obtain a first semantic vector corresponding to the i-th source language text block and a plurality of second semantic vectors corresponding to the plurality of selected target language text blocks, calculate semantic similarities of the first semantic vector and the plurality of second semantic vectors respectively, and select a target language text block with the highest semantic similarity as the target matching text block of the i-th source language text block.

[0154] In an optional embodiment, the matching module 1030 is further configured to: calculate similarity values of the first semantic vector and the plurality of second semantic vectors respectively, and take the similarity values as the semantic similarities.

[0155] In an optional embodiment, the matching module 1030 is further configured to: calculate similarity values of the first semantic vector and the plurality of second semantic vectors respectively using a plurality of similarity algorithms, and determine the semantic similarities based on the similarity values calculated by the plurality of similarity algorithms.

[0156] In an optional embodiment, the matching module 1030 is further configured to: Calculate the similarity values ​​between the first semantic vector and the multiple second semantic vectors respectively; determine the term matching weighted value between the i-th source language text block and the selected multiple target language text blocks; determine the semantic similarity based on the similarity value and the term matching weighted value corresponding to the similarity value.

[0157] In an optional embodiment, the size of the sliding window is dynamically adjusted based on the matching degree between the source language text block and the target language text block.

[0158] In an optional embodiment, the size of the sliding window is dynamically adjusted based on the position of the source language text block.

[0159] In an optional embodiment, the matching module 1030 is further configured to: The i-th source language text block is matched one by one with the selected multiple target language text blocks to obtain multiple matching degree values; the target language text block with the highest matching degree value is selected as a candidate language text block; if the matching degree value of the candidate language text block exceeds a preset matching degree threshold, the candidate language text block is selected as the target matching text block of the i-th source language text block.

[0160] In an optional embodiment, the matching degree threshold is determined in the following manner: Determine the language pair to which the languages ​​in the source language document and the target language document belong; determine the matching degree threshold based on the language pair.

[0161] In an optional embodiment, the matching degree threshold is determined in the following manner: Determine the text type to which the i-th source language text block belongs; determine the matching degree threshold based on the text type.

[0162] In an optional embodiment, the multilingual text alignment device 1000 is further configured to: After matching all source language text blocks in the source language document, it is checked whether there are multiple source language text blocks corresponding to the same target matching text block; if there are multiple source language text blocks corresponding to the same target matching text block, the pair of source language text blocks and target matching text blocks with the highest matching degree are retained.

[0163] In an optional embodiment, the multilingual text alignment device 1000 is further configured to: After matching all source language text blocks in the source language document, quality inspection is performed on all resulting language text block pairs, which include source language text blocks and their corresponding target matching text blocks. After the language text block pair fails the quality inspection process, the language text block pair is marked as abnormal. The quality inspection process includes at least one of the following: length ratio check, terminology consistency check, and back-translation check.

[0164] Example 3 Figure 11 This illustration schematically depicts the hardware architecture of a computer device 10000 suitable for implementing a multilingual text alignment method according to Embodiment 3 of this application. In some embodiments, the computer device 10000 may be a terminal device such as a smartphone, wearable device, tablet computer, personal computer, in-vehicle terminal, game console, virtual device, workbench, digital assistant, set-top box, or robot. In other embodiments, the computer device 10000 may be a rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc. Figure 11 As shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked with each other via a system bus. Wherein: The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of a computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as program code for multilingual text alignment methods. In addition, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0165] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0166] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0167] It should be pointed out that, Figure 11 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0168] In this embodiment, the multilingual text alignment method stored in memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiments of this application.

[0169] Example 4 This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the multilingual text alignment method in the embodiments.

[0170] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the multilingual text alignment method in this embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0171] Example 5 This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in the above embodiments.

[0172] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computer devices. They can be centralized on a single computer device or distributed across a network of multiple computer devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computer device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0173] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A multilingual text alignment method, characterized in that, The method includes: The source language document and the target language document are preprocessed separately to obtain multiple source language text blocks and multiple target language text blocks; For the i-th source language text block in the source language document, using the i-th target language text block in the target language document as the anchor point, multiple target language text blocks are selected from the target language document based on a preset sliding window, where i = 1, 2, ..., n, and n is the number of source language text blocks contained in the source language document. The i-th source language text block is matched one by one with the selected multiple target language text blocks, and the target language text block with the highest matching degree is selected as the target matching text block of the i-th source language text block.

2. The method according to claim 1, characterized in that, The step of matching the i-th source language text block with each of the selected target language text blocks, and selecting the target language text block with the highest matching degree as the target matching text block of the i-th source language text block, includes: Obtain the first semantic vector corresponding to the i-th source language text block and the second semantic vector corresponding to the selected multiple target language text blocks; Calculate the semantic similarity between the first semantic vector and multiple second semantic vectors respectively; The target language text block with the highest semantic similarity is selected as the target matching text block of the i-th source language text block.

3. The method according to claim 2, characterized in that, The step of calculating the semantic similarity between the first semantic vector and multiple second semantic vectors includes: Calculate the similarity values ​​between the first semantic vector and the multiple second semantic vectors respectively, and use the similarity values ​​as the semantic similarity; Specifically, calculating the similarity values ​​between the first semantic vector and multiple second semantic vectors, and using these similarity values ​​as the semantic similarity, includes: Multiple similarity algorithms are used to calculate the similarity values ​​between the first semantic vector and multiple second semantic vectors respectively; The semantic similarity is determined based on the similarity values ​​calculated using multiple similarity algorithms.

4. The method according to claim 2, characterized in that, The step of calculating the semantic similarity between the first semantic vector and multiple second semantic vectors includes: Calculate the similarity values ​​between the first semantic vector and multiple second semantic vectors respectively; Determine the term matching weighting value between the i-th source language text block and the selected multiple target language text blocks; The semantic similarity is determined based on the similarity value and the term matching weighting value corresponding to the similarity value.

5. The method according to any one of claims 1 to 4, characterized in that, The size of the sliding window is dynamically adjusted based on the matching degree between the source language text block and the target language text block.

6. The method according to any one of claims 1 to 4, characterized in that, The size of the sliding window is dynamically adjusted based on the position of the source language text block.

7. The method according to any one of claims 1 to 4, characterized in that, The step of matching the i-th source language text block with each of the selected target language text blocks, and selecting the target language text block with the highest matching degree as the target matching text block of the i-th source language text block, includes: The i-th source language text block is matched one by one with the selected multiple target language text blocks to obtain multiple matching degree values; The target language text block with the highest matching score is selected as the candidate language text block; If the matching degree value of the candidate language text block exceeds the preset matching degree threshold, the candidate language text block is used as the target matching text block of the i-th source language text block.

8. The method according to claim 7, characterized in that, The matching degree threshold is determined in the following way: Determine the language pairs to which the languages ​​in the source language document and the target language document belong; The matching threshold is determined based on the language pair.

9. The method according to claim 7, characterized in that, The matching degree threshold is determined in the following way: Determine the text type to which the i-th source language text block belongs; The matching threshold is determined based on the text type.

10. The method according to claim 1, characterized in that, The method further includes: After completing the matching process for all source language text blocks in the source language document, it is detected whether there are multiple source language text blocks corresponding to the same target matching text block; If multiple source language text blocks correspond to the same target matching text block, then the pair of source language text blocks and target matching text blocks with the highest matching degree will be retained.

11. The method according to claim 1, characterized in that, The method further includes: After matching all source language text blocks in the source language document, quality inspection is performed on all resulting language text block pairs, which include source language text blocks and their corresponding target matching text blocks. After the language text block pair fails the quality inspection process, the language text block pair is marked as abnormal. The quality inspection process includes at least one of the following: length ratio check, terminology consistency check, and back-translation check.

12. A multilingual text alignment device, characterized in that, The device includes: The preprocessing module is used to preprocess the source language document and the target language document respectively, to obtain multiple source language text blocks and multiple target language text blocks; The selection module is used to select multiple target language text blocks from the target language document based on a preset sliding window, with the i-th source language text block of the source language document as the anchor point and the i-th target language text block of the target language document as the anchor point. i=1,2,...n, where n is the number of source language text blocks contained in the source language document. The matching module is used to match the i-th source language text block with the selected multiple target language text blocks one by one, and select the target language text block with the highest matching degree as the target matching text block of the i-th source language text block.

13. A computer device, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein: The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in claims 1 to 11.

Images