Method, System, Device and Medium for Aligning Bilingual Text Paragraphs within a Single Web Page

Abstract

The present invention discloses a method, system, device and medium for aligning bilingual text paragraphs within a single web page, belonging to the field of bilingual corpus construction in natural language processing. The method includes: parsing the web page to extract visible text from the web page, successively cleaning and dividing the extracted text into paragraphs to obtain a list of text paragraphs; filtering the paragraphs in the list of text paragraphs according to set filtering conditions; calculating the ratio between the number of paragraphs in the first language and the number of paragraphs in the second language in the filtered paragraphs, and judging whether the web page is a bilingual web page according to the relationship between the ratio and the set ratio range; when the web page is a bilingual web page, aligning the filtered paragraphs in the web page for bilingual text paragraphs. It can identify and align bilingual text paragraphs from a single noisy bilingual web page, and improve the recall rate and precision rate of bilingual text paragraph alignment in a single web page.

Description

Technical Field

[0001] The present invention belongs to the field of bilingual corpus construction in natural language processing, and more specifically, relates to a method, system, device and medium for aligning bilingual text paragraphs within a single web page. Background Art

[0002] A bilingual corpus (or parallel corpus) provides the translation of a text in one language (i.e., the source language) into another language (i.e., the target language), and it is the data basis for machine translation, cross-language information retrieval, and numerous multilingual language processing tasks based on machine learning methods. Although manually creating a bilingual corpus has relatively high corpus quality, it is costly and has a relatively limited scale. Through the Web, web pages can be crawled by a program, and a much larger and more diverse bilingual corpus can be constructed through means such as cleaning, alignment, and filtering.

[0003] Currently, the basic idea of obtaining aligned sentences or paragraphs from crawled Web pages is as follows: First, perform document alignment between web pages, and the aligned documents are mutual translation documents; then, perform sentence or paragraph alignment between the sentences of the aligned documents, and the aligned sentences or paragraphs are mutual translations. There are the following methods for sentence alignment: alignment based on the length distribution of the original and translated sentences, sentence alignment based on a bilingual dictionary, sentence alignment based on sentence embeddings, and sentence alignment that combines length and dictionary.

[0004] Currently, web page crawling has been well solved in the field of search engines, and researchers have also conducted relatively extensive research on web page alignment between monolingual web pages (such as alignment between Chinese web pages and English web pages) and sentence and paragraph alignment between clean bilingual texts. However, there is a type of foreign language learning website on the Internet, such as an English learning website. In addition to a large number of English learning web pages with mixed Chinese and English (such bilingual web pages may or may not have alignable text), these websites also include some monolingual web pages. The existing methods for alignment between monolingual web pages and alignment between clean bilingual texts cannot perform English-Chinese text paragraph alignment on such noisy single bilingual web pages. Summary of the Invention

[0005] Aiming at the defects and improvement requirements of the prior art, the present invention provides a method, system, device and medium for aligning bilingual text paragraphs within a single web page, aiming to identify and align bilingual text paragraphs from a noisy single bilingual web page and improve the recall rate and precision rate of identification and alignment.

[0006] To achieve the above object, according to one aspect of the present invention, there is provided a method for aligning bilingual text paragraphs within a single web page, including: S1, parsing the web page to extract visible text therefrom, successively cleaning and paragraphing the extracted text to obtain a list of text paragraphs; S2, filtering the paragraphs in the list of text paragraphs according to set filtering conditions; S3, calculating the ratio between the number of paragraphs in the first language and the number of paragraphs in the second language in the filtered paragraphs, and judging whether the web page is a bilingual web page according to the relationship between the ratio and the set ratio range; S4, when the web page is a bilingual web page, performing alignment of bilingual text paragraphs on the filtered paragraphs in the web page.

[0007] Furthermore, the S1 includes: S11, parsing the web page to extract visible text therefrom; S12, performing unescaping processing on HTML escape symbols in the extracted text, and respectively compressing consecutive spaces and consecutive blank lines in the extracted text into single spaces and single blank lines; S13, performing paragraphing on the text after unescaping processing to obtain an initial list of text paragraphs; S14, for each bilingual paragraph in the initial list of text paragraphs, dividing the bilingual paragraph into a first-language sub-paragraph and a second-language sub-paragraph; S15, adding the monolingual paragraphs, the first-language sub-paragraphs and the second-language sub-paragraphs in the initial list of text paragraphs to a new list in sequence as the list of text paragraphs.

[0008] Beneficial effects: Aiming at the noise in content and format in the web page, by analyzing, cleaning and converting the web page, a clean list of text paragraphs containing only one language is obtained from the web page that may mix bilingual paragraphs, laying a foundation for subsequent determination of bilingual web pages, thereby improving the precision rate of text paragraph alignment.

[0009] Furthermore, the S11 includes: using an HTML parser to parse the web page to obtain the DOM tree of the web page; traversing the DOM tree, extracting visible text according to the set collection of visible text tags, and retaining the extracted blank lines.

[0010] Furthermore, when the bilingual paragraph is a single-sentence text, the S14 includes: starting from the starting character, looking for the first bilingual switching entry, and dividing the bilingual paragraph into a first-language sub-paragraph and a second-language sub-paragraph according to the bilingual switching entry.

[0011] Furthermore, when the bilingual paragraph is a multi-sentence text, the S14 includes: starting from the starting character, looking for the first bilingual switching sentence, and dividing the bilingual paragraph into a first-language sub-paragraph and a second-language sub-paragraph according to the bilingual switching sentence.

[0012] Beneficial effects: To address the problem of variable positions and lengths of bilingual translation texts, the bilingual paragraphs are segmented into two sub-paragraphs containing different single languages respectively through the bilingual switching flags of the texts, and the monolingual paragraphs remain unchanged, improving the recall rate and precision rate of text paragraph alignment.

[0013] Furthermore, the S2 includes: filtering out the paragraphs in the text paragraph list that meet the set filtering conditions; the set filtering conditions include one or more of the following conditions: the character length of the paragraph is less than the first set threshold; the entry length of the paragraph is less than the second set threshold; the proportion of characters other than the first language and the second language in the paragraph exceeds the third set threshold; the paragraph conforms to the URL string pattern; the characters in the paragraph are languages other than the first language and the second language; the paragraph is a first-language paragraph and the proportion of second-language characters in the paragraph exceeds the fourth set threshold; the paragraph is a second-language paragraph and the proportion of first-language characters in the paragraph exceeds the fourth set threshold.

[0014] Beneficial effects: Considering the flexibility and diversity of web page texts, etc., a variety of heuristic rules are used to filter the paragraphs before paragraph alignment, excluding non-text paragraphs and non-bilingual web pages, narrowing the range of aligned paragraphs, and further improving the precision rate of bilingual text paragraph alignment.

[0015] Furthermore, the S4 includes: S41, dividing the filtered paragraphs in the web page into a first-language paragraph sub-list and a second-language paragraph sub-list; S42, using the set alignment threshold to find the anchor paragraphs in the first-language paragraph sub-list and the second-language paragraph sub-list; S43, comparing and aligning the paragraphs after the anchor paragraphs in the first-language paragraph sub-list and the second-language paragraph sub-list in sequence.

[0016] Beneficial effects: Considering the non-crossing of aligned paragraphs within the web page and the characteristic of one-to-one alignment of different-language paragraphs after cleaning and filtering, first determining the anchor paragraphs for alignment through a high alignment threshold and then performing sequential alignment can not only improve the alignment speed but also improve the alignment accuracy.

[0017] According to another aspect of the present invention, a bilingual text paragraph alignment system within a single web page is provided, including: an analysis and extraction module for analyzing the web page to extract visible text from the web page, sequentially cleaning and paragraph-dividing the extracted text to obtain a text paragraph list; a filtering module for filtering the paragraphs in the text paragraph list according to set filtering conditions; a judgment module for calculating the ratio between the number of first-language paragraphs and the number of second-language paragraphs in the filtered paragraphs, and judging whether the web page is a bilingual web page according to the relationship between the ratio and the set ratio range; an alignment module for performing bilingual text paragraph alignment on the filtered paragraphs in the web page when the web page is a bilingual web page.

[0018] According to another aspect of the present invention, there is provided an electronic device, including: a processor; a memory storing computer-executable programs, which when executed by the processor, cause the processor to execute the method for aligning bilingual text paragraphs within a single web page as described above.

[0019] According to another aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which when executed by a processor, implements the method for aligning bilingual text paragraphs within a single web page as described above.

[0020] Generally speaking, through the above technical solutions conceived by the present invention, the following beneficial effects can be achieved: providing a method for aligning bilingual text paragraphs within a single web page, considering the diversity of web page formats and the characteristics of HTML codes, cleaning and standardizing the text extracted from the web page to obtain standardized text, and for the possible translation of bilingual sentences within paragraphs in the web page, separating them by segmenting sub-paragraphs within the paragraph, improving the recall rate of paragraph alignment; before paragraph alignment, filtering out non-bilingual web pages, non-text paragraphs and paragraphs that cannot be aligned through flexible configuration based on interfaces, narrowing the alignment range and improving the precision rate of paragraph alignment; the text similarity scoring based on interfaces can be extended and customized, and a default implementation is provided; according to the order and non-crossing of text translation, aligning the first paragraph with a relatively high alignment threshold first and then aligning backward, improving the efficiency and accuracy of alignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of the method for aligning bilingual text paragraphs within a single web page provided by an embodiment of the present invention;

[0022] Figure 2 It is a block diagram of the system for aligning bilingual text paragraphs within a single web page provided by an embodiment of the present invention;

[0023] Figure 3 It is a block diagram of the electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] In order to make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0025] In the present invention, terms such as "first" and "second" in the present invention and the accompanying drawings (if any) are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0026] Figure 1 This is a flowchart of the bilingual text paragraph alignment method within a single web page provided by an embodiment of the present invention. Refer to Figure 1 , the method includes operations S1 - S4.

[0027] Operation S1: Parse the web page to extract visible text, and successively clean and divide the extracted text into paragraphs to obtain a list of text paragraphs.

[0028] According to an embodiment of the present invention, operation S1 includes sub - operations S11 - S15.

[0029] In sub - operation S11, parse the web page to extract visible text.

[0030] According to an embodiment of the present invention, sub - operation S11 includes: Use an HTML parser to parse the web page to obtain the DOM tree of the web page; traverse the DOM tree, extract visible text according to the set collection of visible text tags, and retain the empty lines extracted.

[0031] Specifically, establish a DOM tree through the hierarchical inclusion relationship between HTML elements in the web page HTML code. The parent nodes in the DOM tree correspond to elements that include the child - node corresponding elements. Perform a breadth - first traversal of the DOM tree, extract text from the predefined visible HTML elements (by default including title elements H1, H2, H3, and H4, paragraph element P, block element DIV, span element SPAN, list element LI, etc., which can be configured and adjusted according to the characteristics of the web page), and retain the empty lines and spaces in the extracted text.

[0032] In sub - operation S12, perform un - escaping processing on the HTML escape symbols in the extracted text, and compress consecutive spaces and consecutive empty lines in the extracted text into single spaces and single empty lines respectively.

[0033] Specifically, un - escape the escaped HTML symbol entities (including entity numbers and entity names) in the extracted text, normalize the text using Unicode NFC, and use regular expressions to match and replace multiple consecutive spaces with a single space, and multiple consecutive empty lines with a single empty line for the normalized text.

[0034] In sub - operation S13, divide the text after un - escaping processing into paragraphs to obtain an initial list of text paragraphs.

[0035] Specifically, divide the text into paragraphs according to the line breaks in the text. Each line break indicates the end of a paragraph. Then, remove the blank paragraphs with a length of 0 or only containing whitespace to obtain an initial list of text paragraphs L1.

[0036] In sub - operation S14, for each bilingual paragraph in the initial text paragraph list, the bilingual paragraph is divided into a first - language sub - paragraph and a second - language sub - paragraph. For example, the first language is Chinese and the second language is English; or the first language is Chinese and the second language is Japanese, etc.

[0037] When the bilingual paragraph is a single - sentence text, sub - operation S14 includes: starting from the starting character, finding the first bilingual - switching entry, and dividing the bilingual paragraph into a first - language sub - paragraph and a second - language sub - paragraph according to the bilingual - switching entry.

[0038] When the bilingual paragraph is a multi - sentence text, sub - operation S14 includes: starting from the starting character, finding the first bilingual - switching sentence, and dividing the bilingual paragraph into a first - language sub - paragraph and a second - language sub - paragraph according to the bilingual - switching sentence.

[0039] In this embodiment, for example, a pluggable language - determination interface is used to determine the language type. Taking the text as input, it returns the most likely language code (English, Chinese, or others). Taking the bilingual languages as English and Chinese as an example, it counts the proportion of English symbols and Chinese symbols in the text through an English symbol set and a Chinese symbol set. When the proportion is greater than a given threshold, it is determined as the corresponding language; otherwise, it is determined as other languages.

[0040] In sub - operation S15, the monolingual paragraphs, the first - language sub - paragraphs, and the second - language sub - paragraphs in the initial text paragraph list are added to a new list in sequence as the text paragraph list.

[0041] The specific details of sub - operation S14 and sub - operation S15 are as follows. Initialize the candidate paragraph list L2, and process each paragraph p in the initial text paragraph list L1 as follows: Sentence - break the paragraph p according to the bilingual - sentence end symbol to obtain a sentence list S; when the length of the sentence list S is 1, find the first bilingual - switching entry t (the text language changes after t) from left to right. If t exists, divide the paragraph p into two parts according to it to obtain two sub - paragraphs and add them to the candidate paragraph list L2. If t does not exist, directly add the paragraph p to the candidate paragraph list L2; when the length of the sentence list S is greater than 1, find the first bilingual - switching sentence s (the text language changes after s) from left to right. If s exists, divide the paragraph p into two parts according to it to obtain two sub - paragraphs and add them to the candidate paragraph list L2. If s does not exist, directly add the paragraph p to the candidate paragraph list L2. The finally obtained candidate paragraph list L2 is the text paragraph list.

[0042] Operation S2, filter the paragraphs in the text paragraph list according to the set filtering conditions.

[0043] According to an embodiment of the present invention, in operation S2, paragraphs in the text paragraph list that meet the set filtering conditions are filtered out. The set filtering conditions include one or more of the following conditions: the character length of the paragraph is less than the first set threshold (for example, the default is 6); the number of terms in the paragraph is less than the second set threshold (for example, the default is 2); the proportion of characters other than the first language and the second language in the paragraph exceeds the third set threshold (for example, the default is 0.2); the paragraph conforms to the URL string pattern; the characters in the paragraph are in a language other than the first language and the second language; the paragraph is a first-language paragraph, and the proportion of second-language characters in the paragraph exceeds the fourth set threshold (for example, the default is 0.3); the paragraph is a second-language paragraph, and the proportion of first-language characters in the paragraph exceeds the fourth set threshold.

[0044] In operation S3, calculate the ratio between the number of first-language paragraphs and the number of second-language paragraphs in the filtered paragraphs, and determine whether the web page is a bilingual web page according to the relationship between the ratio and the set ratio range.

[0045] Only when the length of the paragraph list after filtering is non-empty and the ratio between the number of first-language paragraphs and the number of second-language paragraphs is within the set ratio range, the corresponding web page is a bilingual web page, and the corresponding filtered paragraph list can enter the next processing step. Taking the first language as English and the second language as Chinese as an example, the set ratio range is preferably the default 0.3 - 0.7.

[0046] In operation S4, when the web page is a bilingual web page, perform bilingual text paragraph alignment on the filtered paragraphs in the web page.

[0047] According to an embodiment of the present invention, operation S4 includes: sub-operations S41 - S43.

[0048] In sub-operation S41, divide the filtered paragraphs in the web page into a first-language paragraph sub-list and a second-language paragraph sub-list.

[0049] Specifically, initialize empty first-language paragraph sub-list and second-language paragraph sub-list, sequentially scan the filtered text paragraph list, and add the first-language paragraphs and the second-language paragraphs to the end of the first-language paragraph sub-list and the second-language paragraph sub-list respectively.

[0050] In sub-operation S42, use the set alignment threshold to find the anchor paragraphs in the first-language paragraph sub-list and the second-language paragraph sub-list.

[0051] Specifically, select one paragraph from each of the two sub-lists in sequence, use the scorer Scorer to score the two selected paragraphs, obtain the score of the possibility of mutual translation between the two paragraphs, and when the score higher than the set alignment threshold is obtained for the first time, it indicates that the two currently selected paragraphs are aligned, and these two paragraphs are anchor paragraphs.

[0052] In this embodiment, for example, the above scoring operation is implemented by a pluggable scorer that implements a scoring interface. The scoring interface takes a first-language paragraph and a second-language paragraph as inputs and returns a score representing the likelihood that they are translations of each other. The default scorer for implementing the scoring interface can be based on a bilingual dictionary, and the ratio of the number of words matched based on the bilingual dictionary is returned as the score representing the likelihood of being translations of each other.

[0053] In sub-operation S43, the paragraphs after the anchor paragraphs in the first-language paragraph sub-list and the second-language paragraph sub-list are compared and aligned in sequence until both paragraph sub-lists have been scanned.

[0054] Figure 2 is a block diagram of a bilingual text paragraph alignment system within a single web page provided by an embodiment of the present invention. Refer to Figure 2 , the bilingual text paragraph alignment system 200 within a single web page includes a parsing and extraction module 210, a filtering module 220, a judgment module 230, and an alignment module 240.

[0055] The parsing and extraction module 210, for example, performs operation S1, which is used to parse a web page to extract visible text, and sequentially clean and divide the extracted text to obtain a list of text paragraphs.

[0056] The filtering module 220, for example, performs operation S2, which is used to filter the paragraphs in the list of text paragraphs according to set filtering conditions.

[0057] The judgment module 230, for example, performs operation S3, which is used to calculate the ratio between the number of first-language paragraphs and the number of second-language paragraphs in the filtered paragraphs, and judge whether the web page is a bilingual web page according to the relationship between the ratio and the set ratio range.

[0058] The alignment module 240, for example, performs operation S4, which is used to perform bilingual text paragraph alignment on the filtered paragraphs in the web page when the web page is a bilingual web page.

[0059] The bilingual text paragraph alignment system 200 within a single web page is used to execute the Figure 1 bilingual text paragraph alignment method within a single web page shown in the above embodiments. For details not described in this embodiment, please refer to the Figure 1 bilingual text paragraph alignment method within a single web page shown in the foregoing embodiments, which will not be elaborated here.

[0060] An embodiment of the present disclosure also shows an electronic device, as Figure 3 shown, the electronic device 300 includes a processor 310 and a readable storage medium 320. The electronic device 300 can execute the Figure 1 bilingual text paragraph alignment method within a single web page described above.

[0061] Specifically, the processor 310 may, for example, include a general - purpose microprocessor, an instruction - set processor, and / or an associated chipset, and / or a dedicated microprocessor (e.g., an application - specific integrated circuit (ASIC)), etc. The processor 310 may also include on - board memory for caching purposes. The processor 310 may be a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiments of the present disclosure described in reference Figure 1 description.

[0062] The readable storage medium 320 may be, for example, any medium capable of containing, storing, transmitting, propagating, or transporting instructions. For example, the readable storage medium may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, components, or propagation media. Specific examples of the readable storage medium include: magnetic storage devices, such as magnetic tapes or hard disk drives (HDDs); optical storage devices, such as compact discs (CD - ROMs); memories, such as random access memories (RAMs) or flash memories; and / or wired / wireless communication links.

[0063] The readable storage medium 320 may include a computer program 321, and the computer program 321 may include code / computer - executable instructions, which, when executed by the processor 310, cause the processor 310 to execute, for example, the method flow described above in connection with Figure 1 and any variations thereof.

[0064] The computer program 321 may be configured to have computer program code, for example, including computer program modules. For example, in an exemplary embodiment, the code in the computer program 321 may include one or more program modules, such as module 321A, module 321B,.... It should be noted that the way of dividing the modules and the number of modules are not fixed, and those skilled in the art can use appropriate program modules or combinations of program modules according to the actual situation. When these combinations of program modules are executed by the processor 310, the processor 310 can execute, for example, the method flow described above in connection with Figure 1 and any variations thereof.

[0065] Embodiments of the present invention also provide a computer - readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements Figure 1 the method for aligning bilingual text paragraphs within a single web page shown.

[0066] It is easy for those skilled in the art to understand that the above - mentioned are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for aligning bilingual text paragraphs within a single web page, characterized in that: include: S1, parsing a web page to extract visible text from the web page, cleaning and dividing the extracted text into paragraphs in sequence, and obtaining a text paragraph list; S2, filtering the paragraphs in the text paragraph list according to the set filtering conditions; S3, calculating the ratio between the number of paragraphs in the first language and the number of paragraphs in the second language in the filtered paragraphs, and determining whether the webpage is a bilingual webpage based on a relationship between the ratio and a set ratio range; S4, when the webpage is a bilingual webpage, performing bilingual text paragraph alignment on the filtered paragraphs in the webpage; Said S1 comprises: S11, parsing the web page to extract visible text from the web page; S12, de-escaping the HTML escape symbols in the extracted text, and compressing the extracted continuous spaces and continuous blank lines into single spaces and single blank lines respectively; S13, dividing the de-escaped text into paragraphs to obtain an initial text paragraph list; S14, for each bilingual paragraph in the initial text paragraph list, dividing the bilingual paragraph into a first language sub-paragraph and a second language sub-paragraph; S15, adding the monolingual paragraph, the first language sub-paragraph, and the second language sub-paragraph in the initial text paragraph list to a new list in order as the text paragraph list; The S4 includes: S41, dividing the filtered paragraphs in the webpage into a first language paragraph sublist and a second language paragraph sublist; S42, using a set alignment threshold, searching for anchor paragraphs in the first language paragraph sublist and the second language paragraph sublist; S43: Compare and align the paragraphs following the anchor paragraph in the first language paragraph sub-list and the second language paragraph sub-list in sequence.

2. The method for aligning bilingual text paragraphs in a single web page according to claim 1, wherein: The S11 includes: Parsing a web page using an HTML parser to obtain a DOM tree of the web page; The DOM tree is traversed, visible text is extracted according to a set visible text tag set, and the extracted blank lines are retained.

3. The method for aligning bilingual text paragraphs in a single web page according to claim 1, wherein: When the bilingual paragraph is a single-sentence text, S14 includes: starting from the starting character, searching for the first bilingual switching term, and dividing the bilingual paragraph into a first language sub-paragraph and a second language sub-paragraph according to the bilingual switching term.

4. The method for aligning bilingual text paragraphs in a single web page according to claim 1, wherein: When the bilingual paragraph is a multi-sentence text, S14 includes: starting from the start character, searching for the first bilingual switching sentence, and dividing the bilingual paragraph into a first language sub-paragraph and a second language sub-paragraph according to the bilingual switching sentence.

5. The method for aligning bilingual text paragraphs in a single web page according to claim 1, wherein: Said S2 comprises: filtering out paragraphs in said text paragraph list that meet said set filtering condition; The set filtering conditions include one or more of the following conditions: the character length of the paragraph is less than a first set threshold; the term length of the paragraph is less than a second set threshold; the proportion of characters in the paragraph other than the first language and the second language exceeds a third set threshold; the paragraph conforms to a URL string pattern; the characters in the paragraph are in a language other than the first language and the second language; the paragraph is a first language paragraph, and the proportion of the second language characters in the paragraph exceeds a fourth set threshold; the paragraph is a second language paragraph, and the proportion of the first language characters in the paragraph exceeds the fourth set threshold.

6. A bilingual text paragraph alignment system within a single web page, characterized by: include: A parsing and extraction module is used to parse the web page to extract visible text from the web page, clean the extracted text and divide it into paragraphs to obtain a text paragraph list; A filtering module, configured to filter the paragraphs in the text paragraph list according to set filtering conditions; a determination module configured to calculate a ratio between the number of paragraphs in the first language and the number of paragraphs in the second language in the filtered paragraphs, and determine whether the webpage is a bilingual webpage based on a relationship between the ratio and a set ratio range; an alignment module, configured to align the filtered paragraphs in the webpage in bilingual text when the webpage is a bilingual webpage; The parsing and extraction module parses the web page to extract visible text from the web page, cleans and divides the extracted text into paragraphs in sequence, and obtains a text paragraph list, including: S11, parsing the web page to extract visible text from the web page; S12, de-escaping the HTML escape symbols in the extracted text, and compressing the extracted continuous spaces and continuous blank lines into single spaces and single blank lines respectively; S13, dividing the de-escaped text into paragraphs to obtain an initial text paragraph list; S14, for each bilingual paragraph in the initial text paragraph list, dividing the bilingual paragraph into a first language sub-paragraph and a second language sub-paragraph; S15, adding the monolingual paragraph, the first language sub-paragraph, and the second language sub-paragraph in the initial text paragraph list to a new list in order as the text paragraph list; The alignment module, when the webpage is a bilingual webpage, performs bilingual text paragraph alignment on the filtered paragraphs in the webpage, including: S41, dividing the filtered paragraphs in the webpage into a first language paragraph sublist and a second language paragraph sublist; S42, using a set alignment threshold, searching for anchor paragraphs in the first language paragraph sublist and the second language paragraph sublist; S43: Compare and align the paragraphs following the anchor paragraph in the first language paragraph sub-list and the second language paragraph sub-list in sequence.

7. An electronic device, characterized in that: include: processor; A memory storing a computer executable program, wherein when the program is executed by the processor, the processor executes the method for aligning bilingual text paragraphs in a single web page according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for aligning bilingual text paragraphs in a single web page according to any one of claims 1 to 5 is implemented.

Images