Automatic webpage translation method and system, medium and equipment

Abstract

The invention discloses an automatic webpage translation method and system, a medium and equipment, and the method comprises the following steps: a language detection step: analyzing character codes, vocabulary distribution and grammar structures of webpage contents through a natural language processing technology, and determining the language type of a webpage; a webpage field detection step: automatically identifying the field type to which the webpage belongs and selecting the translation model of the corresponding field by analyzing the theme, keyword and context of the webpage content; a webpage translation selection step: carrying out full-text translation or self-defined translation selection on the webpage, if the full-text translation is selected, carrying out overall translation on webpage contents, and if the self-defined translation is selected, carrying out cutting and box selection on foreign language contents in the webpage; according to the webpage field detection method and device, through setting of the webpage field detection step, the field type to which the webpage belongs can be automatically recognized, the translation model of the corresponding field can be selected, and therefore the accuracy of webpage translation is improved.

Description

Technical Field

[0001] The present invention relates to the field of web page processing, and in particular to a method, system, medium and device for automatic translation of a web page. Background Art

[0002] With the development of the internet, web pages have become an important way for people to obtain information. However, due to language barriers, many users cannot directly understand the content of web pages in their non-native language, which limits the flow and sharing of information. To address this problem, automatic web translation technology has emerged. Traditional web translation methods typically rely on manual translation or rule-based machine translation, which suffer from low translation efficiency and unstable translation quality. Therefore, a more efficient and accurate automatic web translation method is needed to meet user demand for multilingual web content. Summary of the Invention

[0003] The technical problem solved by the present invention is to provide a method for automatically translating a web page to improve the accuracy of the translation.

[0004] The technical solution adopted by the present invention to solve the technical problem is: a method for automatic translation of a web page, comprising the following steps:

[0005] Language detection step: Using natural language processing technology, analyze the character encoding, vocabulary distribution, and grammatical structure of the web page content to determine the main language type of the web page;

[0006] Web page domain detection step: By analyzing the theme, keywords and context of the web page content, the domain type of the web page is automatically identified and the corresponding translation model is selected;

[0007] Webpage translation selection steps: Choose between full-text translation or custom translation. If you choose full-text translation, the entire webpage content will be translated. If you choose custom translation, you can select the Chinese and foreign language content on the webpage.

[0008] Custom translation steps: Check the boxes that need to be translated, and translate the checked content using the translation model of the corresponding field.

[0009] Furthermore, it also includes the steps of independent translation, specifically:

[0010] Users can add, edit, and manage custom dictionaries that include source and target language translations of terms;

[0011] During translation, if there is a term in the custom dictionary that matches the source text, the translation result in the custom dictionary is used directly; otherwise, the translation model is used for translation.

[0012] Furthermore, it also includes real-time translation caching and incremental update steps, specifically including:

[0013] Cache translation results, using hash tables or databases to store translated text segments;

[0014] For dynamically loaded content, use incremental translation technology to translate the newly loaded text;

[0015] By monitoring the DOM changes of the web page, dynamic content is captured in real time and translation is triggered.

[0016] Furthermore, it also includes translation quality assessment and optimization steps, specifically:

[0017] Conduct quality assessment on the translated text and adjust or optimize the translation model parameters based on the assessment results;

[0018] Collect user feedback, manually correct the translation results, and use the corrected data for iterative training of the model.

[0019] Furthermore, the language detection step also includes a pre-processing step before the language detection step, specifically:

[0020] Removing text content: Identifying and removing advertisements, scripts, and stylesheets from web pages by analyzing the DOM structure of the web page.

[0021] Text content extraction: extract the core text content from the preprocessed web pages;

[0022] Preprocessing result cache: cache the preprocessed web page content and use hash tables or databases to store the processed web page content;

[0023] Dynamic content monitoring: By monitoring the DOM changes of the web page, dynamically loaded content can be captured in real time, and the newly loaded content can be pre-processed.

[0024] The present invention also discloses a webpage automatic translation system, which includes a language detection module, a webpage domain detection module, a webpage translation selection module and a custom translation module;

[0025] The language detection module is used to analyze the character encoding, vocabulary distribution and grammatical structure of the web page content through natural language processing technology to determine the main language type of the web page;

[0026] The webpage domain detection module is used to automatically identify the domain type of the webpage by analyzing the subject, keywords and context of the webpage content and select a translation model for the corresponding domain;

[0027] The webpage translation selection module is used to select full-text translation or custom translation of the webpage. If full-text translation is selected, the webpage content is translated as a whole. If custom translation is selected, the Chinese and foreign language content of the webpage is segmented and selected.

[0028] The custom translation module is used to check the boxes that need to be translated, and translate the checked content using the translation model of the corresponding field.

[0029] Furthermore, it also includes an independent translation module, specifically:

[0030] Users can add, edit, and manage custom dictionaries that include source and target language translations of terms;

[0031] During translation, if there is a term in the custom dictionary that matches the source text, the translation result in the custom dictionary is used directly; otherwise, the translation model is used for translation.

[0032] Furthermore, it also includes a real-time translation cache and incremental update module, specifically including:

[0033] Cache translation results, using hash tables or databases to store translated text segments;

[0034] For dynamically loaded content, use incremental translation technology to translate the newly loaded text;

[0035] By monitoring the DOM changes of the web page, dynamic content is captured in real time and translation is triggered.

[0036] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for automatic web page translation are implemented.

[0037] The present invention also discloses a computer device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; wherein:

[0038] The memory is used to store computer programs;

[0039] The processor is used to execute the steps of the above-mentioned method for automatic webpage translation by running the program stored in the memory.

[0040] The beneficial effects of the present invention are:

[0041] 1. The present invention can automatically identify the domain type of a web page and select a translation model for the corresponding domain through the setting of the web page domain detection step, thereby improving the accuracy of web page translation.

[0042] 2. The present invention allows users to add, edit and manage custom dictionaries through the setting of autonomous translation steps, thereby achieving accurate translation of specific terms and further improving the accuracy of translation.

[0043] 3. The present invention realizes real-time translation of dynamically loaded content through the setting of real-time translation cache and incremental update steps, avoids repeated translation, and improves translation efficiency.

[0044] 4. The present invention can continuously optimize the translation model and improve the translation quality by setting translation quality evaluation and optimization steps, thereby meeting users' demand for high-quality translation.

[0045] 5. The present invention can remove irrelevant content from web pages and extract core text through the setting of pre-processing steps, thereby providing more accurate and efficient input for subsequent translation steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the flow of the automatic webpage translation method according to an embodiment of the present application.

[0047] Figure 2 This is a schematic diagram of the framework of the automatic webpage translation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0049] like Figure 1 As shown, the embodiment of the present application discloses a method for automatic translation of a web page, comprising the following steps:

[0050] Language detection step: Using natural language processing technology, analyze the character encoding, vocabulary distribution, and grammatical structure of the web page content to determine the main language type of the web page;

[0051] Web page domain detection step: By analyzing the theme, keywords and context of the web page content, the domain type of the web page is automatically identified and the corresponding translation model is selected;

[0052] Webpage translation selection steps: Choose between full-text translation or custom translation. If you choose full-text translation, the entire webpage content will be translated. If you choose custom translation, you can select the Chinese and foreign language content on the webpage.

[0053] Custom translation steps: Check the boxes that need to be translated, and translate the checked content using the translation model of the corresponding field.

[0054] Specifically, during the language detection step, the system first uses natural language processing (NLP) technology to conduct an in-depth analysis of the character encoding of the webpage content. By analyzing the character encoding, the system can initially narrow down the possible languages. Next, the system further analyzes the vocabulary distribution of the webpage content. Different languages have significant differences in vocabulary usage, such as vocabulary composition, word frequency, and the presence of specific words. By analyzing these lexical features, the system can further determine the primary language type of the webpage. After determining the language type, it analyzes the content of the webpage, such as news, technology, and entertainment, and selects the most appropriate translation model based on the domain type. Professional terminology and expressions vary significantly across different fields, and selecting a translation model specifically for that domain can significantly improve translation accuracy and professionalism. For example, webpages in the technology field contain a large amount of professional terminology and technical descriptions. Selecting a translation model specifically for that domain can more accurately translate this specialized content, avoiding mistranslations of terminology or unclear expressions. After determining the webpage's domain, the system will choose between full text translation or customized translation based on the user's needs. If the user selects full text translation, the system will translate the entire webpage content, ensuring that all information on the webpage is accurately translated. If the user chooses custom translation, the system allows the user to cut and select the foreign language content on the web page and only translate the content that the user is interested in or needs, which improves the flexibility and practicality of the translation.

[0055] In this embodiment, an autonomous translation step is also included, specifically:

[0056] Users can add, edit, and manage custom dictionaries that include source and target language translations of terms;

[0057] During translation, if there is a term in the custom dictionary that matches the source text, the translation result in the custom dictionary is used directly; otherwise, the translation model is used for translation.

[0058] Specifically, the autonomous translation step allows users to customize translation content based on personal needs or specialized terminology in specific fields. Users can easily add, edit, and manage custom dictionaries through a simple operation interface. This custom dictionary contains translation pairs of the source language and target language of the term. When translating, the system will first check whether there is a term in the custom dictionary that matches the source text. If a matching term is found, the system will directly use the translation result in the custom dictionary, which can ensure the accurate translation of the specific term. If no matching term is found in the custom dictionary, the system will use a general translation model for translation.

[0059] This autonomous translation method greatly improves the accuracy and flexibility of translation, and is especially suitable for users who need to frequently deal with professional terminology or content in specific fields.

[0060] This embodiment also includes real-time translation caching and incremental updating steps, specifically including:

[0061] Cache translation results, using hash tables or databases to store translated text segments;

[0062] For dynamically loaded content, use incremental translation technology to translate the newly loaded text;

[0063] By monitoring the DOM changes of the web page, dynamic content is captured in real time and translation is triggered.

[0064] Specifically, during the translation process, the system will cache the translated text fragments. By using efficient data storage structures such as hash tables or databases, the system can quickly store and retrieve translation results, avoiding repeated translations, thereby significantly improving translation efficiency. In addition, for dynamically loaded content in web pages, the system uses incremental translation technology. This technology can intelligently identify newly loaded text and only translate the new content without having to re-translate the entire web page. This not only further improves the translation speed, but also effectively reduces the consumption of system resources. At the same time, by monitoring the DOM (Document Object Model) changes of the web page, the system can capture dynamic content in the web page in real time and trigger translation immediately. This function ensures the real-time update of web page content and the synchronization of translation, allowing users to obtain the latest and most accurate translation results at any time.

[0065] This embodiment also includes translation quality assessment and optimization steps, specifically:

[0066] Conduct quality assessment on the translated text and adjust or optimize the translation model parameters based on the assessment results;

[0067] Collect user feedback, manually correct the translation results, and use the corrected data for iterative training of the model.

[0068] Specifically, during the translation quality assessment and optimization step, the system comprehensively analyzes the translated text using a variety of evaluation metrics, including accuracy, fluency, and semantic consistency. By comprehensively considering these metrics, the system objectively evaluates translation quality and, accordingly, makes targeted parameter adjustments or optimizations to the translation model. This feedback-based continuous optimization mechanism continuously improves the performance of the translation model, making it more adaptable to diverse translation needs.

[0069] The system also prioritizes user feedback. Users can manually correct translations through a simple interface and provide the corrected translation. The system then uses this corrected data for iterative model training. Through continuous learning and improvement, it gradually reduces translation errors and improves translation accuracy and reliability.

[0070] In this embodiment, the language detection step further includes a preprocessing step, specifically:

[0071] Removing text content: Identifying and removing advertisements, scripts, and stylesheets from web pages by analyzing the DOM structure of the web page.

[0072] Text content extraction: extract the core text content from the preprocessed web pages;

[0073] Preprocessing result cache: cache the preprocessed web page content and use hash tables or databases to store the processed web page content;

[0074] Dynamic content monitoring: By monitoring the DOM changes of the web page, dynamically loaded content can be captured in real time, and the newly loaded content can be pre-processed.

[0075] Specifically, during the preprocessing step, the system first conducts an in-depth analysis of the web page, focusing on its DOM structure. The DOM structure is the web page's Document Object Model, which describes the relationships and hierarchy between elements on the web page. By analyzing the DOM structure, the system can accurately identify and remove irrelevant content from the web page, such as advertisements, scripts, and styles. After removing irrelevant content, the system further extracts the core text content from the web page. To improve translation efficiency, the system also caches the preprocessed web page content. By using efficient data storage structures such as hash tables or databases, the system can quickly store and retrieve processed web page content. This allows the system to directly access cached content during subsequent translation, avoiding duplicate processing and significantly improving translation efficiency. Furthermore, the system features dynamic content monitoring. By monitoring changes in the web page's DOM, the system can capture dynamically loaded content in real time and preprocess the newly loaded content. This feature ensures that web page content is updated in real time and translation proceeds simultaneously, ensuring that users always receive the latest and most accurate translation results.

[0076] like Figure 2 As shown, the present invention also discloses a web page automatic translation system, including a language detection module, a web page domain detection module, a web page translation selection module and a custom translation module;

[0077] The language detection module is used to analyze the character encoding, vocabulary distribution and grammatical structure of the web page content through natural language processing technology to determine the main language type of the web page;

[0078] The webpage domain detection module is used to automatically identify the domain type of the webpage by analyzing the subject, keywords and context of the webpage content and select a translation model for the corresponding domain;

[0079] The webpage translation selection module is used to select full-text translation or custom translation of the webpage. If full-text translation is selected, the webpage content is translated as a whole. If custom translation is selected, the Chinese and foreign language content of the webpage is segmented and selected.

[0080] The custom translation module is used to check the boxes that need to be translated, and translate the checked content using the translation model of the corresponding field.

[0081] In this embodiment, an autonomous translation module is also included, specifically:

[0082] Users can add, edit, and manage custom dictionaries that include source and target language translations of terms;

[0083] During translation, if there is a term in the custom dictionary that matches the source text, the translation result in the custom dictionary is used directly; otherwise, the translation model is used for translation.

[0084] This embodiment also includes a real-time translation cache and incremental update module, specifically including:

[0085] Cache translation results, using hash tables or databases to store translated text segments;

[0086] For dynamically loaded content, use incremental translation technology to translate the newly loaded text;

[0087] By monitoring the DOM changes of the web page, dynamic content is captured in real time and translation is triggered.

[0088] The present invention can automatically identify the domain type of a web page and select a translation model for the corresponding domain through the setting of a web page domain detection step, thereby improving the accuracy of web page translation. At the same time, the present invention allows users to add, edit and manage custom dictionaries through the setting of an autonomous translation step, thereby achieving accurate translation of specific terms, further improving the accuracy of translation.

[0089] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for automatic web page translation are implemented.

[0090] In addition, the computer-readable storage medium of this embodiment may adopt any combination of one or more computer-readable storage media, wherein the computer-readable storage medium includes electrical, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination thereof.

[0091] The present invention also discloses a computer device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; wherein:

[0092] The memory is used to store computer programs;

[0093] The processor is used to execute the steps of the above-mentioned method for automatic webpage translation by running the program stored in the memory.

[0094] As an embodiment of the present invention, the communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0095] As an embodiment of the present invention, the communication interface is used for communication between the above-mentioned terminal and other devices.

[0096] As an embodiment of the present invention, the memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Optionally, the memory may also be at least one storage device located remote from the processor.

[0097] As an embodiment of the present invention, the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0098] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatic translation of a web page, characterized in that: The steps include: Language detection step: Using natural language processing technology, analyze the character encoding, vocabulary distribution, and grammatical structure of the web page content to determine the main language type of the web page; Web page domain detection step: By analyzing the theme, keywords and context of the web page content, the domain type of the web page is automatically identified and the corresponding translation model is selected; Webpage translation selection steps: Choose between full-text translation or custom translation. If you choose full-text translation, the entire webpage content will be translated. If you choose custom translation, you can select the Chinese and foreign language content on the webpage. Custom translation steps: Check the boxes that need to be translated, and translate the checked content using the translation model of the corresponding field.

2. The method for automatic web page translation according to claim 1, wherein: It also includes independent translation steps, specifically: Users can add, edit, and manage custom dictionaries that include source and target language translations of terms; During translation, if there is a term in the custom dictionary that matches the source text, the translation result in the custom dictionary is used directly; otherwise, the translation model is used for translation.

3. The method for automatic web page translation according to claim 1, wherein: It also includes real-time translation caching and incremental update steps, including: Cache translation results, using hash tables or databases to store translated text segments; For dynamically loaded content, use incremental translation technology to translate the newly loaded text; By monitoring the DOM changes of the web page, dynamic content is captured in real time and translation is triggered.

4. The method for automatic web page translation according to claim 1, wherein: It also includes translation quality assessment and optimization steps, specifically: Conduct quality assessment on the translated text and adjust or optimize the translation model parameters based on the assessment results; Collect user feedback, manually correct the translation results, and use the corrected data for iterative training of the model.

5. The method for automatic web page translation according to claim 1, wherein: The language detection step also includes a pre-processing step, specifically: Removing text content: Identifying and removing advertisements, scripts, and stylesheets from web pages by analyzing the DOM structure of the web page. Text content extraction: extract the core text content from the preprocessed web pages; Preprocessing result cache: cache the preprocessed web page content and use hash tables or databases to store the processed web page content; Dynamic content monitoring: By monitoring the DOM changes of the web page, dynamically loaded content can be captured in real time, and the newly loaded content can be pre-processed.

6. The automatic webpage translation system is characterized by: It includes language detection module, web page domain detection module, web page translation selection module and custom translation module; The language detection module is used to analyze the character encoding, vocabulary distribution and grammatical structure of the web page content through natural language processing technology to determine the main language type of the web page; The webpage domain detection module is used to automatically identify the domain type of the webpage by analyzing the subject, keywords and context of the webpage content and select a translation model for the corresponding domain; The webpage translation selection module is used to select full-text translation or custom translation of the webpage. If full-text translation is selected, the webpage content is translated as a whole. If custom translation is selected, the Chinese and foreign language content of the webpage is segmented and selected. The custom translation module is used to check the boxes that need to be translated, and translate the checked content using the translation model of the corresponding field.

7. The webpage automatic translation system according to claim 6, wherein: It also includes independent translation modules, specifically: Users can add, edit, and manage custom dictionaries that include source and target language translations of terms; During translation, if there is a term in the custom dictionary that matches the source text, the translation result in the custom dictionary is used directly; otherwise, the translation model is used for translation.

8. The webpage automatic translation system according to claim 6, wherein: It also includes real-time translation cache and incremental update modules, including: Cache translation results, using hash tables or databases to store translated text segments; For dynamically loaded content, use incremental translation technology to translate the newly loaded text; By monitoring the DOM changes of the web page, dynamic content is captured in real time and translation is triggered.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for automatic webpage translation according to any one of claims 1 to 5 are implemented.

10. A computer device, characterized in that: The system comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: The memory is used to store computer programs; The processor is configured to execute the steps of the webpage automatic translation method according to any one of claims 1 to 5 by running the program stored in the memory.

Images