Method, device and computer program product for generating Chinese newsletter for English website

Abstract

The present invention provides a method, device and computer program product for generating Chinese newsletters for English websites. The method extracts text content from English web pages in the form of a plug-in embedded in a browser, and generates Chinese newsletters by scheduling a pre-trained English website newsletter generation model through an interface. The English website newsletter generation model is fine-tuned by LORA based on a question-and-answer data set containing English web page content and its corresponding Chinese newsletter, and can accurately output Chinese newsletters for website content in an English context, thereby improving accuracy and professionalism.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and in particular to a method, device and computer program product for generating Chinese short messages for an English website. Background Art

[0002] In today's information age, open source intelligence is playing an increasingly important role as a key means of obtaining information, insight into trends and assisting decision-making. It involves extensive collection, in-depth analysis and use of publicly accessible information resources from public channels such as the Internet, social media, and news media. However, with the exponential growth of data volume, manual screening and analysis of this information has become increasingly difficult, which is not only time-consuming and labor-intensive, but also prone to omissions and errors. Therefore, it is particularly important to automatically generate Chinese newsletters of open source intelligence to support fast and accurate intelligence analysis and decision-making.

[0003] In recent years, large language models have made significant breakthroughs in generating text. They have demonstrated powerful text generation, understanding, and generalization capabilities, and their cross-domain application potential is also enormous. These models can not only understand complex instructions, but also generate text responses close to human level, which is amazing. Large language models perform particularly well in general knowledge and a wide range of skills. They can conduct multi-round conversations, answer complex questions, provide creative solutions, summarize text, generate code, and even participate in creative writing and other tasks.

[0004] Although large language models have demonstrated excellent capabilities in many fields, they still face some limitations when dealing with open source intelligence. Although the knowledge covered by general pre-trained models is extensive, it is often not in-depth enough. For military questions, the model may not be able to provide accurate and professional answers. This is because the field of open source intelligence involves a large amount of professional terminology and knowledge in specific fields, which requires the model to have a deeper understanding and judgment ability. In particular, open source intelligence data in the English context lacks accurate understanding capabilities and cannot obtain effective Chinese news. Therefore, a new solution is urgently needed to generate accurate Chinese news for English web pages. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide a method, device and computer program product for generating Chinese short messages for English websites to eliminate or improve one or more defects in the prior art and solve the problem that the prior art cannot generate Chinese short messages with accurate meanings for English websites.

[0006] One aspect of the present invention provides a method for generating Chinese newsletters for an English website, the method comprising the following steps:

[0007] Calling a plug-in written in a preset computer language in the browser, parsing the HTML structure of the English webpage to obtain DOM structure data, parsing the DOM structure data, and extracting multiple paragraphs of text content marked by paragraph numbers;

[0008] Cleaning the text contents of the plurality of paragraphs, removing single words and texts shorter than a set length, and merging them to obtain the original English text contents;

[0009] The English website newsletter generation model is scheduled to generate Chinese newsletters through a preset interface; wherein the English website newsletter generation model is obtained by fine-tuning the general large language model through LORA based on a preset question-and-answer data set, and each sample in the question-and-answer data set contains the original article content of an English web page and its corresponding Chinese newsletter.

[0010] In some embodiments, the preset computer language uses JavaScript, and the plug-in setting interface uses HTTP request scheduling to access the English website newsletter generation model.

[0011] In some embodiments, the universal macro model adopts the Qwen1.5-14B model, the MOSS model, the Bloom model, the Llama model, the GLM model or the Baichuan-13B model.

[0012] In some embodiments, the step of forming the question-answer dataset includes:

[0013] Collect English original texts from multiple English web pages and construct an English original text set T = {t1,…,t j ,…,t m}, where t j is the jth English original text, and m is the number of the English original texts; the English original texts include an equal number of long text categories and short text categories, and both the long text category and the short text category contain an equal number of English original texts in multiple fields; the fields include: news articles, academic articles, and social media articles;

[0014] Preprocess the original English text to remove special symbols and invalid characters, replace the same expression and inject noise to achieve data enhancement;

[0015] Get the Chinese short message corresponding to each English original text, and construct a Chinese short message set C = {c1,…,c j ,…,c m}, c j is the jth Chinese message, and m is the number of Chinese messages; wherein the length of the Chinese message is controlled to be less than or equal to the set word length;

[0016] The pre-processed English original text set and the Chinese newsletter set are compared and merged to form the question and answer data set.

[0017] In some embodiments, the method also includes: in the process of performing LORA fine-tuning on the general large language model based on a preset question-answering dataset, each round of iteration freezes the parameters of the attention layer in the general large language model at a ratio of 90% to 99%, and updates the parameters of the remaining part.

[0018] In some embodiments, the method further includes, during the LORA fine-tuning process of the general large language model based on a preset question-answering dataset, constructing a cross entropy loss to update the parameters according to the predicted text of the general large language model and the corresponding text in the Chinese short message set, and the expression of the loss function is:

[0019]

[0020] Where T is the target length of the predicted text, x represents the original English text, and y t represents the t-th subword unit of the predicted text, y 1：t-1 represents all subword units generated before the tth word embedding of the predicted text; P(y t |y 1：t-1 , x) represents the general large language model predicting the current sub-word unit y given the English original text x and the first t-1 sub-word units that have been generated t Probability.

[0021] In some embodiments, the method further comprises:

[0022] In the process of fine-tuning the general large language model based on the preset question-answering dataset, the predicted text is scored based on the pre-trained evaluation model, and the score is introduced into the loss function to construct a joint loss for parameter update. The expression is:

[0023]

[0024] S′(y)=sigmoid(S(y));

[0025] Where T is the target length of the predicted text, x represents the original English text, and y t represents the t-th subword unit of the predicted text, y 1：t-1 represents all subword units generated before the tth word embedding of the predicted text; P(y t |y 1：t-1, x) represents the general large language model predicting the current sub-word unit y given the English original text x and the first t-1 sub-word units that have been generated t Probability; S(y) represents the score of the predicted text.

[0026] In some embodiments, a Chinese newsletter generation device for an English website includes a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0027] In some embodiments, a computer-readable storage medium stores a computer program / instruction thereon, which implements the steps of the above method when executed by a processor.

[0028] In some embodiments, a computer program product includes a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0029] The beneficial effects of the present invention are at least:

[0030] The method, device and computer program product for generating Chinese newsletters for English websites described in the present invention extract text content from English web pages by embedding a plug-in in a browser, and generate Chinese newsletters by scheduling a pre-trained English website newsletter generation model through an interface; the English website newsletter generation model is fine-tuned by LORA based on a question-and-answer data set containing English web page content and its corresponding Chinese newsletter, and can accurately output Chinese newsletters for website content in an English context, thereby improving accuracy and professionalism.

[0031] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0032] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0034] Figure 1The figure is a flow chart of a method for generating Chinese short messages for an English website according to an embodiment of the present invention.

[0035] Figure 2 Schematic diagram of the process of forming a question and answer data set in the method for generating Chinese newsletters for an English website according to another embodiment of the present invention.

[0036] Figure 3 The figure is a logical diagram of a method for training a Chinese newsletter generation model for an English website based on a large language model according to an embodiment of the present invention.

[0037] Figure 4 A logical schematic diagram is formed by re-forming a question and answer data set for a training method of a Chinese newsletter generation model for an English website based on a large language model according to an embodiment of the present invention.

[0038] Figure 5 The figure is a logic diagram of forming a Chinese newsletter generation model for an English website according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0040] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0041] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0042] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0043] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0044] In view of the problems of low efficiency, limited accuracy, and difficulty in discovering deep-level associations in open source intelligence processing, the present invention begins to try to apply large models to this field in order to find a more effective solution. In the invention, a dialogue-based data set OSINTD is constructed using open source intelligence field data collected online. By fine-tuning the Qwen1.5-14B model, a large language model Qwen1.5-OSINT specifically for the field of open source intelligence is proposed. This model not only inherits the advantages of the general large language model, but also optimizes and customizes it for the field of open source intelligence, so that it has higher accuracy and professionalism when processing texts in this field. At the same time, a text message generation plug-in is designed to realize the function of generating Chinese text messages for English original pages with one click. This innovation not only greatly improves the work efficiency of intelligence personnel, but also revolutionizes the way of intelligence collection, making intelligence analysis faster and more accurate.

[0045] Specifically, the present invention provides a method for generating Chinese short messages for an English website, such as Figure 1 As shown, the method includes the following steps S101 to S103:

[0046] Step S101: calling a plug-in in a browser that is written based on a preset computer language, parsing the HTML structure of an English webpage to obtain DOM structure data, parsing the DOM structure data, and extracting multiple paragraphs of text content marked by paragraph numbers.

[0047] Step S102: cleaning multiple texts, removing single words and texts shorter than a set length, and merging them to obtain the original English texts.

[0048] Step S103: Scheduling the English website newsletter generation model to generate Chinese newsletters through a preset interface; wherein the English website newsletter generation model is obtained by fine-tuning the general large language model through LORA based on a preset question-and-answer data set, and each sample in the question-and-answer data set contains an original article content of an English web page and its corresponding Chinese newsletter.

[0049] In step S101, the HTML of a web page is the source code of the web page, which is the static text that constitutes the content of the web page. It contains tags, attributes and content, and is written in a tree-like hierarchical structure, but it is essentially a plain text file. DOM (Document Object Model) is the memory representation of HTML, and is a structured model generated in the browser at runtime. When a browser loads an HTML document, the browser downloads the HTML file and reads it line by line, parses the HTML tags, checks whether they comply with the syntax specifications, converts each HTML tag node into a DOM node, and constructs the nested tags into a DOM tree hierarchically.

[0050] The plug-in can be in the form of a browser extension, written in JavaScript, embedded in a web page to directly operate and parse the HTML structure to obtain DOM structure data, and run directly in the user's browser environment, which is easy to integrate and deploy. You can also use an automated script based on Puppeteer or Selenium, which is highly flexible and suitable for batch processing of web pages. Puppeteer scripts can be written based on JavaScript / Node.js, and Selenium scripts can be based on Python, Java, C# and other languages. It can also be configured as a server segment plug-in, using Python (such as BeautifulSoup, lxml) and Node.js (such as cheerio). HTML data can be captured through HTTP requests or processed through uploaded web page content. Server-side tools parse HTML and DOM structures and extract paragraph content.

[0051] In step S102, the cleaning of text content includes format unification, structural processing and noise removal. Removing single words or texts shorter than a set length is mainly to remove invalid information and clean up useless information in the English original text, such as advertising paragraphs, copyright statements, etc.

[0052] In step S103, the plug-in setting interface uses HTTP request scheduling to access the English website newsletter generation model. The English website newsletter generation model is obtained by fine-tuning the general large language model with LORA based on the question-answering dataset constructed for the application scenario. In some embodiments, the general large model uses the Qwen1.5-14B model, the MOSS model, the Bloom model, the Llama model, the GLM model or the Baichuan-13B model.

[0053] MOSS is a multilingual open source large language model developed by a Chinese team that supports Chinese tasks and is suitable for handling localization needs.

[0054] Bloom is an open source large-scale multilingual model that supports Chinese tasks and performs well in non-English fields.

[0055] Baichuan-13B is an open source language model focused on Chinese, which can be used to generate short messages through fine-tuning.

[0056] You can also use ChatGLM-6B / 10B, which is an open source large language model optimized for Chinese scenarios and supports multi-round conversations and text generation.

[0057] Various versions of Qwen models that can also be used: various versions of Qwen-7B, various versions of Qwen-14B, various versions of Qwen-32B, various versions of Qwen-72B models; or open source Llama models: various versions of llama-8b, various versions of llama-13b, various versions of llama-70b models; and generative interactive models such as the GLM model can all be used as substitutes for the base model in this technology.

[0058] In some embodiments, Figure 2 As shown, the steps of forming the question-answering dataset include S201 to S204:

[0059] Step S201: Collect English original texts from multiple English web pages to construct an English original text set T = {t1, ..., t j ,…,t m}, where t j is the jth English original text, and m is the number of English original texts; the English original texts include an equal number of long text categories and short text categories, and both the long text category and the short text category contain an equal number of English original texts in multiple fields; the fields include: news articles, academic articles, and social media articles.

[0060] Including S202: pre-processing the original English text to remove special symbols and invalid characters, and performing equivalent expression replacement and noise injection to achieve data enhancement.

[0061] S203 includes: obtaining the Chinese short message corresponding to each English original text, and constructing a Chinese short message set C = {c1, ..., c j ,…,c m}, c j is the jth Chinese text message, and m is the number of Chinese text messages; wherein the length of the Chinese text message is controlled to be less than or equal to the set word length.

[0062] The process includes S204: comparing and merging the pre-processed English original text set and the Chinese newsletter set to form a question-answering data set.

[0063] In steps S201 to S204, original English texts are collected from different fields such as news, academic articles, technical blogs, and social media to enhance the generalization ability of the model. Data in professional fields (such as medicine and law) can be annotated so that the model can be weighted for specific fields when fine-tuning. At the same time, ensure that the number of samples in different categories (such as long texts, short texts, and different fields) is roughly balanced to avoid the model being biased towards certain categories. It is also possible to increase the samples of categories where the model generates poor performance (such as long text and short message generation).

[0064] For Chinese text messages generated based on English originals, the length of the text messages should be controlled according to the length and complexity of the original text content to avoid text messages that are too long or too short. Information condensation should be performed to ensure that the text messages only contain the core content of the original text without redundant descriptions. Furthermore, multiple versions of text messages can be generated through synonym replacement and phrase transformation, and minor grammatical errors or spelling variations can be added manually to enhance the robustness of the model. Unlabeled English original texts can be input into the preliminarily trained model, and manual correction can be performed after the text messages are generated to expand the data scale.

[0065] At the same time, the present invention can also dynamically expand the size of the data set, especially in areas where the model training is insufficient in the early stages. Use a crawler program to capture the latest English web page content and write its corresponding Chinese newsletter.

[0066] In some embodiments, the method further includes: in the process of fine-tuning the general large language model with LORA based on a preset question-answering dataset, each round of iteration freezes the parameters of the attention layer in the general large language model at a ratio of 90% to 99%, and updates the parameters of the remaining part. This high-proportion freezing strategy mainly benefits from the low-rank update method of LORA, which enables the model to efficiently adapt to new tasks without completely updating all parameters. This form focuses on the Query and Value matrices in the attention mechanism, and usually only unfreezes the parameters of these matrices or adds the LORA layer. In addition to the attention layer, some feedforward network parameters may also be unfrozen to adapt to the complexity of the task. On the basis of mainly adjusting the attention mechanism, some embedding layer or output layer parameters are unfrozen to cope with the adaptation of multilingual input or specific domain knowledge.

[0067] In some embodiments, the method further includes, during the LORA fine-tuning process of the general large language model based on a preset question-answering dataset, constructing a cross entropy loss to update the parameters according to the predicted text of the general large language model and the corresponding text in the Chinese short message set, wherein the expression of the loss function is:

[0068]

[0069] Among them, T is the target length of the predicted text, x represents the original English text, and y t Represents the t-th subword unit of the predicted text, y 1：t-1 represents all subword units generated before the tth word embedding of the predicted text; P(y t |y 1：t-1 , x) represents the general large language model predicting the current subword unit y given the English original text x and the first t-1 subword units that have been generated t Probability.

[0070] In some embodiments, the method further includes: in the process of fine-tuning the general large language model based on the preset question-answering dataset, scoring the predicted text based on the pre-trained evaluation model, introducing the score into the loss function to construct a joint loss for parameter update, and the expression is,

[0071]

[0072] S′(y)=sigmoid(S(y));

[0073] Among them, T is the target length of the predicted text, x represents the original English text, and y t Represents the t-th subword unit of the predicted text, y 1：t-1 represents all subword units generated before the tth word embedding of the predicted text; P(y t |y 1：t-1 , x) represents the general large language model predicting the current subword unit y given the English original text x and the first t-1 subword units that have been generated t Probability; S(y) represents the score of the predicted text.

[0074] In some embodiments, a Chinese newsletter generation device for an English website includes a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0075] In some embodiments, a computer-readable storage medium stores a computer program / instruction thereon, which implements the steps of the above method when executed by a processor.

[0076] In some embodiments, a computer program product includes a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0077] The present invention is described below in conjunction with a specific embodiment:

[0078] This embodiment proposes a training method for a Chinese newsletter generation model for an English website based on a large language model, such as Figure 3 As shown, the method includes:

[0079] Step 1) construct a text message generation question and answer dataset;

[0080] The steps to construct a text message generation question answering dataset are as follows Figure 4 As shown, specifically including:

[0081] The goal of this step is to create a question-answering dataset containing English web page content and its corresponding Chinese news for subsequent model training.

[0082] Step 1-1) collecting original text content of a number of English web pages to form an English original text collection;

[0083] Collect English content from various public English news websites, blogs, academic article libraries, etc., ensure that each article is saved in text format, ensure that the content is complete and the format is unified, and facilitate subsequent processing.

[0084] Step 1-2) Based on the existing English original text collection, write a Chinese newsletter according to each English original text content in the English original text collection;

[0085] A corresponding Chinese newsletter is manually written for each English original article. The Chinese newsletter should be concise and cover the core information of the article, so as to facilitate the training model to understand and generate high-quality newsletters.

[0086] Step 1-3) Obtain an English-Chinese message generation question-answering dataset from the corresponding training samples in the English original text set and the Chinese message set.

[0087] Ensure that each English original text corresponds to a Chinese newsletter to form a one-to-one question-answer pair, and organize these question-answer pairs into a question-answer dataset in a unified format to facilitate subsequent model training.

[0088] Step 2) Based on the question-answering dataset, Qwen1.5-14B is fine-tuned with LORA to obtain the large language model Qwen1.5-OSINT with enhanced SMS generation capability;

[0089] In this step, we fine-tune the existing Qwen1.5-14B model to enhance its ability to generate Chinese text messages based on English original texts.

[0090] An example of a fine-tuned model is Figure 5 , specifically including:

[0091] Step 2-1) Select the Qwen1.5-14B model as the base model.

[0092] Step 2-2) Use the short message to generate a question-answering dataset as a training set, and use the LORA method to fine-tune Qwen1.5-14B to obtain the LORA weights.

[0093] The question-answering dataset collected in step 1 is formatted into training data suitable for fine-tuning the model, and the question-answering dataset is used for LORA fine-tuning training to adjust a small number of parameters of the model to enhance its SMS generation capability. Finally, the LORA weights are obtained, which represent the changes required for fine-tuning the model.

[0094] Step 2-3) Merge the LORA weights into the base model Qwen1.5-14B to obtain the large language model Qwen1.5-OSINT with enhanced SMS generation capabilities.

[0095] The trained LORA weights are merged into the original Qwen1.5-14B model to form a new model, Qwen1.5-OSINT, which has the specialized capability of SMS generation.

[0096] Figure 5 In the figure, OSINTD on the left represents the input data, which is the question-answering dataset established in this embodiment. Through the fine-tuning process Fine-tuning, the pre-trained general model (Qwen) is adjusted to a model that is more suitable for specific tasks. The pre-trained weights are loaded and the LoRA is updated using the OSINTD dataset. The Qwen model represents a large language model based on Transformer. A and B in the figure represent low-rank matrices (weight adjustment components in LoRA), respectively. The key to LoRA is to use low-rank matrices A and B for efficient parameter adjustment. The low-rank matrix is ​​slightly adjusted through the output of the pre-trained weights to achieve the purpose of fine-tuning the model with limited resources. Pretrained weights Pretrained Weights represent the original weights obtained after the model is trained on a large-scale dataset. These weights contain the general capabilities of the language model and are the basis for fine-tuning. The fine-tuned model is named Qwen1.5-OSINT.

[0097] Step 3) Based on the Qwen1.5-OSINT model, develop a text message generation browser plug-in to generate Chinese text messages based on the Chinese and English web page content of the browser.

[0098] The steps to generate Chinese text messages include:

[0099] Step 3-1) According to the opened English website, click the plug-in button to obtain the original English content of the website;

[0100] The steps for obtaining the original English content of a web page specifically include:

[0101] Step 3-1-1) Click the plug-in, and the background will automatically parse the HTML structure of the web page and obtain the content within all p tags in the web page.

[0102] Use JavaScript or other front-end technologies to parse the DOM structure of the current web page and extract all The text content inside the tag.

[0103] Step 3-1-2) Filter the content in the p tag, and delete the p tag content that is a single word or has too few words.

[0104] Filter out single words or texts with fewer words, such as advertisements and irrelevant information, to ensure the validity and quality of the content.

[0105] Step 3-1-3) All the filtered p tag contents are concatenated to obtain the original English content of the web page.

[0106] The filtered paragraph contents are stitched together in sequence to form the complete original text content, ready to be input into the large language model for processing.

[0107] Step 3-2) Call the Qwen1.5-OSINT model to generate a Chinese message based on the acquired English original content.

[0108] Implement the communication interface between the plug-in and the Qwen1.5-OSINT model, transmit the original content and receive the generated Chinese text message, use HTTP requests or other communication methods to call the model interface, and obtain the Chinese text message output by the model.

[0109] Step 3-3) Display the Chinese text message in the plug-in opened in the browser.

[0110] This embodiment provides a training method for a Chinese newsletter generation model for an English website based on a large language model. In view of the problem that the current large language model has weak newsletter generation capabilities in the field of open source intelligence, a newsletter generation question and answer dataset is constructed. Model fine-tuning training is performed on this dataset to obtain a large language model with enhanced newsletter generation capabilities, thereby effectively enhancing the newsletter generation capabilities of the large language model.

[0111] The method of this embodiment solves the problem that large language models have weak SMS generation capabilities in the field of open source intelligence. Experiments have shown that the Qwen1.5-OSINT model obtained by fine-tuning Qwen1.5-14B on the SMS generation task can significantly improve the SMS generation capability. The results show that the ROUGE-1 index of our model is improved by 0.3842, the ROUGE-2 index is improved by 0.2021, the ROUGE-L index is improved by 0.2494, and the BLEU index is improved by 0.1348.

[0112] Corresponding to the above method, the present invention also provides an apparatus / system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system implements the steps of the method described above.

[0113] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0114] In summary, the method, device and computer program product for generating Chinese newsletters for English websites described in the present invention extract text content from English web pages by embedding a plug-in in a browser, and generate Chinese newsletters by scheduling a pre-trained English website newsletter generation model through an interface; the English website newsletter generation model is fine-tuned by LORA based on a question-and-answer data set containing English web page content and its corresponding Chinese newsletter, and can accurately output Chinese newsletters for website content in an English context, thereby improving accuracy and professionalism.

[0115] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0116] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0117] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0118] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating Chinese newsletters for an English website, characterized in that: The method comprises the following steps: Calling a plug-in written in a preset computer language in the browser, parsing the HTML structure of the English webpage to obtain DOM structure data, parsing the DOM structure data, and extracting multiple paragraphs of text content marked by paragraph numbers; Cleaning the text contents of the plurality of paragraphs, removing single words and texts shorter than a set length, and merging them to obtain the original English text contents; The English website newsletter generation model is scheduled to generate Chinese newsletters through a preset interface; wherein the English website newsletter generation model is obtained by fine-tuning the general large language model through LORA based on a preset question-and-answer data set, and each sample in the question-and-answer data set contains the original article content of an English web page and its corresponding Chinese newsletter.

2. The method for generating Chinese short messages for an English website according to claim 1, characterized in that: The preset computer language adopts JavaScript, and the plug-in setting interface adopts HTTP request scheduling to access the English website newsletter generation model.

3. The method for generating Chinese short messages for an English website according to claim 1, characterized in that: The general large model adopts Qwen1.5-14B model, MOSS model, Bloom model, Llama model, GLM model or Baichuan-13B model.

4. The method for generating Chinese short messages for an English website according to claim 3, characterized in that: The steps of forming the question-answering dataset include: Collect English original texts from multiple English web pages and construct an English original text set T = {t1,…,t j ,…,t m }, where t j is the jth English original text, and m is the number of the English original texts; the English original texts include an equal number of long text categories and short text categories, and both the long text category and the short text category contain an equal number of English original texts in multiple fields; the fields include: news articles, academic articles, and social media articles; Preprocess the original English text to remove special symbols and invalid characters, replace the same expression and inject noise to achieve data enhancement; Get the Chinese short message corresponding to each English original text, and construct a Chinese short message set C = {c1,…,c j ,…,c m }, c j is the jth Chinese message, and m is the number of Chinese messages; wherein the length of the Chinese message is controlled to be less than or equal to the set word length; The pre-processed English original text set and the Chinese newsletter set are compared and merged to form the question and answer data set.

5. The method for generating Chinese short messages for an English website according to claim 4, characterized in that: The method also includes: in the process of performing LORA fine-tuning on the general large language model based on a preset question-answering dataset, freezing the parameters of the attention layer in the general large language model at a ratio of 90% to 99% in each round of iteration, and updating the parameters of the remaining part.

6. The method for generating Chinese short messages for an English website according to claim 5, characterized in that: The method also includes, in the process of fine-tuning the general large language model based on the preset question-answering dataset, constructing a cross entropy loss to update the parameters according to the predicted text of the general large language model and the corresponding text in the Chinese short message set, and the expression of the loss function is: Where T is the target length of the predicted text, x represents the original English text, and y t represents the tth subword unit of the predicted text, y 1：t-1 represents all subword units generated before the tth word embedding of the predicted text; P(y t |y 1：t-1 , x) represents the general large language model predicting the current sub-word unit y given the English original text x and the first t-1 sub-word units that have been generated t Probability.

7. The method for generating Chinese short messages for an English website according to claim 6, characterized in that: The method further comprises: In the process of fine-tuning the general large language model based on the preset question-answering dataset, the predicted text is scored based on the pre-trained evaluation model, and the score is introduced into the loss function to construct a joint loss for parameter update. The expression is: S′(y)=sigmoid(S(y)); Where T is the target length of the predicted text, x represents the original English text, and y t represents the tth subword unit of the predicted text, y 1：t-1 represents all subword units generated before the tth word embedding of the predicted text; P(y t |y 1：t-1 , x) represents the general large language model predicting the current sub-word unit y given the English original text x and the first t-1 sub-word units that have been generated t Probability; S(y) represents the score of the predicted text.

8. A device for generating Chinese short messages for an English website, comprising a processor, a memory, and a computer program / instruction stored in the memory, characterized in that: The processor is used to execute the computer program / instructions. When the computer program / instructions are executed, the device implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images