Data processing method, model training method and related device

Abstract

The invention discloses a data processing method, a model training method and a related device, and the data processing method can comprise the steps: obtaining a text data set which comprises a plurality of pieces of text data, and the plurality of pieces of text data comprise a plurality of keywords; obtaining content description information corresponding to the collocation word of each keyword; based on each keyword, the collocation word of each keyword and the content description information corresponding to the collocation word of each keyword, first training data is generated, and the first training data is used for training a text generation model. It can be seen that the finally obtained first training data comprises the content description information corresponding to the collocation word of each keyword, and the information is not pre-labeled data, so that the process of training the text generation model by using the first training data essentially belongs to weak supervised learning. And the text generation model is trained by using the data subsequently, so that the generalization ability of the text generation model can be improved, and the text generation quality can be improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a data processing method, a model training method, and related devices. Background Art

[0002] In daily office work, whether it is a functional department or an enterprise or institution, various documents are needed to issue or handle various office affairs. The quality of the documents may affect the image of the unit, office efficiency, and the implementation of decisions. Therefore, the quality of the documents is very important to each unit.

[0003] In practical applications, text generation models can be used to generate the required text to improve writing efficiency. However, due to the mediocre performance of current text generation models, the quality of the text generated using these models is poor and cannot meet practical needs. Summary of the Invention

[0004] The embodiments of the present application provide a data processing method, a model training method, and related devices to improve the performance of a text generation model, thereby improving the quality of the text generated by the model.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, comprising:

[0006] Acquire a text data set, wherein the text data set includes a plurality of text data, and the plurality of text data includes a plurality of keywords;

[0007] Obtain the content description information corresponding to each keyword's collocation words;

[0008] Based on each keyword, a collocation word of each keyword, and content description information corresponding to the collocation word of each keyword, first training data is generated, and the first training data is used to train a text generation model.

[0009] Optionally, obtaining content description information corresponding to the collocation words of each keyword includes:

[0010] Based on each keyword, generating a collocation word for each keyword;

[0011] Based on the collocation words of each keyword, the content description information corresponding to the collocation words of each keyword is obtained by querying.

[0012] Optionally, generating first training data based on each keyword, a collocation word of each keyword, and content description information corresponding to the collocation word of each keyword includes:

[0013] Based on each keyword and its collocation words, construct a first text generation instruction as training input data in the first training data;

[0014] Based on the content description information corresponding to the collocation words of each keyword, response information of the first text generation instruction is constructed as training output data in the first training data.

[0015] Optionally, the text data set further includes labels for the text data, and the labels for the text data include at least one of the following: a title label, an occasion label, a content label, an outline label, a content reference label, and a viewpoint supporting label, wherein the title label is used to describe the title of the text data, the occasion label is used to describe the occasion to which the text data is applicable, the content label is used to describe the summary content of the text data, the outline label is used to describe the discourse framework of the text data, the content reference label is used to describe the referenced content in the text data, and the viewpoint supporting label is used to describe the viewpoint in the text data and the evidence corresponding to the viewpoint;

[0016] The method further comprises:

[0017] Based on the text data and the label of the text data, second training data is generated, where the second training data is used to train the text generation model.

[0018] Optionally, generating second training data based on the text data and the label of the text data includes:

[0019] Based on the label of the text data, construct a second text generation instruction as training input data in the second training data;

[0020] Based on the text data, response information of the second text generation instruction is constructed as training output data in the second training data.

[0021] Optionally, the text data set further includes labels for the text data, wherein the labels for the text data include opinion supporting labels, and the opinion supporting labels are used to describe opinions in the text data and evidence corresponding to the opinions;

[0022] The method further comprises:

[0023] Based on the opinion supporting label, the evidence corresponding to the opinion is adjusted to obtain new evidence;

[0024] Based on the opinion supporting label and the new evidence, third training data is generated, and the third training data is used to train the text generation model.

[0025] Optionally, generating third training data based on the opinion supporting label and the new evidence, wherein the third training data is used to train the text generation model, includes:

[0026] Based on the viewpoint in the viewpoint supporting tag, construct a third text generation instruction as training input data in the third training data;

[0027] Based on the new evidence, response information of the third text generation instruction is constructed as training output data in the third training data.

[0028] Optionally, obtaining a text dataset includes:

[0029] Collect some text data;

[0030] Preprocessing is performed on a number of collected text data, and a text data set is formed from the preprocessed text data, wherein the preprocessing includes one or more of the following processes: filtering the text data based on the target language, filtering the text data based on the distribution information of punctuation marks and / or sentence length information in the text data, filtering the text data based on target words, filtering the text data based on the perplexity of the text data, deduplication processing on the text data, and filtering sensitive information in the text data.

[0031] In a second aspect, an embodiment of the present application provides a model training method, comprising:

[0032] Acquire training data, where the training data is obtained using the data processing method described in any one of the first aspects above;

[0033] The initial text generation model is trained using the training data.

[0034] In a third aspect, an embodiment of the present application provides a data processing device, including:

[0035] A text data acquisition module, configured to acquire a text data set, wherein the text data set includes a plurality of text data, and the plurality of text data includes a plurality of keywords;

[0036] A description information acquisition module is used to obtain the content description information corresponding to each keyword's collocation words;

[0037] The first training data generating module is configured to generate first training data based on each keyword, a collocation word of each keyword, and content description information corresponding to the collocation word of each keyword, wherein the first training data is used to train a text generation model.

[0038] In a fourth aspect, an embodiment of the present application provides a model training device, characterized in that:

[0039] A training data acquisition module, configured to acquire training data, wherein the training data is obtained using the data processing method described in any one of the first aspects above;

[0040] The model training module is used to train the initial text generation model using the training data.

[0041] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0042] The memory is used to store computer programs;

[0043] The processor is used to execute the computer program so that the electronic device can implement the data processing method as described in any one of the first aspects above, and / or implement the model training method as described in the second aspect above.

[0044] In the sixth aspect, an embodiment of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the data processing method described in any one of the first aspects above, and / or implement the model training method described in the second aspect above.

[0045] In the seventh aspect, an embodiment of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are run on an electronic device, the electronic device implements the data processing method as described in any one of the first aspects above, and / or implements the model training method as described in the second aspect above.

[0046] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0047] In an embodiment of the present application, first, a text data set can be obtained, which includes a number of text data, and the number of text data includes a plurality of keywords for generating text. Then, the content description information corresponding to the collocation words of each keyword can be obtained. Then, based on each keyword, the collocation words of each keyword, and the content description information corresponding to the collocation words of each keyword, a first training data can be generated, and the first training data can be used to train a text generation model. It can be seen that the training data of the text generation model finally obtained (i.e., the first training data) includes the content description information corresponding to the collocation words of each keyword. Since the content description information is not pre-labeled data, the process of training the text generation model using the first training data is essentially a process of weakly supervised learning. In this way, the subsequent use of such weakly supervised data to train the text generation model can improve the generalization ability of the text generation model, that is, improve the model performance, thereby helping to improve the quality of text generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0049] Figure 2 A flowchart of another data processing method provided in an embodiment of the present application;

[0050] Figure 3 A flowchart of a model training method provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of the structure of a model training device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] As mentioned above, due to the mediocre performance of current text generation models, the quality of text generated using text generation models is poor and difficult to meet actual usage needs.

[0054] To address the above-mentioned issues, an embodiment of the present application provides a data processing method, comprising: first, obtaining a text dataset comprising a plurality of text data, wherein the plurality of text data includes a plurality of keywords used to generate text. Next, obtaining content description information corresponding to the collocation words of each keyword. Then, based on each keyword, each keyword's collocation words, and each keyword's collocation words, first training data can be generated, and the first training data can be used to train a text generation model.

[0055] As can be seen, the resulting training data for the text generation model (i.e., the first training data) includes the content description information corresponding to each keyword's collocation. Since this content description information is not pre-labeled data, the process of training the text generation model using this first training data is essentially a weakly supervised learning process. As a result, subsequent training of the text generation model using this weakly supervised data can significantly improve the generalization ability of the text generation model, that is, improve the model performance, thereby helping to improve the quality of text generation.

[0056] It should be noted that the embodiments of the present application do not limit the execution entity of the data processing method. For example, the data processing method of the embodiments of the present application can be applied to data processing devices such as terminal devices or servers. The terminal device can be a smartphone, computer, smart dictionary, voice recorder, vehicle-mounted device, tablet computer, smart home device, etc. The server can be a standalone server, a cluster server, or a cloud server.

[0057] In order to make the purpose, technical solutions and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0058] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application. Figure 1 As shown, the data processing method provided in the embodiment of the present application may include the following steps S101 to S103.

[0059] S101: Acquire a text data set, where the text data set includes a number of text data, and the number of text data includes a plurality of keywords.

[0060] In an embodiment of the present application, text data may be data generated in various scenarios such as official document data, news report data, social comment data, or contract documents.

[0061] Keywords refer to the keywords corresponding to the text to be generated. For example, keywords can be hot words in various fields, such as Internet hot words or hot words in work reports.

[0062] Furthermore, embodiments of the present application may not limit the method for obtaining the above-mentioned text dataset. For example, the data processing device (i.e., the execution subject of the embodiments of the present application) may pre-store the text dataset locally and obtain the above-mentioned text dataset by locally reading it. In another example, the text dataset may be stored on another electronic device with data storage capabilities, and the data processing device may obtain the above-mentioned text dataset by accessing the electronic device.

[0063] S102: Obtain content description information corresponding to the collocation words of each keyword.

[0064] In the embodiments of the present application, for each keyword, the collocation words of the keyword refer to words that can be used in combination with the keyword. For example, taking the keyword "enterprise quality management system" as an example, its collocation words may include "meaning", "construction elements", "advantages" or "impact on enterprise development" and the like. After the keyword and the collocation words are combined, combined words can be obtained, such as "the meaning of the enterprise quality management system", "the construction elements of the enterprise quality management system", "the advantages of the enterprise quality management system" or "the impact of the enterprise quality management system on enterprise development" and the like. Accordingly, the content description information corresponding to the collocation words of the keyword is the specific content of the above-mentioned combined words.

[0065] Based on this, for ease of understanding, the embodiment of the present application may provide a possible implementation method to exemplify step S102.

[0066] As a possible implementation, with respect to step S102 , ie, the process of obtaining the above-mentioned content description information, first, collocation words for each keyword may be generated based on each keyword.

[0067] Here, the embodiments of the present application do not limit the method for generating collocation words for each keyword, and can be implemented using any existing or future implementation methods. For example, for each keyword, semantic recognition can be performed on the keyword first, and then expansion and association can be performed based on the semantics of the keyword to generate collocation words for the keyword from different perspectives or different contexts, thereby providing richer and more comprehensive training data for the subsequent text generation model.

[0068] Next, based on the collocation words of each keyword, the content description information corresponding to the collocation words of each keyword can be searched and obtained.

[0069] Here, the embodiments of the present application do not limit the query method for the content description information corresponding to the collocation words of each keyword, and can be implemented using any existing or future implementation methods. For example, for the collocation words of each keyword, if the data processing device is configured with a corresponding database, the corresponding content description information can be obtained by querying the database. Alternatively, if the data processing device is configured with an application programming interface (API) with an online query function, the corresponding content description information can be directly queried from the Internet by calling the API.

[0070] S103: Generate first training data based on each keyword, a collocation word of each keyword, and content description information corresponding to the collocation word of each keyword, where the first training data is used to train a text generation model.

[0071] In the embodiment of the present application, the text generation model refers to a model used to generate text, such as a large language model (LLM).

[0072] Based on this, the process of generating the first training data, i.e., step S103, can specifically be as follows: first, based on each keyword and each keyword's collocation words, a first text generation instruction is constructed as training input data in the first training data; then, based on the content description information corresponding to each keyword's collocation words, a response information of the first text generation instruction is constructed as training output data in the first training data. In this way, each keyword, each keyword's collocation words, and corresponding content description information can be used to construct training data pairs (i.e., the first training data) to train the text generation model, thereby improving the performance of the text generation model.

[0073] For example, using the LLM as a text generation model, the first text generation instruction is an instruction directed to the LLM input, and its purpose is to make the LLM generate the corresponding text based on the instruction. Accordingly, the first text generation instruction may be, for example, "As a text writing expert, you are extremely sensitive to hot words. Please help me explain the meaning of 'enterprise quality management system'."

[0074] Correspondingly, the response information of the first text instruction can be reflected as text data matching the first text instruction information, such as "Enterprise quality management system refers to the management system that directs and controls the organization in terms of quality. It is a systematic quality management model established within the organization and necessary to achieve quality goals. It is a strategic decision of the organization. The quality management system combines resources and processes, and conducts system management with process management methods. It covers the requirements of planning, implementation, correction and improvement activities for the entire process from determining customer needs, design and development, production, inspection, sales, and delivery. It generally becomes a requirement for internal quality management work of the organization in a documented manner."

[0075] Based on the relevant contents of S101-S103 above, it can be known that in an embodiment of the present application, first, a text data set can be obtained, and the text data set includes a number of text data, and the number of text data includes a plurality of keywords. Then, the content description information corresponding to the collocation words of each keyword can be obtained. Then, based on each keyword, the collocation words of each keyword, and the content description information corresponding to the collocation words of each keyword, a first training data can be generated, and the first training data can be used to train the text generation model. It can be seen that the training data of the text generation model finally obtained (i.e., the first training data) includes the content description information corresponding to the collocation words of each keyword. Since the content description information is not pre-labeled data, the process of training the text generation model using the first training data is essentially a process of weakly supervised learning. In this way, the subsequent use of such weakly supervised data to train the text generation model can improve the generalization ability of the text generation model, that is, improve the model performance, thereby helping to improve the quality of text generation.

[0076] In practical applications, each parameter in a text generation model may require approximately 20 text units (tokens) for training. However, the training data used may be too old, may be missing important information, or may contain errors. As a result, the text generation model may learn incorrect information from the training data during training. Alternatively, the text generation model can introduce new training data during the fine-tuning phase, but this is more likely to generate incorrect information when learning new training data, resulting in mediocre training results.

[0077] Based on the above issues, in an embodiment of the present application, in addition to constructing training data for weakly supervised learning, the text generation model can also be trained using annotated data to reduce the possibility of the model learning erroneous information, further improving the performance of the text generation model, and thus helping to improve the quality of text generation. Based on this, an embodiment of the present application provides another implementation of the data processing method, which can specifically include steps S201-S204. Steps S201-S203 are described below in conjunction with the embodiments and accompanying drawings.

[0078] Figure 2 A flowchart of another data processing method provided in an embodiment of the present application. Figure 2 As shown, the data processing method provided in the embodiment of the present application may include the following steps S201 to S204.

[0079] S201: Acquire a text dataset, where the text dataset includes a number of text data and labels of the text data, wherein the number of text data includes multiple keywords, and the labels of the text data include at least one of the following: a title label, an occasion label, a content label, an outline label, a content reference label, and an opinion support label, wherein the title label is used to describe the title of the text data, the occasion label is used to describe the occasion to which the text data is applicable, the content label is used to describe the summary content of the text data, the outline label is used to describe the discourse framework of the text data, the content reference label is used to describe the reference content in the text data, and the opinion support label is used to describe the opinion in the text data and the evidence corresponding to the opinion.

[0080] In the embodiments of the present application, considering that the text data may be mixed with low-quality information or include repeated information, the collected text data can be pre-processed to obtain higher-quality text data to construct a text dataset. This can improve the quality of the training data, thereby helping to improve the performance of the text generation model.

[0081] Based on this, for step S201, first, some text data can be collected. Here, for text data, it can be collected in advance from the Internet by crawling data. Of course, for the process of crawling text data, since it involves data that may be related to the privacy of the data publisher, when the embodiment of the present application is applied to a specific product or technology, it is necessary to obtain the permission or consent of the data publisher, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, before crawling text data, it is necessary to first seek permission or consent from the data publisher through the Internet to obtain authorization to crawl text data.

[0082] Next, the collected text data can be preprocessed, and the preprocessed text data can be used to form a text data set. The preprocessing process can include one or more of the following processes: filtering the text data based on the target language, filtering the text data based on the distribution information of punctuation marks and / or sentence length information in the text data, filtering the text data based on target keywords, filtering the text data based on the evaluation indicators of the text generation model, deduplicating the text data, and filtering the sensitive information in the text data. In this way, the quality of the text data set can be improved based on the preprocessed text data, which helps to improve the accuracy of subsequent model training.

[0083] For ease of understanding, several preprocessing methods are first exemplified below.

[0084] Regarding the process of filtering text data based on the target language, the target language refers to the language of the text to be generated, such as English, Chinese or Arabic. Accordingly, in a specific implementation, text data in other languages that do not conform to the target language can be filtered out.

[0085] Regarding the process of filtering text data based on punctuation distribution information and / or sentence length information in the text data, for example, if the punctuation distribution information indicates that multiple identical punctuation marks appear consecutively in a sentence, it may indicate that the sentence has been misused or misused punctuation marks, and therefore, the sentence can be filtered. And / or, if the length information of a sentence indicates that the sentence is longer than a preset sentence length, it may indicate that the sentence contains no punctuation marks, and therefore, the sentence can be filtered to prevent the model from learning overly long sentences that are difficult to understand during training.

[0086] Regarding the process of filtering text data based on target words, in a specific implementation, a target word can be pre-set. For example, the target word can be a meaningless word such as "nafg." When the target word is recognized in the text data, it can indicate the presence of noise in the text. Therefore, the target word can be filtered out from the text data. The method of identifying the target word in the text data can be implemented using any existing or future text recognition algorithm, which is not limited here.

[0087] The process of filtering text data based on the perplexity of the text data, where perplexity refers to the degree of confusion caused by the text data. When the perplexity is higher, it means that the text is more difficult to understand, the possibility of the presence of meaningless characters such as garbled characters is higher, and the text quality is lower; when the perplexity is lower, it means that the text is less difficult to understand, the possibility of the presence of meaningless characters such as garbled characters is lower, and the text quality is higher. Therefore, in the specific implementation, a perplexity threshold can be set in advance, such as setting the perplexity threshold to 100. Then, the text data with a perplexity greater than the perplexity threshold can be filtered to improve the quality of the text data. Among them, the method of determining the perplexity of text data can be implemented by using any existing or future algorithm for calculating text perplexity, which is not limited here.

[0088] The process of deduplicating text data can be implemented with precise string deduplication. Specifically, a suffix array of the text data is first obtained. Next, the number of occurrences of each character substring in the text data is determined based on the suffix data, and character substrings exceeding a threshold number of occurrences are filtered, thereby achieving precise deduplication of the text data. The method for determining the suffix array of the text data and the number of occurrences of each character substring in the text data can be implemented using any existing or future algorithm for constructing a suffix array and searching for character substrings, and is not limited herein.

[0089] The process of filtering sensitive information in text data, where the sensitive information may include personal information and harmful information. Personal information may include name, phone number, or address, and harmful information may include information suspected of violating regulations.

[0090] After obtaining the text data, the text data can be further annotated to obtain a label for the text data. The annotation method may not be specifically limited in the embodiments of the present application. For example, the quality of data annotation can be improved by manual annotation. Alternatively, the above text data can be annotated by some automated tools, and then the labels of the text data can be manually checked and corrected to improve the annotation efficiency. It should be noted here that in the case of manual participation in annotation, the annotation standard can be pre-specified. For example, the standard can specify the label category or annotation specification of the text data, so as to improve the consistency and accuracy of the data annotation.

[0091] Regarding tag categories, we've mentioned several possible tag categories above. Among them, the title tag is used to describe the title of the text data. Here, the title can refer to the title of the text. If the text doesn't have a title, the title tag can also describe the file name of the text data or the first sentence of the text.

[0092] The occasion tag is used to describe the occasion that the text data is applicable to. For example, the occasion may include a work plan report, an external speech, or a research report report, etc., which are not limited here.

[0093] The content reference tag is used to describe the reference content in the text data. For example, the reference content may include famous quotes, current news, or hot policies, etc., which are not limited here.

[0094] The opinion support tag is used to describe the opinion in text data and the evidence corresponding to the opinion. This evidence can be real examples or materials used to support the opinion.

[0095] In practical applications, for external speeches, the text data labels may also include speaker identity tags, audience role tags, or speech background tags. The speaker identity tag describes the speaker's industry or position, the audience role tag describes the audience members in the text data, and the speech background tag describes the reason for the speech or the background of the speech.

[0096] In addition, it should be noted that, in the embodiment of the present application, for the relevant description content of the above keywords, reference can be made to the relevant description of step S101 in the above embodiment, which will not be repeated here.

[0097] S202: Obtain content description information corresponding to the collocation words of each keyword.

[0098] In the embodiment of the present application, for the relevant description of the above S202, reference can be made to the relevant description of step S102 in the above embodiment, which will not be repeated here.

[0099] S203: Generate first training data based on each keyword, the collocation words of each keyword, and the content description information corresponding to the collocation words of each keyword, where the first training data is used to train the text generation model.

[0100] In the embodiment of the present application, for the relevant description of the above S203, reference can be made to the relevant description of step S103 in the above embodiment, which will not be repeated here.

[0101] S204: Generate second training data based on the text data and labels of the text data in the text dataset, where the second training data is used to train the text generation model.

[0102] In the embodiment of the present application, the process of generating the second training data, i.e., step S103, may specifically include: first, constructing a second text generation instruction based on the label of the text data as the training input data in the second training data; then, constructing the response information of the second text generation instruction based on the text data as the training output data in the second training data. In this way, the text data and the corresponding label can be used to construct a training data pair (i.e., the second training data) to train the text generation model, thereby improving the performance of the text generation model.

[0103] For example, using the LLM as a text generation model, the second text generation instruction is an instruction directed to the LLM input, and its purpose is to make the LLM generate the corresponding text based on the instruction. Accordingly, the second text generation instruction may be, for example, "As a text writing expert, you are very familiar with corporate management strategies. Please provide me with a speech on <Enterprise Quality Management System>."

[0104] Correspondingly, the response information of the second text instruction can be reflected as text data matching the second text instruction information, such as "Dear leaders, distinguished guests, and guests, good afternoon! I am the representative of the enterprise quality system audit manager this year, and I am very honored to be able to stand here and communicate with you. The quality system is the foundation of an organization's operations and the key to the sustainable development of an enterprise. As quality system audit managers, our task is to ensure that the organization's quality system complies with relevant standards and requirements, thereby improving the quality level of products and services and providing customers with better satisfaction."

[0105] In addition, it should be noted that in the embodiment of the present application, the execution order of step S204 is not limited. For example, step S202 and step S203 can be executed first, and then step S204 is executed; step S202 can be executed first, then step S204, and finally step S203; step S204 can also be executed first, and then step S202 and step S203 are executed. Figure 2 The above description is only an exemplary description. In actual application, it is sufficient to ensure that step S204 is executed after step S201.

[0106] Furthermore, in embodiments of the present application, new training data can be generated based on the labels of the text data. Since this data is not pre-labeled, the text generation model can be further trained in a weakly supervised manner. This can reduce the labeling time and cost while increasing the amount of training data, thereby improving the generalization ability of the text generation model, that is, improving model performance and helping to improve the quality of text generation.

[0107] Based on this, the tags for the text data can include opinion-supporting tags, which can be used to annotate the opinion and the corresponding evidence in the text data. Therefore, in a specific implementation, first, based on the opinion-supporting tags, the evidence corresponding to the opinion can be adjusted to obtain new evidence. Adjusting the evidence can, for example, involve optimizing or associating the original evidence to obtain new evidence with similar or identical semantics and more logical reasoning.

[0108] Next, third training data can be generated based on the opinion supporting labels and new evidence, and the third training data is used to train the text generation model.

[0109] Specifically, we can first construct a third text generation instruction based on the viewpoint in the viewpoint support tag as the training input data in the third training data. Then, based on the new evidence, we construct the response information of the third text generation instruction as the training output data in the third training data. In this way, we can use the viewpoint in the viewpoint support tag and the new evidence to construct a training data pair (i.e., the third training data) to train the text generation model, thereby improving the performance of the text generation model.

[0110] For example, using the LLM as the text generation model, the third text generation instruction is an instruction directed to the LLM input, and its purpose is to make the LLM generate the corresponding text based on the instruction. Accordingly, the third text generation instruction may be, for example, "As an excellent text writing assistant, you are very familiar with corporate management regulations. Based on the viewpoint in <>, please provide real and reliable examples to support the above viewpoint. My view is <the company attaches great importance to quality management>."

[0111] Correspondingly, the response information of the third text instruction can be reflected as text data matching the third text instruction information, such as "1. The company will strengthen internal training and education to improve employees' quality awareness and capabilities. 3. The company will introduce advanced quality management tools and methods, and optimize quality processes and process control. 4. The company will establish multiple quality management indicator systems, and through quantitative analysis of data, achieve comprehensive monitoring and control of quality work."

[0112] Based on the relevant contents of S201-S204 above, it can be seen that in the embodiment of the present application, the training data of the text generation model finally obtained can include both pre-labeled data and unlabeled data used for weakly supervised learning. In this way, the subsequent use of such weakly supervised data and labeled data to jointly train the text generation model can further improve the generalization ability of the text generation model, that is, improve the model performance, thereby helping to improve the quality of text generation.

[0113] Based on the data processing method provided in the above embodiment, the present application embodiment further provides a model training method. The model training method is described below in conjunction with the embodiments and drawings.

[0114] Figure 3 A flow chart of a model training method provided in an embodiment of the present application. Figure 3 As shown, the model training method provided in the embodiment of the present application may include the following steps S301-S302.

[0115] S301: Acquire training data, which is obtained using the above-mentioned data processing method.

[0116] S302: Train the initial text generation model using training data.

[0117] In an embodiment of the present application, the initial text generation model may be a pre-trained large language model.

[0118] In addition, there is no specific limitation on the method of model training, and any existing or future model training method can be used for implementation. Based on the relevant contents of S301-S302 above, it can be seen that in the embodiment of the present application, since the first training data, second training data and / or third training data obtained by the above data processing method can be used for model training, the generalization ability of the trained text generation model can be improved, that is, the model performance is improved, thereby helping to improve the quality of text generation.

[0119] Based on the data processing method provided in the above embodiment, the present application also provides a data processing device. The data processing device is described below in conjunction with the embodiments and drawings.

[0120] Figure 4 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Figure 4 As shown, the data processing device 400 provided in this embodiment of the present application may include:

[0121] A text data acquisition module 401 is used to acquire a text data set, wherein the text data set includes a plurality of text data, and the plurality of text data includes a plurality of keywords;

[0122] Description information acquisition module 402, used to obtain content description information corresponding to the collocation words of each keyword;

[0123] The first training data generating module 403 is configured to generate first training data based on each keyword, its collocation words, and content description information corresponding to the collocation words of each keyword, where the first training data is used to train a text generation model.

[0124] As an implementation manner, the description information acquisition module 402 is specifically configured to:

[0125] Based on each keyword, generating a collocation word for each keyword;

[0126] Based on the collocation words of each keyword, the content description information corresponding to the collocation words of each keyword is obtained by querying.

[0127] As an implementation manner, the first training data generating module 403 is specifically configured to:

[0128] Based on each keyword and its collocation words, construct a first text generation instruction as training input data in the first training data;

[0129] Based on the content description information corresponding to the collocation words of each keyword, response information of the first text generation instruction is constructed as training output data in the first training data.

[0130] As an embodiment, the text data set further includes tags for the text data, and the tags for the text data include at least one of the following: a title tag, an occasion tag, a content tag, an outline tag, a content reference tag, and a viewpoint supporting tag, wherein the title tag is used to describe the title of the text data, the occasion tag is used to describe the occasion to which the text data is applicable, the content tag is used to describe the summary content of the text data, the outline tag is used to describe the discourse framework of the text data, the content reference tag is used to describe the referenced content in the text data, and the viewpoint supporting tag is used to describe the viewpoint in the text data and the evidence corresponding to the viewpoint;

[0131] The data processing device 400 further includes:

[0132] The second training data generating module is used to generate second training data based on the text data and the label of the text data, where the second training data is used to train the text generation model.

[0133] As an implementation manner, the second training data generating module is specifically configured to:

[0134] Based on the label of the text data, construct a second text generation instruction as training input data in the second training data;

[0135] Based on the text data, response information of the second text generation instruction is constructed as training output data in the second training data.

[0136] As an embodiment, the text data set further includes labels for the text data, wherein the labels for the text data include opinion supporting labels, and the opinion supporting labels are used to describe opinions in the text data and evidence corresponding to the opinions;

[0137] The data processing device 400 further includes:

[0138] An information adjustment module, configured to adjust the evidence corresponding to the viewpoint based on the viewpoint supporting label to obtain new evidence;

[0139] The third training data generation module is used to generate third training data based on the opinion supporting label and the new evidence, and the third training data is used to train the text generation model.

[0140] As an implementation manner, the third training data generating module is specifically configured to:

[0141] Based on the viewpoint in the viewpoint supporting tag, construct a third text generation instruction as training input data in the third training data;

[0142] Based on the new evidence, response information of the third text generation instruction is constructed as training output data in the third training data.

[0143] As an implementation manner, the text data acquisition module 401 is specifically configured to:

[0144] Collect some text data;

[0145] Preprocessing is performed on a number of collected text data, and a text data set is formed from the preprocessed text data, wherein the preprocessing includes one or more of the following processes: filtering the text data based on the target language, filtering the text data based on the distribution information of punctuation marks and / or sentence length information in the text data, filtering the text data based on target words, filtering the text data based on the perplexity of the text data, deduplicating the text data, and filtering sensitive information in the text data.

[0146] Based on the model training method provided in the above embodiment, the present application also provides a model training device. The model training device is described below in conjunction with the embodiments and drawings.

[0147] Figure 5 A schematic diagram of the structure of a model training device provided in an embodiment of the present application. Figure 5 As shown, the model training device 500 provided in the embodiment of the present application may include:

[0148] A training data acquisition module 501 is used to acquire training data, wherein the training data is obtained using the above-mentioned data processing device;

[0149] The model training module 502 is used to train the initial text generation model using the training data.

[0150] Furthermore, an embodiment of the present application also provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0151] The memory is used to store computer programs;

[0152] The processor is used to execute the computer program so that the electronic device can implement the above-mentioned data processing method and / or the above-mentioned model training method.

[0153] Furthermore, an embodiment of the present application also provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the data processing method as described above, and / or implement the model training method as described above.

[0154] Furthermore, an embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the data processing method as described above, and / or implements the model training method as described above.

[0155] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in each embodiment of the present application or certain parts of the embodiments.

[0156] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.

[0157] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0158] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that: include: Acquire a text data set, wherein the text data set includes a plurality of text data, and the plurality of text data includes a plurality of keywords; Obtain the content description information corresponding to each keyword's collocation words; Based on each keyword, a collocation word of each keyword, and content description information corresponding to the collocation word of each keyword, first training data is generated, and the first training data is used to train a text generation model.

2. The data processing method according to claim 1, wherein: The step of obtaining the content description information corresponding to the collocation words of each keyword includes: Based on each keyword, generating a collocation word for each keyword; Based on the collocation words of each keyword, the content description information corresponding to the collocation words of each keyword is obtained by querying.

3. The data processing method according to claim 1, wherein: The generating of first training data based on each keyword, a collocation word of each keyword, and content description information corresponding to the collocation word of each keyword includes: Based on each keyword and its collocation words, construct a first text generation instruction as training input data in the first training data; Based on the content description information corresponding to the collocation words of each keyword, response information of the first text generation instruction is constructed as training output data in the first training data.

4. The data processing method according to any one of claims 1 to 3, characterized in that: The text data set further includes tags for the text data, wherein the tags for the text data include at least one of the following: a title tag, an occasion tag, a content tag, an outline tag, a content reference tag, and a point of view supporting tag, wherein the title tag is used to describe the title of the text data, the occasion tag is used to describe the occasion to which the text data applies, the content tag is used to describe the summary content of the text data, the outline tag is used to describe the discourse framework of the text data, the content reference tag is used to describe the reference content in the text data, and the point of view supporting tag is used to describe the point of view in the text data and the evidence corresponding to the point of view; The data processing method further includes: Based on the text data and the label of the text data, second training data is generated, where the second training data is used to train the text generation model.

5. The data processing method according to claim 4, characterized in that: The generating second training data based on the text data and the label of the text data includes: Based on the label of the text data, construct a second text generation instruction as training input data in the second training data; Based on the text data, response information of the second text generation instruction is constructed as training output data in the second training data.

6. The data processing method according to any one of claims 1 to 3, characterized in that: The text data set further includes labels for the text data, wherein the labels for the text data include opinion supporting labels, and the opinion supporting labels are used to describe opinions in the text data and evidence corresponding to the opinions; The data processing method further includes: Based on the opinion supporting label, the evidence corresponding to the opinion is adjusted to obtain new evidence; Based on the opinion supporting label and the new evidence, third training data is generated, and the third training data is used to train the text generation model.

7. The data processing method according to claim 6, characterized in that: Generating third training data based on the opinion supporting label and the new evidence, wherein the third training data is used to train the text generation model, includes: Based on the viewpoint in the viewpoint supporting tag, construct a third text generation instruction as training input data in the third training data; Based on the new evidence, response information of the third text generation instruction is constructed as training output data in the third training data.

8. A model training method, characterized in that: include: Acquiring training data, wherein the training data is obtained using the data processing method according to any one of claims 1 to 7; The initial text generation model is trained using the training data.

9. A data processing device, characterized in that: include: A text data acquisition module, configured to acquire a text data set, wherein the text data set includes a plurality of text data, and the plurality of text data includes a plurality of keywords; A description information acquisition module is used to obtain the content description information corresponding to each keyword's collocation words; The first training data generating module is configured to generate first training data based on each keyword, a collocation word of each keyword, and content description information corresponding to the collocation word of each keyword, wherein the first training data is used to train a text generation model.

10. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the data processing method as described in any one of claims 1 to 7, and / or implement the model training method as described in claim 8.

11. A computer storage medium, characterized in that The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the data processing method as described in any one of claims 1 to 7, and / or implement the model training method as described in claim 8.

Images