Model training method and device, business execution method and device and storage medium
Generating summary text through large language models and combining bias training language models, the problem of low training efficiency in the existing technology is solved, and efficient language model training and accurate matching results are achieved.
Patent Information
- Application Number
- CN202411217241.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-27
AI Technical Summary
In the prior art, it takes a long time to train the language model by artificially setting up multiple problem statements, resulting in low training efficiency.
The text data is used to perform semantic analysis on the text data, generate summary text, and input the text data and summary text into the language model to be trained. The loss value is determined based on the deviations of the matching results and the actual matching results for training, avoiding artificial settings Time-consuming question statements.
It greatly improves the training efficiency of the language model, enables the model to learn multi-angle analysis and processing capabilities, and improves the accuracy of matching results.
Smart Images
Figure CN119203933A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a model training method, a business execution method, a device, and a storage medium. Background Art
[0002] In recent years, with the development of computer technology and artificial intelligence technology, the application of language models has become increasingly widespread. Language models can perform related question-and-answer services based on personal data provided by users. For example, users can input text (such as a piece of news) and question statements (such as: Does this text describe the recent changes in economic data in region A?) into the language model, and then the language model can analyze and process the received text and question statements and return the results to the user.
[0003] At present, in order to enable the language model to accurately analyze the content input by the user and obtain results in the actual application process, different question statements can be set in an artificial way in the early stage of training the language model to guide the language model to give corresponding answers, so that the language model can learn the ability to analyze and process different question statements. For example, in the process of training the language model, when the input of the language model is a news article, the question statement can be "Is this article a financial news?", so as to guide the language model to judge whether the article belongs to the financial news. Or, the question statement can also be "Does this article describe the recent changes in the stock market?", so as to guide the language model to judge whether there is content about the recent changes in the stock market in the article.
[0004] However, manually setting multiple question statements takes a long time, resulting in low training efficiency. Summary of the invention
[0005] This specification provides a model training method, a business execution method, a device and a storage medium to partially solve the above-mentioned problems existing in the prior art.
[0006] This manual adopts the following technical solutions:
[0007] This specification provides a model training method, including:
[0008] Get text data;
[0009] Input the text data into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generate a summary text for summarizing the text data from at least a partial perspective based on each text element information, so as to match the summary text with the text data to obtain a matching result between the summary text and the text data as an actual matching result, wherein the text element information is used to characterize the sentence components of the text data, the emotion type expressed by the text content corresponding to the text data, and the description style of the text content corresponding to the text data;
[0010] Inputting the text data and the summary text into a language model to be trained, so that the language model generates a matching result between the text data and the summary text as a matching result to be verified;
[0011] A loss value is determined according to a deviation between the to-be-verified matching result and the actual matching result, and the to-be-trained language model is trained according to the loss value, wherein there is a positive correlation between the deviation and the loss value.
[0012] Optionally, the text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generates a summary text for summarizing the text data from at least part of the perspective according to each text element information, specifically including:
[0013] The text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and selects at least one target element information from the each text element information, so as to combine the selected at least one target element information to obtain combined text element information, and generate a summary text for summarizing the text data from at least a partial perspective based on the combined text element information.
[0014] Optionally, the text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and selects at least one target element information from each text element information, specifically including:
[0015] The text data is input into a preset large language model so that the large language model determines text features corresponding to the text data, and based on the text features, semantic analysis is performed on the text data to determine each text element information corresponding to the text data, and based on the text features, important element information is determined from the each text element information as at least one selected target element information.
[0016] Optionally, the text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and selects at least one target element information from each text element information, specifically including:
[0017] The text data and the acquired task requirement information are input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and according to the task requirement information, determine the text element information that matches the task requirement information from the various text element information as at least one selected target element information.
[0018] Optionally, inputting the text data and the summary text into a language model to be trained so that the language model generates a matching result between the text data and the summary text as a matching result to be verified, specifically including:
[0019] The text data, the summary text and the output regulation information are input into the language model to be trained, so that the language model outputs the matching result between the text data and the summary text as the matching result to be verified according to the information output type required by the output regulation information.
[0020] This specification provides a service execution method, including:
[0021] Acquire data to be matched, wherein the data to be matched includes text data to be matched and summary text to be matched;
[0022] Inputting the data to be matched into a pre-trained language model so that the language model generates a matching result between the text data to be matched and the summary text to be matched, wherein the language model is trained by the above-mentioned model training method;
[0023] The business is executed according to the matching result.
[0024] This specification provides a model training device, including:
[0025] Acquisition module: used to obtain text data;
[0026] A first matching module: used for inputting the text data into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generates a summary text for summarizing the text data from at least a partial perspective based on each text element information, so as to match the summary text with the text data and obtain a matching result between the summary text and the text data as an actual matching result, wherein the text element information is used to characterize the sentence components of the text data, the emotion type expressed by the text content corresponding to the text data, and the description style of the text content corresponding to the text data;
[0027] A second matching module: used for inputting the text data and the summary text into a language model to be trained, so that the language model generates a matching result between the text data and the summary text as a matching result to be verified;
[0028] Training module: used to determine a loss value according to the deviation between the to-be-verified matching result and the actual matching result, and to train the to-be-trained language model according to the loss value, wherein the deviation is positively correlated with the loss value.
[0029] Optionally, the first matching module is specifically used to:
[0030] The text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and selects at least one target element information from the each text element information, so as to combine the selected at least one target element information to obtain combined text element information, and generate a summary text for summarizing the text data from at least a partial perspective based on the combined text element information.
[0031] Optionally, the first matching module is specifically used to:
[0032] The text data is input into a preset large language model so that the large language model determines text features corresponding to the text data, and based on the text features, semantic analysis is performed on the text data to determine each text element information corresponding to the text data, and based on the text features, important element information is determined from the each text element information as at least one selected target element information.
[0033] Optionally, the first matching module is specifically used to:
[0034] The text data and the acquired task requirement information are input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and according to the task requirement information, determine the text element information that matches the task requirement information from the various text element information as at least one selected target element information.
[0035] Optionally, the second matching module is specifically used to:
[0036] The text data, the summary text and the output regulation information are input into the language model to be trained, so that the language model outputs the matching result between the text data and the summary text as the matching result to be verified according to the information output type required by the output regulation information.
[0037] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned model training method or business execution method.
[0038] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned model training method or business execution method when executing the program.
[0039] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:
[0040] The model training method provided in the present specification can first obtain text data, input the text data into a preset large language model, so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generate a summary text for summarizing the text data from at least a partial perspective based on each text element information, so as to match the summary text with the text data to obtain a matching result between the summary text and the text data, and input the text data and the summary text into the language model to be trained as the actual matching result, so that the language model generates a matching result between the text data and the summary text as the matching result to be verified, determine a loss value based on the deviation between the matching result to be verified and the actual matching result, and train the language model to be trained based on the loss value.
[0041] It can be seen from the above method that text data can be input into a large language model so that the large language model determines the summary text according to the text element information corresponding to the text data. In addition, the text data and the summary text determined by the large language model can be input as sample data into the language model to be trained, so as to train the language model, thereby avoiding the problem of low training efficiency caused by the long time spent in training the language model by artificially setting multiple question statements in the prior art, and greatly improving the training efficiency. At the same time, the summary text used in the training process of the above method is determined by the preset large language model according to the text element information obtained by semantic analysis from multiple angles. Therefore, the training using the above method can enable the language model to learn the ability to analyze and process summary texts from multiple angles, thereby improving the accuracy of the matching results output by the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:
[0043] Figure 1 A flowchart of a model training method provided in this specification;
[0044] Figure 2 A flowchart of a business execution method provided in this specification;
[0045] Figure 3 A schematic diagram of a display page provided for this manual;
[0046] Figure 4 A schematic diagram of a display page provided for this manual;
[0047] Figure 5 A schematic diagram of a model training device provided in this specification;
[0048] Figure 6 A schematic diagram of a service execution device provided in this specification;
[0049] Figure 7 A method provided in this specification corresponding to Figure 1 or Figure 2 Schematic diagram of the structure of an electronic device. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.
[0051] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.
[0052] Figure 1 A flow chart of a model training method provided in this specification includes the following steps:
[0053] S101: Acquire text data.
[0054] S102: Input the text data into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generate a summary text for summarizing the text data from at least a partial perspective based on each text element information, so as to match the summary text with the text data to obtain a matching result between the summary text and the text data as an actual matching result, wherein the text element information is used to characterize the sentence components of the text data, the emotion type expressed by the text content corresponding to the text data, and the description style of the text content corresponding to the text data.
[0055] The execution subject of the model training method involved in this specification can be a terminal device such as a desktop computer, a laptop computer, or a client installed in the terminal device, or a server. The following only takes the server as the execution subject as an example to illustrate the model training method in the embodiment of this specification.
[0056] At present, in order to enable the language model to accurately analyze the content input by the user and obtain the results in the actual application process, different question statements can be set manually in the early stage of training the language model to guide the language model to give corresponding answers, so that the language model can learn the ability to analyze and process different question statements. For example, in the process of training the language model, when the input of the language model is a news article, the question statement can be "Is this article a financial news?", so as to guide the language model to judge whether the article belongs to the financial news. Alternatively, the question statement can also be "Does this article describe the recent changes in the stock market?", so as to guide the language model to judge whether there is content about the recent changes in the stock market in the article. However, the method of manually setting multiple question statements takes a long time, resulting in low training efficiency.
[0057] In order to solve the above problems, in this specification, it can be divided into two stages as a whole. The first stage is the stage of obtaining a training data set through a preset large language model, and the second stage is the stage of training the language model to be trained through the obtained training data set.
[0058] In the first stage, the server can first obtain text data (such as news articles, academic papers, user-generated content (such as comments posted by users), etc.). Then, the server can input the text data into a preset large language model (such as GPT-4), which can then enable the large language model to perform semantic analysis on the text data from multiple angles to determine the text element information corresponding to the text data.
[0059] For example, while inputting text data into the large language model, the sentence "Please perform semantic analysis on the input text." can be input into the large language model so that the large language model can determine the text element information corresponding to the text data. Among them, semantic analysis of text data from multiple angles can refer to semantic analysis from the sentence components of the text data, the emotion type expressed by the text content corresponding to the text data, and the description style of the text content corresponding to the text data. In other words, the text element information determined by the large language model through semantic analysis can be used to characterize the sentence components of the text data, the emotion type expressed by the text content corresponding to the text data, and the description style of the text content corresponding to the text data.
[0060] The sentence components of text data may refer to the subject object described in the text content corresponding to the text data, the keywords contained in the text content corresponding to the text data, etc. The emotion type expressed by the text content corresponding to the text data may be emotions such as anger, worry, etc. The description style of the text content corresponding to the text data may refer to the techniques used in the text content corresponding to the text data (such as: contrast, exaggeration, parallelism, etc.), the exposition style of the text content corresponding to the text data (such as: the informal style and argumentative style reflected in the comment text posted by the user, the formal style reflected in an academic paper, etc.).
[0061] Next, each text element information corresponding to the text data will be explained by way of examples.
[0062] For example, the text data is a user-generated content: "Employee A always makes a work plan in advance and completes the work efficiently, while employee B usually does not make a work plan, resulting in low work efficiency. It seems that making a work plan in advance can really achieve twice the result with half the effort, so employee B should quickly make a work plan in advance!". The text element information corresponding to the text data can be: the subject objects described in the text content corresponding to the text data are employee A and employee B; the keywords contained in the text content corresponding to the text data are work plan, work efficiency, and twice the result with half the effort; the emotions expressed in the text content corresponding to the text data are worry and motivation; the technique used in the text content corresponding to the text data is comparison; the exposition style of the text content corresponding to the text data is informal style and argumentative style.
[0063] Of course, this specification only lists one form of text element information. In actual application, the large language model can generate various text element information by performing semantic analysis from different angles, which will not be elaborated in this specification.
[0064] Then, the server may generate a summary text for summarizing the text data from at least some perspectives according to each text element information corresponding to the text data through the large language model.
[0065] Specifically, at least one target element information may be firstly selected from each text element information through a large language model.
[0066] It should be noted that there are multiple ways to select at least one target element information from each text element information. For example, the target element information can be randomly selected, and the sentence "please randomly select at least one target element information" can be input into the large language model, so that the large language model selects at least one target element information from each text element information by randomly selecting.
[0067] Alternatively, at least one target element information can be selected based on the text features corresponding to the text data, wherein first, the text features corresponding to the text data can be determined by a large language model, so as to perform semantic analysis on the text data according to the determined text features, and then the text element information corresponding to the text data can be determined. Then, important element information can be determined from each text element information according to the text features corresponding to the text data, and the determined important element information can be used as at least one selected target element information, wherein the important element information can refer to the text element information that plays a more critical or decisive role in the process of semantic analysis according to the text features. For example, the sentence "Please select important element information from each text element information" can be input into the large language model, so that the large language model selects important element information from each text element information according to the text features, and the important element information is: the subject object described in the text content corresponding to the text data is employee B; the emotion expressed by the text content corresponding to the text data is motivation.
[0068] Alternatively, at least one target element information can be selected according to the actual task requirements. Specifically, when the text data is input into the preset large language model, the acquired task requirement information can also be input into the large language model, and then the large language model can be made to perform semantic analysis on the text data to determine the text element information corresponding to the text data, and according to the task requirement information, determine the text element information matching the task requirement information from the text element information as the selected at least one target element information. Among them, the task requirement information can be derived from the sentence input into the large language model according to the actual task requirements (such as: the text style needs to be identified according to the subsequent trained language model), such as: "Please select the text element information about the text exposition style and the text subject object", and the large language model can determine the text element information for the text exposition style and the text subject object from the text element information according to the task requirement information (such as: the exposition style of the text content corresponding to the text data is informal style; the subject object described in the text content corresponding to the text data is employee B) as the selected target element information.
[0069] Then, the at least one selected target element information may be combined to obtain combined text element information, and a summary text for summarizing the text data from at least a partial perspective may be generated based on the combined text information.
[0070] For example, the selected target element information is: the method used in the text content corresponding to the text data is contrast; the exposition style of the text content corresponding to the text data is argumentative style; the subject object described in the text content corresponding to the text data is employee B. The target element information is combined, and the combined text information obtained is "the method used in the text content corresponding to the text data is contrast, the exposition style is argumentative style, and the subject object described is employee B." Then, the sentence "please generate a summary text for the target element information" can be input into the large language model, so that the large language model generates a summary text for summarizing the text data from at least some perspectives, and the summary text is: "The text discusses the work situation of employee B by using the method of contrast."
[0071] For another example, the selected target element information is: the subject object described in the text content corresponding to the text data is employee A; the emotion expressed in the text content corresponding to the text data is motivation. The target element information is combined, and the combined text information obtained is "the subject object described in the text content corresponding to the text data is employee A, and the emotion expressed is motivation." Then, the sentence "please generate a summary text for the target element information" can be input into the large language model, so that the large language model generates a summary text for summarizing the text data from at least some perspectives, and the summary text is: "The text describes the content that motivates employee A to complete the work."
[0072] Of course, in actual application, multiple combined text information can be determined according to actual needs to generate multiple summary texts for subsequent training of the language model to be trained.
[0073] The server can match each generated summary text with the text data through the large language model (such as semantic matching through word embedding technology), and then obtain the matching results between each summary text and the text data as the actual matching results. For example: for the summary text "The text describes the content that motivates employee A to complete the work", the sentence "Please match the two." can be input into the large language model so that the large language model matches the summary text with the text data, and then obtain the matching results between the summary text and the text data. The matching result is: "Since the text data mentions the content of hoping that employee B will make a work plan to improve his work efficiency, and does not involve the content of motivating employee A to complete the work, the two do not match."
[0074] For another example, for the summary text "The text discusses the work situation of employee B by using a contrasting approach", the sentence "Please match the two." can be input into the large language model so that the large language model matches the summary text with the text data, and then the matching result between the summary text and the text data can be obtained. The matching result is: "The text data mentioned "Employee B's work efficiency is low", "Employee A can always complete the work efficiently", and "Employee B should quickly make a work plan in advance", which shows that Employee A is compared with Employee B, so as to discuss the work situation of Employee B, which is consistent with "The text discusses the work situation of Employee B by using a contrasting approach", so the two are matched.".
[0075] Therefore, a training data set may be constructed based on the text data, each summary text, and the corresponding actual matching results, so as to be used for training the language model to be trained in the second stage.
[0076] S103: Inputting the text data and the summary text into a language model to be trained, so that the language model generates a matching result between the text data and the summary text as a matching result to be verified.
[0077] In the second stage, the server can train the language model to be trained based on the training data set constructed in the first stage. Specifically, the server can input the text data and the summary text generated by the large language model into the language model to be trained, so that the language model can generate a matching result between the text data and the summary text as the matching result to be verified.
[0078] It should be noted that in this specification, the type of matching results generated by the language model can be limited. Therefore, while inputting text data and summary text into the language model to be trained, output specification information can also be input into the language model to be trained, so that the language model outputs the matching result between the text data and the summary text according to the information output type required by the output specification information as the matching result to be verified. For example, the input output specification information can be "Please match the summary text with the text data and return the analysis process and analysis results", and then the language model can use the analysis process and analysis results as the matching result to be verified.
[0079] Of course, in order to reduce the complexity of language model training, the matching results of the language model can be limited to be output only in the form of analysis results. For example, the corresponding output specification information can be "please return the analysis results of matching the summary text with the text data", and then the language model can use the analysis results as the matching results to be verified.
[0080] In addition, the matching results of the language model can be limited to be output only in the form of an analysis process. For example, the corresponding output specification information can be "please return the analysis process of matching the summary text with the text data", and then the language model can use the analysis process as the matching result to be verified.
[0081] Accordingly, in actual application, the type of actual matching result generated in the first stage can be adjusted accordingly according to the type of matching result generated by the required language model.
[0082] S104: determining a loss value according to a deviation between the to-be-verified matching result and the actual matching result, and training the to-be-trained language model according to the loss value, wherein there is a positive correlation between the deviation and the loss value.
[0083] The server can determine the loss value based on the deviation between the to-be-verified matching result and the actual matching result. Then, the language model to be trained can be trained based on the determined loss value. There is a positive correlation between the deviation and the loss value. In other words, the server can train the language model to be trained with the optimization goal of gradually reducing the deviation between the to-be-verified matching result and the actual matching result.
[0084] It can be seen from this that the above method can input text data into a large language model so that the large language model determines the summary text according to the text element information corresponding to the text data. In addition, the text data and the summary text determined by the large language model can be input as sample data into the language model to be trained, so as to train the language model, thereby avoiding the problem of low training efficiency caused by the long time spent in training the language model by artificially setting multiple question statements in the prior art, and greatly improving the training efficiency. At the same time, the summary text used in the training process of the above method is determined by the preset large language model according to the text element information obtained by semantic analysis from multiple angles. Therefore, the training using the above method can enable the language model to learn the ability to analyze and process summary texts from multiple angles, thereby improving the accuracy of the matching results output by the model.
[0085] It should be pointed out that the above method involves two models in the implementation process, one is a large language model, and the other is a language model to be trained. These two models are not the same model. The large language model is used to give summary text corresponding to text data from multiple angles, and give the matching results between text data and summary text. This matching result can be used as label data used by the language model to be trained during the training process.
[0086] The above mainly introduces the model training method. After the model training is completed, it can be deployed for business execution. Next, a business execution method provided in this specification will be described.
[0087] Figure 2 A flowchart of a business execution method provided in this specification includes the following steps:
[0088] S201: Acquire data to be matched, wherein the data to be matched includes text data to be matched and summary text to be matched.
[0089] For the service execution method provided in this specification, the execution subject can be a server or a terminal device such as a desktop computer, a laptop computer, etc. The following only takes the terminal device as an example to explain the subsequent content in detail.
[0090] In this specification, the language model trained by the above model training method can be deployed in the terminal device. The terminal device can respond to the user's matching request and obtain the data to be matched, wherein the data to be matched can include text data to be matched and summary text to be matched.
[0091] In actual application, users can input the text data to be matched and the summary text to be matched into the terminal device through the display page of the terminal device where the language model is deployed, so as to perform business execution according to the matching results of the two given by the terminal device. The display page can be specifically referred to as follows Figure 3 .
[0092] Figure 3 A schematic diagram of a display page provided in this manual.
[0093] Depend on Figure 3 It can be seen that the display page contains a control named "Upload Document", and the user can select the required text data to be matched by touching the control. In addition, the display page also contains an input box, and the user can enter the summary text to be matched in the input box. The display page also contains a control named "Send", and the user can touch the control after entering the text data to be matched and the summary text to be matched, so that the terminal device can display the matching results in the matching result display area.
[0094] S202: Inputting the data to be matched into a pre-trained language model so that the language model generates a matching result between the text data to be matched and the summary text to be matched, wherein the language model is trained by the above-mentioned model training method.
[0095] The terminal device can generate a matching result between the text data to be matched and the summary text to be matched through the language model in response to the user's specified operation. Figure 3 The "Send" control shown in is touched, and the data to be matched input by the user is input into the deployed language model, so that the language model can generate a matching result between the text data to be matched and the summary text to be matched, and the matching result can be displayed in the matching result display area. For details, please refer to the following Figure 4 .
[0096] Figure 4 A schematic diagram of a display page provided in this manual.
[0097] Depend on Figure 4 It can be seen that the user inputs the text data to be matched (a piece of news) and the summary text to be matched "Does the text describe the economic information of region A?" into the terminal device, and the terminal device can match the text data to be matched with the summary text data to be matched through the deployed language model, and return the matching result "Because the input text data to be matched describes "economic indicators such as the tax policy, inflation rate, unemployment rate, etc. in the region", this information belongs to economic information, so the two match."
[0098] S203: Execute the service according to the matching result.
[0099] Then, business execution (such as text classification business) can be performed based on the matching results. For example, according to the obtained matching result "Since the input text data to be matched describes "economic indicators such as tax policies, inflation rates, and unemployment rates in region A", this information belongs to economic information, so the two are matched.", the text data to be matched (a piece of news) can be classified as financial news. Of course, natural language processing businesses such as text sentiment analysis can also be performed based on the matching results, which will not be elaborated in this manual.
[0100] The above are one or more implementation methods of this specification. Based on the same idea, this specification also provides a corresponding model training device, such as Figure 5 shown.
[0101] Figure 5 A schematic diagram of a model training device provided in this specification includes:
[0102] Acquisition module 501: used to acquire text data;
[0103] The first matching module 502 is used to input the text data into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generate a summary text for summarizing the text data from at least a partial perspective based on each text element information, so as to match the summary text with the text data to obtain a matching result between the summary text and the text data as an actual matching result, wherein the text element information is used to characterize the sentence components of the text data, the emotion type expressed by the text content corresponding to the text data, and the description style of the text content corresponding to the text data;
[0104] The second matching module 503 is used for inputting the text data and the summary text into the language model to be trained, so that the language model generates a matching result between the text data and the summary text as a matching result to be verified;
[0105] Training module 504: used to determine a loss value according to the deviation between the to-be-verified matching result and the actual matching result, and train the to-be-trained language model according to the loss value, wherein there is a positive correlation between the deviation and the loss value.
[0106] Optionally, the first matching module 502 is specifically used to: input the text data into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and select at least one target element information from the each text element information, to combine the selected at least one target element information to obtain combined text element information, and generate a summary text for summarizing the text data from at least a partial perspective based on the combined text element information.
[0107] Optionally, the first matching module 502 is specifically used to: input the text data into a preset large language model so that the large language model determines the text features corresponding to the text data, and based on the text features, perform semantic analysis on the text data to determine each text element information corresponding to the text data, and based on the text features, determine important element information from the each text element information as at least one selected target element information.
[0108] Optionally, the first matching module 502 is specifically used to: input the text data and the acquired task requirement information into a preset large language model, so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and according to the task requirement information, determine the text element information that matches the task requirement information from the various text element information as at least one selected target element information.
[0109] Optionally, the second matching module 503 is specifically used to: input the text data, the summary text and the output specification information into the language model to be trained, so that the language model outputs the matching result between the text data and the summary text according to the information output type required by the output specification information, as the matching result to be verified.
[0110] This specification also provides a service execution device, such as Figure 6 shown.
[0111] Figure 6 A schematic diagram of a service execution device provided in this specification includes:
[0112] Acquisition module 601: used to acquire data to be matched, wherein the data to be matched includes text data to be matched and summary text to be matched;
[0113] Matching module 602: used for inputting the data to be matched into a pre-trained language model so that the language model generates a matching result between the text data to be matched and the summary text to be matched, wherein the language model is trained by the above-mentioned model training method;
[0114] Execution module 603: used to execute the service according to the matching result.
[0115] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 The model training method provided or the above Figure 2 Provides business execution methods.
[0116] This manual also provides Figure 7 The schematic structure diagram of the electronic device shown in FIG. Figure 7 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The model training method shown or the above Figure 2Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0117] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0118] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0119] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0120] For the convenience of description, the above device is described by dividing it into various units according to its functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0121] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0122] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0123] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0125] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0126] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0127] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0128] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0129] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0130] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0131] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0132] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.
Claims
1. A model training method, comprising: Get text data; Input the text data into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generate a summary text for summarizing the text data from at least a partial perspective based on each text element information, so as to match the summary text with the text data to obtain a matching result between the summary text and the text data as an actual matching result, wherein the text element information is used to characterize the sentence components of the text data, the emotion type expressed by the text content corresponding to the text data, and the description style of the text content corresponding to the text data; Inputting the text data and the summary text into a language model to be trained, so that the language model generates a matching result between the text data and the summary text as a matching result to be verified; A loss value is determined according to a deviation between the to-be-verified matching result and the actual matching result, and the to-be-trained language model is trained according to the loss value, wherein there is a positive correlation between the deviation and the loss value.
2. The method according to claim 1, wherein the text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generates a summary text for summarizing the text data from at least a partial perspective based on each text element information, specifically comprising: The text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and selects at least one target element information from the each text element information, so as to combine the selected at least one target element information to obtain combined text element information, and generate a summary text for summarizing the text data from at least a partial perspective based on the combined text element information.
3. The method according to claim 2, wherein the text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and selects at least one target element information from each text element information, specifically comprising: The text data is input into a preset large language model so that the large language model determines text features corresponding to the text data, and based on the text features, semantic analysis is performed on the text data to determine each text element information corresponding to the text data, and based on the text features, important element information is determined from the each text element information as at least one selected target element information.
4. The method according to claim 2, wherein the text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and selects at least one target element information from each text element information, specifically comprising: The text data and the acquired task requirement information are input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and according to the task requirement information, determine the text element information that matches the task requirement information from the various text element information as at least one selected target element information.
5. The method according to claim 1, inputting the text data and the summary text into a language model to be trained so that the language model generates a matching result between the text data and the summary text as a matching result to be verified, specifically comprising: The text data, the summary text and the output regulation information are input into the language model to be trained, so that the language model outputs the matching result between the text data and the summary text as the matching result to be verified according to the information output type required by the output regulation information.
6. A business execution method, comprising: Acquire data to be matched, wherein the data to be matched includes text data to be matched and summary text to be matched; Inputting the data to be matched into a pre-trained language model so that the language model generates a matching result between the text data to be matched and the summary text to be matched, wherein the language model is trained by the method according to any one of claims 1 to 5; The business is executed according to the matching result.
7. A model training device, comprising: Acquisition module: used to obtain text data; A first matching module: used for inputting the text data into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and generates a summary text for summarizing the text data from at least a partial perspective based on each text element information, so as to match the summary text with the text data and obtain a matching result between the summary text and the text data as an actual matching result, wherein the text element information is used to characterize the sentence components of the text data, the emotion type expressed by the text content corresponding to the text data, and the description style of the text content corresponding to the text data; A second matching module: used for inputting the text data and the summary text into a language model to be trained, so that the language model generates a matching result between the text data and the summary text as a matching result to be verified; Training module: used to determine a loss value according to the deviation between the to-be-verified matching result and the actual matching result, and to train the to-be-trained language model according to the loss value, wherein the deviation is positively correlated with the loss value.
8. The device according to claim 7, wherein the first matching module is specifically configured to: The text data is input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and selects at least one target element information from the each text element information, so as to combine the selected at least one target element information to obtain combined text element information, and generate a summary text for summarizing the text data from at least a partial perspective based on the combined text element information.
9. The device according to claim 8, wherein the first matching module is specifically configured to: The text data is input into a preset large language model so that the large language model determines text features corresponding to the text data, and based on the text features, semantic analysis is performed on the text data to determine each text element information corresponding to the text data, and based on the text features, important element information is determined from the each text element information as at least one selected target element information.
10. The device according to claim 8, wherein the first matching module is specifically configured to: The text data and the acquired task requirement information are input into a preset large language model so that the large language model performs semantic analysis on the text data to determine each text element information corresponding to the text data, and according to the task requirement information, determine the text element information that matches the task requirement information from the various text element information as at least one selected target element information.
11. The device according to claim 7, wherein the second matching module is specifically configured to: The text data, the summary text and the output regulation information are input into the language model to be trained, so that the language model outputs the matching result between the text data and the summary text as the matching result to be verified according to the information output type required by the output regulation information.
12. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 6.
13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.