Model training method, text generation method and device

Abstract

The application provides a model training method and device and a text generation method and device. The method comprises: obtaining sample text generated based on a sample table; inputting the sample text into an encoding module of a text processing model for encoding processing to obtain an encoding vector; inputting the encoding vector into a decoding module of the text processing model for decoding processing to obtain a decoding vector; determining a first loss value according to the encoding vector and the decoding vector; and if the first loss value is greater than a first threshold value, adjusting model parameters of the text processing model according to the first loss value to obtain a text processing model, which can generate accurate text content based on table content.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to model training methods, text generation methods and apparatus. Background Technology

[0002] Table-to-text (Table-to-text) is a technique that generates natural language text descriptions from information elements in a given structured table, helping users quickly extract key information from tables. It has wide applications in scenarios such as biographical creation, weather forecasts, and news / sports reporting. Therefore, how to convert tables into text with high quality is an important research topic worthy of study.

[0003] In related technologies, text processing models are used to process tables and generate text. Specifically, these models are trained by adjusting their parameters using the loss values ​​of the predicted and labeled values. However, this method results in text processing models with low accuracy in generating text. Summary of the Invention

[0004] This application provides several aspects of model training methods, text generation methods, and apparatus to train a text processing model that can improve the accuracy of text generated from tables.

[0005] The first aspect of this application provides a model training method, comprising: acquiring sample text generated based on a sample table; inputting the sample text into the encoding module of a text processing model for encoding processing to obtain an encoding vector; inputting the encoding vector into the decoding module of the text processing model for decoding processing to obtain a decoding vector; determining a first loss value based on the encoding vector and the decoding vector; and adjusting the model parameters of the text processing model based on the first loss value if the first loss value is greater than a first threshold.

[0006] The second aspect of this application provides a text generation method, comprising: acquiring a text to be processed, the text to be processed including a query text and an answer text, the answer text being extracted from a target table based on the query text; inputting the text to be processed into a text processing model for text processing to obtain a target text for replying to the query text, the text processing model being trained according to the model training method of the first aspect.

[0007] The third aspect of this application provides a text generation method applied to a terminal device. The text generation method includes: acquiring text to be processed; sending the text to be processed to a server; and receiving target text sent by the server, wherein the target text is determined by the server according to the text generation method of the second aspect.

[0008] A fourth aspect of this application provides a model training method applied to a cloud server, comprising: acquiring sample text generated based on a sample table; inputting the sample text into the encoding module of a text processing model for encoding processing to obtain an encoding vector; inputting the encoding vector into the decoding module of the text processing model for decoding processing to obtain a decoding vector; determining a first loss value based on the encoding vector and the decoding vector; and adjusting the model parameters of the text processing model based on the first loss value if the first loss value is greater than a first threshold.

[0009] A fifth aspect of this application provides a text generation system, comprising:

[0010] Cloud servers and terminal devices, with pre-trained text processing models deployed on the cloud servers;

[0011] Terminal devices are used to acquire the text to be processed and send the text to the server.

[0012] A cloud server is used to acquire the text to be processed, which includes query text and answer text. The answer text is extracted from the target table based on the query text. The text to be processed is input into the text processing model for text processing to obtain the target text used to reply to the query text. The text processing model is trained according to the model training method in the first aspect.

[0013] Terminal devices are used to receive target text sent by the server.

[0014] A sixth aspect of this application provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a model training method as described in the first aspect, or a text generation method as described in the second or third aspect.

[0015] A seventh aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement a model training method as described in the first aspect, or a text generation method as described in the second or third aspect.

[0016] This application embodiment is applied to the scenario of generating text from a table. It obtains sample text generated based on a sample table; inputs the sample text into the encoding module of the text processing model for encoding processing to obtain an encoding vector; inputs the encoding vector into the decoding module of the text processing model for decoding processing to obtain a decoding vector; determines a first loss value based on the encoding vector and the decoding vector; if the first loss value is greater than a first threshold, adjusts the model parameters of the text processing model based on the first loss value to train a text processing model that can generate accurate text content based on table content. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 An application scenario diagram provided for an exemplary embodiment of this application;

[0019] Figure 2 Another application scenario diagram provided for an exemplary embodiment of this application;

[0020] Figure 3 A flowchart illustrating the steps of a model training method provided in an exemplary embodiment of this application;

[0021] Figure 4 A schematic diagram of the structure of a text processing model provided as an exemplary embodiment of this application;

[0022] Figure 5 A heterogeneous graph of text provided for an exemplary embodiment of this application;

[0023] Figure 6 A flowchart illustrating the steps of a model training method provided in an exemplary embodiment of this application;

[0024] Figure 7 A flowchart illustrating the steps of a text generation method provided in an exemplary embodiment of this application;

[0025] Figure 8 A structural block diagram of a model training apparatus provided for an exemplary embodiment of this application;

[0026] Figure 9 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Table-to-text generation aims to generate corresponding text descriptions from given structured tabular data, helping users quickly extract key information from structured data. It has wide applications in scenarios such as biographical generation, weather forecasts, and news and sports event reporting. However, current table-to-text generation methods suffer from the following three problems: the generated text is not accurate enough relative to the input tabular data.

[0029] To address the aforementioned issues, this application proposes a model training method. This method involves obtaining sample text generated from a sample table; inputting the sample text into the encoding module of a text processing model for encoding processing to obtain an encoding vector; inputting the encoding vector into the decoding module of the text processing model for decoding processing to obtain a decoding vector; determining a first loss value based on the encoding and decoding vectors; and adjusting the model parameters of the text processing model based on the first loss value if the first loss value is greater than a first threshold, thereby training a text processing model capable of generating accurate text content based on table content.

[0030] In this embodiment, the execution device of the model training method is not limited. Optionally, the model training method can be implemented using a cloud computing system. For example, the model training method can be applied to a cloud server to leverage the advantages of cloud resources to run various models; alternatively, the model training method can also be applied to server-side devices such as conventional servers, cloud servers, or server arrays.

[0031] In addition, refer to Figure 1 This is an application scenario diagram of this application. Terminal device 11 sends a table to be processed to server 12. Server 12 generates text to be processed based on the table, such as "Product Name, X No. 1; Income Type, Principal-Protected; Rate of Return, Greater than 3%; Company Name: AA". Furthermore, server 12 is equipped with a text processing model. After the text to be processed is input into the text processing model, it outputs the target text in natural language, such as "AA Company's X No. 1 is a principal-protected product with a rate of return > 3%". It can be seen that in... Figure 1 The target text generated by the text processing model is a description of the content in the table to be processed.

[0032] Furthermore, referring to Figure 2In another application scenario provided by this application, the terminal device 11 sends the user-input query text to the server 12. The query text might be, "What products have a principal-protected return type and a yield > 3%?". The server 12 queries the target table for the corresponding answer text based on the query text. The answer text might be, "Product Name, X No. 1". Then, the server 12 combines the query text and the answer text to generate a text to be processed, such as "Product Name; Return Type, Principal-Protected; Yield > 3% | Product Name: X No. 1". This text is then input into a text processing model to obtain the target text, such as "The principal-protected product with a yield > 3% is X No. 1". Finally, the target text is sent to the terminal device 11 to reply to the query text. Alternatively, in this embodiment, the target table can be stored in the terminal device 11, the text to be processed can be generated in the terminal device, and the text to be processed can be sent to the server 12. After processing by the text processing model, the target text is obtained.

[0033] Figure 1 and Figure 2 This is merely an exemplary application scenario of this application. This application can also be applied to other scenarios where tables are generated into text, and is not limited thereto.

[0034] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0035] Figure 3 A flowchart illustrating the steps of a model training method provided for an exemplary embodiment of this application. Figure 3 The model training method shown includes the following steps:

[0036] S301, Obtain sample text generated based on the sample table.

[0037] The sample text can be content from a sample table, or it can be partly from the query text and partly from the sample table. For example, refer to... Figure 1 If the sample text is "Product Name, X No. 1; Income Type, Principal-Protected; Rate of Return, Greater than 3%; Company Name: AA", then this sample text is in the sample table (e.g., ...). Figure 1 Extracted from the table to be processed. (Refer to...) Figure 2 If the sample text is "Product Name; Return Type, Principal-Protected; Rate of Return, >3% | Product Name: X No. 1", then part of the text "Product Name; Return Type, Principal-Protected; Rate of Return, >3%" comes from the query text, and the other part of the text "Product Name: X No. 1" comes from the sample table (e.g., ...). Figure 2 Extracted from the target table in the document.

[0038] Furthermore, the sample text can consist of type text and value text describing the type text. For example, Figure 1The sample text can be “Product Name, X No. 1; Income Type, Principal-Protected; Rate of Return, Greater than 3%; Company Name: AA”, where the type text is “Product Name” and the corresponding value text is “X No. 1”; the type text is “Income Type” and the corresponding value text is “Price-Protected”; the type text is “Rate of Return” and the value text is “>3%”; and the type text is “Company Name” and the value text is “AA”.

[0039] In this embodiment, the sample text is non-natural language, generated based on a sample table. Furthermore, the sample text is pre-generated and may be annotated text for use in training the text processing model.

[0040] S302, input the sample text into the encoding module of the text processing model for encoding processing to obtain the encoding vector.

[0041] Reference Figure 4 This is a schematic diagram of the text processing model, which includes an encoding module, a decoding module, and a word predictor. The sample text is input into the encoding module for encoding processing, which yields the corresponding encoded vector.

[0042] Furthermore, in this embodiment of the application, if the sample text contains query text and answer text, the query text and answer text can be modeled as graph structures respectively, and then a unified heterogeneous graph can be constructed according to the attribute connections between the query text and answer text. The sample text is represented by this heterogeneous graph, and the heterogeneous graph is input into the text processing model for training.

[0043] The query text includes selection text and conditional text; the answer text is the text in the table that meets the conditional text and whose text type is selection text. For example, in... Figure 2 In the context of a query text like "What products offer a principal-protected return with a yield greater than 3%?", the selection text within that query text would be "Product Type," and the condition text would be "Principal-protected return and yield greater than 3%". Similarly, an answer text like "Product Name, X No. 1" would have the same type text as the selection text in the query text, also named "Product Name".

[0044] For example, refer to Figure 5 , which is a heterogeneous graph corresponding to the sample text, including structure graph 51 corresponding to the query text and structure graph 52 corresponding to the answer text. Among them, in Figure 5In this diagram, SQL represents the query text, S represents the selection text, and W represents the condition text. When there are multiple conditions, an "and" node is added. Nodes under the key-value nodes represent the type text and the corresponding value text. Under the node corresponding to the selection text, a node corresponding to the specific selection text (e.g., "product name") is connected. This node then connects to the structure diagram corresponding to the answer text.

[0045] In the embodiments of this application, Figure 5 This is just one example of a heterogeneous graph for sample text. Sample text can also be represented by other heterogeneous graphs, which are not limited here.

[0046] Furthermore, referring to Figure 4 The encoding module includes an encoder, adapter, layer normalization, and a graph attention network (GAT). In the encoder module, a transformer encoder (a neural network encoding technique) is used, and a structured adapter is added after the feedforward sublayer of this encoder to model the input text. Furthermore, layer normalization can normalize at different time steps to handle both single samples and variable-length sequences. The graph attention network (GAT) leverages hidden self-attention layers to address the shortcomings of previous graph convolutional or approximation-based methods. By stacking layers, nodes can participate in the features of their neighbors, assigning different weights to different nodes in the neighborhood without requiring any costly matrix operations or prior knowledge of the graph structure.

[0047] In this embodiment, the encoding module can also have other structures, which are not limited here. The sample text provided in this embodiment can be encoded into an encoding vector H by the encoding module. This encoding vector can be composed of a series of sub-encoding vectors, which encode a sub-text of the sample text. For example, for the sample text "Product Name, X No. 1; Income Type, Principal-Protected; Rate of Return, Greater than 3%; Company Name: AA", then "Product Name" corresponds to sub-encoding vector a1, "X No. 1" corresponds to sub-encoding vector a2, "Income Type" corresponds to sub-encoding vector a3, "Price-Protected" corresponds to sub-encoding vector a4, "Rate of Return" corresponds to sub-encoding vector a5, "Greater than 3%" corresponds to sub-encoding vector a6, "Company Name" corresponds to sub-encoding vector a7, and "AA" corresponds to sub-encoding vector a8. Sub-encoding vectors a1 to a8 are combined to obtain the encoding vector.

[0048] S303, input the encoded vector into the decoding module of the text processing model for decoding processing to obtain the decoded vector.

[0049] In this embodiment, the decoding module includes a decoder, which employs a transformer decoder (a neural network decoding technique) as the encoder. The decoded vector is the feature vector corresponding to the predicted text. (Refer to...) Figure 4 After the decoded vector Z passes through the word predictor, the predicted text can be obtained.

[0050] The decoding vector can be composed of a series of sub-decoding vectors, each sub-decoding vector corresponding to a sub-text in the predicted text. For example, if the predicted text is "yield > 3% is X No. 1", then sub-decoding vector b1 corresponds to "yield", sub-decoding vector b2 corresponds to ">3%", sub-decoding vector b3 corresponds to "is", and sub-decoding vector b4 corresponds to "X No. 1".

[0051] S304. Determine the first loss value based on the encoded vector and the decoded vector.

[0052] In this embodiment, a preset loss function can be used to determine the loss value between the encoding vector and the decoding vector, which is the first loss value. For example, the first loss value can be determined using the following formula (1):

[0053]

[0054] In the above formula, T represents the first loss value. ij x is a constant i Let y represent the i-th sub-coded vector. j Let d(x) represent the j-th sub-decoded vector. m represents the number of sub-coded vectors, and n represents the number of sub-decoded vectors. i y j ) indicates the calculation of x i and y j The KL distance (Kullback-Leibler, relative entropy) between them.

[0055] In the embodiments of this application, other loss functions can be used to determine the loss value between the encoded vector and the decoded vector.

[0056] S305, If the first loss value is greater than the first threshold, adjust the model parameters of the text processing model according to the first loss value.

[0057] In this context, a smaller first loss value indicates that the encoding and decoding vectors are closer, and the semantics of the predicted text will be more consistent with the semantics of the sample text. Conversely, a larger loss value indicates that the encoding and decoding vectors are further apart, and the semantics of the predicted text will be less consistent with the semantics of the sample text.

[0058] In this embodiment, the trained text processing model needs to accurately represent the text content in the table when generating natural language text content based on the text content in the table. Therefore, if the first loss value is greater than a preset first threshold, the model parameters of the text processing model are adjusted according to the first loss value. The model parameters of the text processing model include the encoding parameters of the encoding module and the decoding parameters of the decoding module.

[0059] In this embodiment, the first threshold is preset, and the model parameters can be represented by matrix W. When the first loss is greater than the first threshold, the model parameters of the text processing model can be adjusted by adjusting the values ​​in matrix W.

[0060] Furthermore, if the first loss value is less than or equal to the first threshold, another target text can be used to continue training the text processing model until the text processing model meets the preset training requirements, thus obtaining a trained text processing model. The preset training requirements, for example, include reaching a preset number of training iterations.

[0061] This application embodiment is applied to the scenario of generating text from a table. It obtains sample text generated based on a sample table; inputs the sample text into the encoding module of the text processing model for encoding processing to obtain an encoding vector; inputs the encoding vector into the decoding module of the text processing model for decoding processing to obtain a decoding vector; determines a first loss value based on the encoding vector and the decoding vector; if the first loss value is greater than a first threshold, adjusts the model parameters of the text processing model based on the first loss value to train a text processing model that can generate accurate text content based on table content.

[0062] Reference Figure 6 The flowchart illustrates another model training method provided as an exemplary embodiment of this application. Figure 6 The model training method shown includes the following steps:

[0063] S601, Obtain sample text generated based on the sample table.

[0064] The specific implementation process of this step is described in S301 and will not be repeated here.

[0065] S602, with some parameters of the encoding module discarded, the sample text is input into the encoding module for encoding processing to obtain the encoding vector.

[0066] In this embodiment, the encoding parameters of the encoding module can be represented by W, where W is a matrix. Discarding some parameters of the encoding module can be achieved by setting a certain item, row, or column of the matrix to 0, resulting in new encoding parameters. Different parameters can be discarded to obtain different encoding parameters, such as W1, W2, and W3.

[0067] Furthermore, by discarding some parameters from the encoding module and inputting the sample text into the encoding module for encoding processing, the corresponding encoding vectors can be obtained. For example, the encoding vector obtained with encoding parameter W1 is h1, the encoding vector obtained with encoding parameter W2 is h2, and the encoding vector obtained with encoding parameter W3 is h3.

[0068] S603, determine the second loss value based on the encoding vector obtained by discarding different parameters of the encoding module.

[0069] In this embodiment of the application, it is required that when the encoding module discards the parameters of different encoding modules, the difference between the encoding vectors obtained by encoding the same sample text should not be too large, so as to ensure the modeling capability of the encoding module.

[0070] Furthermore, a pre-defined loss function can be used to determine the second loss value based on the encoding vectors obtained from different parameters of the discarding encoding module, as shown in the following formula (2):

[0071]

[0072] In the above formula, l i H represents the second loss value. i This indicates that the encoding vector corresponding to the encoding module parameters is not lost. This represents the i-th encoded vector obtained when the encoding module parameters are lost. This represents the j-th encoded vector obtained when the encoding module parameters are lost, where N represents the number of new encoding parameters, and τ is a constant.

[0073] In the embodiments of this application, other loss functions may also be used to determine the second loss value.

[0074] S604, If the second loss value is greater than the second threshold, adjust the encoding parameters of the encoding module according to the second loss value.

[0075] Specifically, a smaller second loss value indicates a smaller difference between the encoded vectors, meaning the loss of parameters in the encoding module has a smaller impact on the encoding module's encoding capability, thus indicating a stronger encoding capability. Conversely, a larger loss value indicates a larger difference between the encoded vectors, meaning the loss of parameters in the encoding module has a larger impact on the encoding module's encoding capability, thus indicating a weaker encoding capability.

[0076] In this embodiment, the trained encoding module needs to have strong encoding capabilities, meaning it can accurately encode the input text even when some encoding parameters are lost. Therefore, if the second loss value is greater than a preset second threshold, the encoding parameters of the encoding module are adjusted according to the second loss value.

[0077] Furthermore, if the first loss value is less than or equal to the first threshold, another target text can be used to continue training the encoding module until the encoding parameters of the encoding module meet the preset training requirements, thus obtaining a trained text processing model.

[0078] S605 inputs the encoded vector into the decoding module of the text processing model for decoding to obtain the decoded vector.

[0079] The specific implementation process of this step is described in S303 and will not be repeated here.

[0080] S606, determine the first loss value based on the encoded vector and the decoded vector.

[0081] The specific implementation process of this step is described in S304 and will not be repeated here.

[0082] S607, If the first loss value is greater than the first threshold, adjust the model parameters of the text processing model according to the first loss value.

[0083] The specific implementation process of this step is described in S305 and will not be repeated here.

[0084] S608 determines the third loss value based on multiple sub-decoding vectors.

[0085] The decoding vector includes multiple sub-decoding vectors.

[0086] Furthermore, a preset loss function can be used to determine a third loss value among multiple sub-encoding vectors. This application does not limit the scope of this loss function.

[0087] S609, If the third loss value is less than the third threshold, adjust the model parameters of the text processing model according to the third loss value.

[0088] In this embodiment, the text processing model trained needs to generate diverse natural language text content based on the text content in the table. That is, the generated predicted text should ideally not contain duplicate text segments. For example, if the predicted text is "A principal-protected product with a yield > 3% is X-1", this predicted text does not contain duplicate text segments, meeting the training requirements. However, if the predicted text is "A principal-protected product with a yield > 3%, and the principal-protected product's name is X-1", this predicted text contains the duplicate text segment "principal-protected product", then this predicted text does not meet the training requirements. Therefore, the training requirement necessitates that the sub-decoding vectors be as different as possible.

[0089] Therefore, if the third loss value is less than the preset third threshold, it is determined that the sub-encoding and decoding vectors are highly similar, and the model parameters of the text processing model are adjusted according to the third loss value.

[0090] Furthermore, if the third loss value is greater than or equal to the third threshold, another target text can be used to continue training the text processing model until the text processing model meets the preset training requirements.

[0091] In the embodiments of this application, the text processing model is trained using a third loss value, which can improve the diversity of natural language text generated by the text processing model based on the text content in the table.

[0092] S610, obtain the label text corresponding to the sample text.

[0093] The label text is the truth value annotated against the sample text. For example, the label text is "The principal-protected product with a yield > 3% is X No. 1".

[0094] S611, determine the fourth loss value based on the decoded vector and the label text.

[0095] The decoded vector can be input into the word predictor, which outputs the corresponding predicted text. The fourth loss value is then determined based on the predicted text and the label text.

[0096] In this embodiment of the application, a preset loss function can be used to determine the fourth loss value, and this embodiment of the application does not limit the loss function.

[0097] S612, If the fourth loss value is greater than the fourth threshold, adjust the model parameters of the text processing model according to the fourth loss value.

[0098] In this embodiment, the predicted text output by the trained text processing model is required to be closer to the labeled text. Therefore, if the fourth loss value is greater than a preset fourth threshold, it is determined that the similarity between the predicted text and the labeled text is low, and the model parameters of the text processing model are adjusted according to the fourth loss value. Furthermore, if the fourth loss value is less than or equal to the fourth threshold, another target text can be used to continue training the text processing model until the text processing model meets the preset training requirements.

[0099] In this embodiment of the application, the text processing model is trained using a fourth loss value, which can improve the accuracy of the text processing model in generating natural language text based on the text content in the table.

[0100] S613, determine the first similarity between the sub-decoding vectors.

[0101] The first similarity between sub-encoding vectors can be determined using a similarity function.

[0102] S614, determine the second similarity between the decoded vector and the tag text.

[0103] Specifically, the label text can be encoded into a corresponding label vector, and then a second similarity between the decoded vector and the label vector can be determined.

[0104] S615, determine the fifth loss value based on the difference between the first similarity and the second similarity.

[0105] The fifth loss value can be determined by a preset loss function. The smaller the fifth loss value, the smaller the similarity between the sub-decoding vectors and the greater the similarity between the predicted text and the label text, which better meets the training requirements.

[0106] Furthermore, a preset loss function can be used to determine the fifth loss, for example, the fifth loss value can be determined using the following formula (3):

[0107]

[0108] In formula (3), and Both represent the fifth loss value, z x Represents a sub-decoding vector. Indicates another sub-decoding vector, This represents the feature vector corresponding to a segment of text in the labeled text. Indicates the first similarity. Indicates the second similarity. This represents a set containing sub-decoding vectors and tag vectors, where ξ is a fixed constant.

[0109] S616, If the fifth loss value is greater than the fifth threshold, adjust the model parameters of the text processing model according to the fifth loss value.

[0110] In this embodiment of the application, a preset loss function can be used to determine the fourth loss value, and this embodiment of the application does not limit the loss function.

[0111] S612, If the fourth loss value is greater than the fourth threshold, adjust the model parameters of the text processing model according to the fourth loss value.

[0112] In this embodiment, the predicted text output by the trained text processing model is required to be closer to the labeled text, and the predicted text should exhibit diversity. Therefore, if the fifth loss value is greater than a preset fifth threshold, the text processing model is determined to not meet the training requirements, and the model parameters are adjusted based on the fifth loss value. Furthermore, if the fifth loss value is less than or equal to the fourth threshold, another target text can be used to continue training the text processing model until it meets the preset training requirements.

[0113] In the embodiments of this application, the fifth loss value is used to train the text processing model, which can improve the accuracy and diversity of the text processing model in generating natural language text based on the text content in the table.

[0114] This application improves the encoding module's ability to model input text by employing contrastive learning to adjust its encoding parameters during text processing model training. Furthermore, it enhances the accuracy of text generation by adjusting the model parameters between the decoding module's input and output using contrastive learning. Finally, it increases the diversity of generated text and reduces repetition by adding contrastive learning at the decoding module's output. Ultimately, this application significantly improves the effectiveness of table-to-text generation.

[0115] Reference Figure 7 The text generation method provided in this application specifically includes the following steps:

[0116] S701, Obtain the text to be processed.

[0117] The text to be processed includes query text and answer text, with the answer text being extracted from the target table based on the query text.

[0118] In the embodiments of this application, the text to be processed may be received from a terminal device, or the query text may be received from a terminal device, and then the answer text may be extracted from the target table based on the query text. The query text and the answer text are then combined to obtain the text to be processed.

[0119] S702, input the text to be processed into the text processing model for text processing to obtain the target text for replying to the query text.

[0120] The text processing model was trained using the model training method described in the above embodiments.

[0121] The specific implementation process of the embodiments of this application can be referred to the above embodiments, and will not be repeated here.

[0122] Furthermore, the text processing model obtained by the above-described model training method in this application embodiment can process the text to be processed, and can obtain accurate and diverse target text.

[0123] In addition, this application also provides a text generation method applied to a terminal device. The text generation method includes: acquiring text to be processed; sending the text to be processed to a server; and receiving target text sent by the server, wherein the target text is determined by the server according to the above-described text generation method.

[0124] In addition, this application also provides a text generation system, including:

[0125] Cloud servers and terminal devices, with pre-trained text processing models deployed on the cloud servers;

[0126] Terminal devices are used to acquire the text to be processed and send the text to the server.

[0127] A cloud server is used to retrieve the text to be processed, which includes query text and answer text. The answer text is extracted from the target table based on the query text. The text to be processed is input into the text processing model for text processing to obtain the target text used to reply to the query text. The text processing model is trained according to the model training method described above.

[0128] Terminal devices are used to receive target text sent by the server.

[0129] The specific implementation process is the same as described in the above embodiments, and will not be repeated here.

[0130] In this application embodiment, in addition to providing a text generation method, a model training device is also provided, such as... Figure 8 As shown, the model training device 80 includes:

[0131] Module 81 is used to acquire sample text generated based on the sample table;

[0132] The processing module 82 is used to input the sample text into the encoding module of the text processing model for encoding processing to obtain the encoding vector, and to input the encoding vector into the decoding module of the text processing model for decoding processing to obtain the decoding vector;

[0133] The determination module 83 is used to determine the first loss value based on the encoded vector and the decoded vector;

[0134] The adjustment module 84 is used to adjust the model parameters of the text processing model according to the first loss value if the first loss value is greater than the first threshold.

[0135] In an optional embodiment, when the processing module 82 inputs the sample text into the encoding module of the text processing model for encoding processing to obtain an encoding vector, it is specifically used to: input the sample text into the encoding module for encoding processing while discarding some parameters of the encoding module to obtain an encoding vector.

[0136] In an optional embodiment, different parameters of the discarding encoding module correspond to the obtained encoding vectors. The adjustment module 84 is further configured to: determine a second loss value based on the encoding vectors obtained by discarding different parameters of the encoding module; and adjust the encoding parameters of the encoding module based on the second loss value if the second loss value is greater than a second threshold.

[0137] In an optional embodiment, the decoding vector includes multiple sub-decoding vectors, and the adjustment module 84 is further configured to: determine a third loss value based on the multiple sub-decoding vectors; if the third loss value is less than a third threshold, adjust the model parameters of the text processing model based on the third loss value.

[0138] In an optional embodiment, the adjustment module 84 is further configured to: obtain the label text corresponding to the sample text; determine the fourth loss value based on the decoding vector and the label text; and if the fourth loss value is greater than the fourth threshold, adjust the model parameters of the text processing model based on the fourth loss value.

[0139] In an optional embodiment, the decoding vector includes multiple sub-decoding vectors, and the adjustment module 84 is further configured to: determine a first similarity between the sub-decoding vectors; obtain the label text corresponding to the sample text; determine a second similarity between the decoding vector and the label text; determine a fifth loss value based on the difference between the first and second similarities; and if the fifth loss value is greater than a fifth threshold, adjust the model parameters of the text processing model based on the fifth loss value.

[0140] The model training apparatus provided in this application provides a text processing model that can improve the accuracy of text generated based on tabular data. The specific implementation process is the same as described in the above method embodiments and will not be repeated here.

[0141] Furthermore, embodiments of this application also provide a text generation apparatus (not shown), comprising:

[0142] The acquisition module is used to acquire the text to be processed, which includes the query text and the answer text. The answer text is extracted from the target table based on the query text.

[0143] The processing module is used to input the text to be processed into the text processing model for text processing, and obtain the target text for replying to the query text. The text processing model is trained according to the model training method described above.

[0144] This application embodiment also provides another text generation apparatus (not shown), applied to a terminal device, including:

[0145] The acquisition module is used to acquire the text to be processed;

[0146] The sending module is used to send the text to be processed to the server;

[0147] The receiving module is used to receive the target text sent by the server. The target text is determined by the server according to the text generation method described above.

[0148] The text generation apparatus provided in this application embodiment can improve the accuracy of text generated from tabular data based on a text processing model. The specific implementation process is the same as described in the above method embodiment and will not be repeated here.

[0149] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0150] Figure 9 This is a schematic diagram of the structure of an electronic device 90 provided as an exemplary embodiment of this application. The electronic device is used to run the aforementioned text generation method. For example... Figure 9 As shown, the electronic device includes a memory 94 and a processor 95.

[0151] Memory 94 is used to store computer programs and can be configured to store various other data to support operation on electronic devices. This memory 94 may be object storage (OSS).

[0152] The memory 94 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0153] The processor 95, coupled to the memory 94, is used to execute the computer program in the memory 94 for: acquiring sample text generated based on a sample table; inputting the sample text into the encoding module of the text processing model for encoding processing to obtain an encoding vector; inputting the encoding vector into the decoding module of the text processing model for decoding processing to obtain a decoding vector; determining a first loss value based on the encoding vector and the decoding vector; and adjusting the model parameters of the text processing model based on the first loss value if the first loss value is greater than a first threshold.

[0154] Further optionally, when the processor 95 inputs the sample text into the encoding module of the text processing model for encoding processing to obtain the encoding vector, it specifically performs the following: while discarding some parameters of the encoding module, it inputs the sample text into the encoding module for encoding processing to obtain the encoding vector.

[0155] In an optional embodiment, different parameters of the discarding encoding module correspond to the obtained encoding vectors. The processor 95 is further configured to: determine a second loss value based on the encoding vectors obtained by discarding different parameters of the encoding module; if the second loss value is greater than a second threshold, adjust the encoding parameters of the encoding module based on the second loss value.

[0156] In an optional embodiment, the decoding vector includes multiple sub-decoding vectors, and the processor 95 is further configured to: determine a third loss value based on the multiple sub-decoding vectors; and if the third loss value is less than a third threshold, adjust the model parameters of the text processing model based on the third loss value.

[0157] In an optional embodiment, the processor 95 is further configured to: obtain the label text corresponding to the sample text; determine a fourth loss value based on the decoding vector and the label text; and if the fourth loss value is greater than a fourth threshold, adjust the model parameters of the text processing model based on the fourth loss value.

[0158] In an optional embodiment, the decoding vector includes multiple sub-decoding vectors, and the processor 95 is further configured to: determine a first similarity between the sub-decoding vectors; obtain the label text corresponding to the sample text; determine a second similarity between the decoding vector and the label text; determine a fifth loss value based on the difference between the first and second similarities; and if the fifth loss value is greater than a fifth threshold, adjust the model parameters of the text processing model based on the fifth loss value.

[0159] In one alternative embodiment, the processor 95, coupled to the memory 94, is configured to execute a computer program in the memory 94, and is further configured to: acquire text to be processed, the text to be processed including query text and answer text, the answer text being extracted from a target table based on the query text; input the text to be processed into a text processing model for text processing to obtain target text for replying to the query text, the text processing model being trained according to the model training method described above.

[0160] In one alternative embodiment, the processor 95, coupled to the memory 94, is configured to execute a computer program in the memory 94, and is further configured to: acquire text to be processed; send the text to be processed to the server; and receive target text sent by the server, the target text being determined by the server according to the text generation method described above.

[0161] Furthermore, such as Figure 9As shown, the electronic device also includes other components such as a firewall 91, a load balancer 92, a communication component 96, and a power supply component 93. Figure 9 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 9 The components shown.

[0162] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the method described above.

[0163] Accordingly, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, cause the processor to implement the steps in the method described above.

[0164] The above Figure 9 The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related text from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0165] The above Figure 9 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0166] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable text processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable text processing device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0168] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable text processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable text processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0170] In a typical configuration, a computing device includes one or more processors (CPU and / or GPU), input / output interfaces, network interfaces, and memory.

[0171] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0172] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be used to store text by any method or technology. Text can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store text accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0173] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0174] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A model training method, characterized in that, include: Obtain sample text generated based on the sample table; the sample text consists of type text and value text describing the type. The sample text is input into the encoding module of the text processing model for encoding processing to obtain an encoding vector; The encoded vector is input into the decoding module of the text processing model for decoding to obtain the decoded vector; Determine the loss value between the encoded vector and the decoded vector, and use it as the first loss value; If the first loss value is greater than the first threshold, the model parameters of the text processing model are adjusted according to the first loss value; The model training method also includes: With some parameters of the encoding module discarded, the sample text is input into the encoding module for encoding processing to obtain the encoding vector; The second loss value is determined based on the encoding vectors obtained from different parameters of the discard coding module; If the second loss value is greater than the second threshold, the encoding parameters of the encoding module are adjusted according to the second loss value.

2. The model training method according to claim 1, characterized in that, The decoding vector includes multiple sub-decoding vectors, and the model training method further includes: Based on the multiple sub-decoding vectors, determine the third loss value; If the third loss value is less than the third threshold, the model parameters of the text processing model are adjusted according to the third loss value.

3. The model training method according to claim 2, characterized in that, The model training method also includes: Obtain the tag text corresponding to the sample text; Based on the decoded vector and the tag text, a fourth loss value is determined; If the fourth loss value is greater than the fourth threshold, the model parameters of the text processing model are adjusted according to the fourth loss value.

4. The model training method according to claim 1, characterized in that, The decoding vector includes multiple sub-decoding vectors, and the model training method further includes: Determine the first similarity between the sub-decoding vectors; Obtain the tag text corresponding to the sample text; Determine the second similarity between the decoded vector and the tag text; The fifth loss value is determined based on the difference between the first similarity and the second similarity. If the fifth loss value is greater than the fifth threshold, the model parameters of the text processing model are adjusted according to the fifth loss value.

5. A text generation method, characterized in that, include: Obtain the text to be processed, which includes a query text and an answer text, wherein the answer text is extracted from the target table based on the query text; The text to be processed is input into a text processing model for text processing to obtain target text for replying to the query text. The text processing model is trained by the model training method according to any one of claims 1 to 4.

6. A text generation method, characterized in that, Applied to terminal devices, the text generation method includes: Get the text to be processed; Send the text to be processed to the server; The server receives target text sent by the server, the target text being determined by the server according to the text generation method of claim 5.

7. A model training method, characterized in that, The model training method, applied to a cloud server, includes: Obtain sample text generated based on the sample table; the sample text consists of type text and value text describing the type. The sample text is input into the encoding module of the text processing model for encoding processing to obtain an encoding vector; The encoded vector is input into the decoding module of the text processing model for decoding to obtain the decoded vector; Determine the loss value between the encoded vector and the decoded vector, and use it as the first loss value; If the first loss value is greater than the first threshold, the model parameters of the text processing model are adjusted according to the first loss value; With some parameters of the encoding module discarded, the sample text is input into the encoding module for encoding processing to obtain the encoding vector; The second loss value is determined based on the encoding vectors obtained from different parameters of the discard coding module; If the second loss value is greater than the second threshold, the encoding parameters of the encoding module are adjusted according to the second loss value.

8. A text generation system, characterized in that, include: A cloud server and a terminal device, wherein a pre-trained text processing model is deployed on the cloud server; The terminal device is used to acquire the text to be processed and send the text to be processed to the server; The cloud server is used to acquire the text to be processed, which includes a query text and an answer text. The answer text is extracted from a target table based on the query text. The text to be processed is input into a text processing model for text processing to obtain target text for replying to the query text. The text processing model is trained using the model training method according to any one of claims 1 to 4. The terminal device is used to receive the target text sent by the server.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the model training method as described in any one of claims 1 to 4, and / or the text generation method as described in claim 5 or 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the model training method as described in any one of claims 1 to 4, and / or the text generation method as described in claim 5 or 6.

11. A computer program product, characterized in that, Includes a computer program / instruction, which, when executed by a processor, causes the processor to implement the model training method as described in any one of claims 1 to 4, and / or the text generation method as described in claim 5 or 6.

Images