Text generation method and device, electronic equipment and storage medium
By splicing the initial text with structured text and fusion of features using attention mechanisms, the problem of insufficient accuracy in structured data processing is solved, and the controllability and quality of text generation are improved.
Patent Information
- Application Number
- CN202411009444.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-12-06
AI Technical Summary
When the existing text generation model processes structured data, the generated text is difficult to accurately reflect the content of the original input text, resulting in a decrease in controllability and quality of the output text.
By obtaining the initial text and its corresponding structured text, stitching and inputting it into the text generation model, feature fusion is performed using the attention mechanism based on the structured text encoding features to enhance the attention of the structured text in the initial text, thereby Improve the accuracy of text generation.
Improve the controllability and quality of text generation, so that the generated text more accurately reflects the true intention or meaning of the original input text.
Smart Images

Figure CN119089866A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of financial technology, and in particular to a text generation method and device, an electronic device and a storage medium. Background Art
[0002] In recent years, with the continuous development of deep learning technology, the application of text generation in various scenarios has been promoted, especially in the financial technology field, such as financial report analysis, credit assessment and investment strategy recommendation. There is a large amount of structured data in the financial technology field, and the accuracy of text generation from structured data is required to be high.
[0003] In the related art, text generation tasks in the field of financial technology are often handled by text generation models based on deep learning. However, the text generated by the existing text generation models is difficult to accurately reflect the content of the original input text. Especially when facing structured data, the generated text deviates further from the true intention or meaning of the original input text, thus reducing the controllability of the output text and thus reducing the quality of text generation. Summary of the invention
[0004] The main purpose of the embodiments of the present application is to propose a text generation method and device, an electronic device and a storage medium, which can improve the quality of text generation.
[0005] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application proposes a text generation method, including: obtaining an initial text, and a structured text corresponding to key data in the initial text; splicing the initial text with the structured text to obtain an original input text, and inputting the original input text and the structured text into a text generation model; encoding the original input text to obtain a first encoding feature, and encoding the structured text to obtain a second encoding feature; using an attention mechanism constructed based on the second encoding feature, performing feature fusion processing on the first encoding feature and the second encoding feature to obtain a fused feature; generating text based on the fused feature to obtain a target generated text.
[0006] In some embodiments, the obtaining of the initial text and the structured text corresponding to the key data in the initial text includes: obtaining the initial text, wherein the initial text contains the key data; extracting the key data in the initial text to obtain the data content of the key data under the corresponding data type; and outputting the structured text based on the data type and the corresponding data content.
[0007] In some embodiments, the extracting the key data in the initial text to obtain the data content of the key data under the corresponding data type includes: inputting the initial text into a preset information extraction model to extract the key data to obtain the data content of the key data under the corresponding data type; wherein the information extraction model is jointly trained with the text generation model.
[0008] In some embodiments, the inputting the original input text and the structured text into the text generation model includes: constructing a tag key-value pair using the data type corresponding to the key data in the structured text as a key, and using the data content corresponding to the data type in the structured text as a value; inputting the original input text and the tag key-value pair into the text generation model; encoding the structured text to obtain a second encoding feature includes: encoding the tag key-value pair to obtain a second encoding feature.
[0009] In some embodiments, the step of concatenating the initial text with the structured text to obtain the original input text includes: serializing the structured text to obtain a serialized text; and concatenating the initial text with the serialized text to obtain the original input text.
[0010] In some embodiments, the text generation model includes an encoder and a decoder, the encoder is used to encode the original input text and the structured text; the text generation based on the fused features to obtain the target generated text includes: inputting the fused features into the decoder, decoding the fused features to obtain decoded features; and text generation based on the decoded features to obtain the target generated text.
[0011] In some embodiments, the text generation method also includes: obtaining an initial sample text, and a sample structured text corresponding to key data in the initial sample text; splicing the initial sample text with the sample structured text to obtain a sample input text, and inputting the sample input text and the sample structured text into an initial text generation model; encoding the sample input text to obtain a first sample encoding feature, and encoding the sample structured text to obtain a second sample encoding feature; using an attention mechanism constructed based on the second sample encoding feature, performing feature fusion processing on the first sample encoding feature and the second sample encoding feature to obtain a sample fusion feature; generating text based on the sample fusion feature to obtain a sample generated text, and training the initial text generation model based on the sample generated text to obtain the trained text generation model.
[0012] To achieve the above-mentioned purpose, the second aspect of an embodiment of the present application proposes a text generation device, comprising: a text acquisition module, used to acquire an initial text, and a structured text corresponding to key data in the initial text; a splicing module, used to splice the initial text with the structured text to obtain an original input text, and input the original input text and the structured text into a text generation model; an encoding module, used to encode the original input text to obtain a first encoding feature, and encode the structured text to obtain a second encoding feature; a fusion module, used to utilize an attention mechanism constructed based on the second encoding feature to perform feature fusion processing on the first encoding feature and the second encoding feature to obtain a fused feature; and a generation module, used to generate text based on the fused feature to obtain a target generated text.
[0013] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the text generation method described in the first aspect when executing the computer program.
[0014] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the text generation method described in the first aspect is implemented.
[0015] The text generation method and device, electronic device and storage medium proposed in the present application, by obtaining the initial text, and the structured text corresponding to the key data in the initial text, the initial text and the structured text are spliced to obtain the original input text, which can provide a comprehensive information source containing both rich context and accurate data for the text generation model. Subsequently, the original input text and the structured text are input into the text generation model, the original input text is encoded to obtain the first encoding feature, and the structured text is encoded to obtain the second encoding feature, and the attention mechanism constructed based on the second encoding feature is used to perform feature fusion processing on the first encoding feature and the second encoding feature, so as to strengthen the attention of the structured text in the initial text, so as to accurately reflect the content of the original input text. Finally, the fusion feature is obtained by fusion, and then the text is generated based on the fusion feature, so that the target generated text finally generated can be close to the true intention or meaning of the original input text. Therefore, the embodiment of the present application improves the controllability when outputting the text, thereby improving the quality of text generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is a flowchart of a text generation method provided in an embodiment of the present application;
[0017] Figure 2 yes Figure 1 Flow chart of step S101 in FIG.
[0018] Figure 3 It is a schematic diagram of extracting structured text from initial text provided by an embodiment of the present application;
[0019] Figure 4 yes Figure 1 Flow chart of step S102 in FIG.
[0020] Figure 5 yes Figure 1 Another flowchart of step S102 in FIG. 1 ;
[0021] Figure 6 is a schematic diagram of a text generation process provided by an embodiment of the present application;
[0022] Figure 7 It is a structural diagram of a text generation model provided in an embodiment of the present application;
[0023] Figure 8 yes Figure 1 Flow chart of step S105 in FIG.
[0024] Fig. 9 The text generation method provided in the embodiment of the present application further includes a flowchart;
[0025] Fig.10 is a structural schematic diagram of a text generation device provided in an embodiment of the present application;
[0026] Fig.11 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0028] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0030] First, some nouns involved in this application are analyzed:
[0031] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0032] Information Extraction: A text processing technology that extracts specified types of entity, relationship, event and other factual information from natural language text and forms structured data output. Information extraction is a technology that extracts specific information from text data. Text data is composed of some specific units, such as sentences, paragraphs, and chapters. Text information is composed of some small specific units, such as characters, words, phrases, sentences, paragraphs, or a combination of these specific units. Extracting noun phrases, names, place names, etc. from text data are all text information extraction. Of course, the information extracted by text information extraction technology can be various types of information.
[0033] The attention mechanism originates from the study of human vision. In cognitive science, due to the bottleneck of information processing, humans selectively focus on part of all information while ignoring other visible information. In deep learning, the attention mechanism allows the model to "focus" on the important parts of the data when processing input data, rather than treating all parts equally, thereby improving the efficiency and accuracy of the model when processing complex tasks.
[0034] In recent years, with the continuous development of deep learning technology, the fluency of text generation is constantly improving. Large language models have promoted the generation of texts with diverse styles and rich creativity, which are applied to various scenarios, such as dialogue systems and creative writing. This has promoted the application of text generation in various scenarios, especially in the financial technology field, such as financial report analysis, credit assessment and investment strategy recommendation. There is a large amount of structured data in the financial technology field, and the accuracy of text generation from structured data is required to be high.
[0035] In the related art, text generation tasks in the field of financial technology are often handled by text generation models based on deep learning. However, at this stage, existing text generation models often lack controllability when outputting text, that is, the generated text is difficult to accurately reflect the content of the original input text, which brings challenges to the widespread application of text generation models, especially in the field of structured data generation, such as generating financial analysis texts through financial data, and generating diagnostic texts based on medical records. In this field of structured text generation, the accuracy of the generated text is required to be higher, and the generated text deviates further from the true intention or meaning of the original input text, thus reducing the controllability when outputting the text, thereby reducing the quality of text generation.
[0036] Based on this, the embodiments of the present application provide a text generation method and device, an electronic device and a storage medium, aiming to improve the quality of text generation.
[0037] The text generation method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the text generation method in the embodiments of the present application is described.
[0038] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0039] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0040] The text generation method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The text generation method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms; the software can be an application that implements the text generation method, etc., but is not limited to the above forms.
[0041] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0042] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first. For example, when obtaining the initial text, it is necessary to obtain the user's permission or consent first. Moreover, the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or separate consent will be obtained by means of a pop-up window or jumping to a confirmation page. After clearly obtaining the user's separate permission or separate consent, the necessary user-related data for enabling the normal operation of the embodiment of the present application will be obtained.
[0043] Figure 1 is an optional flowchart of the text generation method provided in the embodiment of the present application. Figure 1The method may include but is not limited to steps S101 to S105.
[0044] Step S101, obtaining an initial text and a structured text corresponding to key data in the initial text;
[0045] Step S102, concatenating the initial text with the structured text to obtain an original input text, and inputting the original input text and the structured text into a text generation model;
[0046] Step S103, encoding the original input text to obtain a first encoding feature, and encoding the structured text to obtain a second encoding feature;
[0047] Step S104, using an attention mechanism constructed based on the second coding feature, performing feature fusion processing on the first coding feature and the second coding feature to obtain a fusion feature;
[0048] Step S105, generating text based on the fused features to obtain a target generated text.
[0049] In the above step S101, the initial text is the basic text that needs to be generated, and is also the text that provides rich context information. Therefore, the initial text contains rich natural language descriptions for describing an event, situation or theme. Key data is the data contained in the initial text, which is the specific data, indicators or parameters contained in the initial text. Understanding the key data in the initial text plays a vital role in the quality of text generation.
[0050] Exemplarily, the initial text may be a financial report text, in which the key data are specific financial-related data, indicators or parameters. For example, the initial text may be a report text of a financial institution, such as "The growth rate of scale remains stable. In the first half of 2020, the growth rate of a bank's total assets slightly decreased by 0.1% month-on-month, remaining stable, and the asset-liability structure improved. On the asset side, loans increased by 2.8% month-on-month, and the number of businesses increased steadily. On the liability side, deposits increased by 3.1% month-on-month, and capital acquisition was good. The interest rate spread narrowed as expected. In the first half of 2020, the interest rate spread fell by 5% month-on-month, in line with expectations. On the asset side, the yield rate decreased by 0.13% month-on-month, and the interest rates of various assets all fell by 0.02% compared with 2019, and the impact was controllable. On the liability side, the cost rate decreased by 0.08% month-on-month. Among them, the deposit cost decreased by 0.04% compared with 2019, and the current situation of deposit costs is expected to improve." In this initial text, the relevant key data include the time period of the first half of 2020, the total assets growth rate was 0.1% month-on-month, the loans were +2.8% month-on-month, the deposits were +3.1% month-on-month, the interest rate spread was -5% month-on-month, the yield was -0.13% month-on-month, the asset interest rate spread was -0.2%, the cost rate was -0.08% month-on-month, and the deposit cost difference rate was -0.04%.
[0051] Exemplarily, the initial text may be obtained from various channels, such as news reports, market analysis, company announcements or related websites, etc. The initial text may also be obtained by user input, which is not specifically limited here.
[0052] Structured text refers to data that corresponds to the key data in the initial text and is presented in a structured form (such as a table, a key-value pair, a database record, etc.), that is, data that presents the key data in a structured form. These data are specific values, proportions, percentages, etc. mentioned in the initial text, which are extracted and represented in a form that is easier for computers to process. Exemplarily, the structured text can be directly extracted from the initial text, or it can be obtained by the user selecting or inputting the corresponding key data based on the initial text, and no specific restrictions are made here.
[0053] In the above step S102, after obtaining the initial text and the structured text, the initial text and the structured text need to be spliced. Splicing is the process of merging the initial text and the structured text into a single text sequence. The splicing method depends on the specific implementation and model requirements, but the goal is to ensure that the model can simultaneously receive the contextual information of the initial text and the accurate data in the structured text, and the embodiment of the present application does not make specific restrictions on this. Further, during the splicing process, the structured text can be converted into a text form so that it can be inserted into the initial text.
[0054] The original input text is the text obtained after concatenation. Therefore, the original input text is both an extension of the initial text and contains precise data in the structured text. This comprehensive information source provides a richer and more comprehensive input for the text generation model, allowing the subsequent model to consider both contextual information and precise data when generating text, thereby generating more accurate and meaningful text.
[0055] A text generation model (or simply a model) is a neural network model trained using machine learning or deep learning techniques that can automatically generate natural language text. The text generation model is trained to capture the statistical laws, semantic structure, and contextual information of the language so that it can generate text that conforms to language habits, is grammatically correct, and has coherent content. In the field of financial technology, text generation models can be applied to a variety of scenarios such as financial report analysis, credit assessment, and investment strategy recommendations. They can automatically generate relevant text descriptions, analysis reports, or recommendations based on input data (such as financial reports, market data, credit scores, etc.).
[0056] Furthermore, the specific implementation of the text generation model in the embodiments of the present application may include a variety of technologies, such as statistical-based natural language processing (NLP) methods, sequence-to-sequence (Seq2Seq)-based deep learning models (such as RNN, LSTM, Transformer, etc.), etc., and the embodiments of the present application do not impose specific limitations on this.
[0057] In the above step S103, the text generation model needs to encode the input data. The original input text can be encoded to obtain the first encoding feature. For example, the model will encode each word (or character, subword, etc., depending on the specific encoding method) in the original input text and convert it into a vector representation in a high-dimensional space. These vectors capture the semantics, grammar and other information of the words, so that the model can understand and process these texts. The vector set (or matrix) obtained after encoding is the first encoding feature, which reflects the numerical representation of the original input text and is the basis for the model to generate text later.
[0058] Similar to the original input text, structured text also needs to be encoded into a numerical form that the model can understand. Similarly, the model will encode each word in the structured text and convert it into a vector representation in a high-dimensional space. The numerical representation of the structured text obtained after encoding is the second encoding feature, which reflects the numerical representation of the precise data in the structured text and is an important factor that the model needs to consider when generating text.
[0059] In the above-mentioned step S104, the attention mechanism can help the model to pay more attention to the key parts in the input text when generating text, especially the parts closely related to the current generation task, and by assigning different attention weights to different input parts, the model can more accurately capture and express these key information. The second coding feature is a numerical representation obtained by encoding the structured text, and because the structured text contains accurate data and key information, the second coding feature is very important for the text generation process. Based on this, in the embodiment of the present application, the attention mechanism constructed based on the second coding feature can be utilized to perform feature fusion processing on the first coding feature and the second coding feature to obtain a fusion feature.
[0060] Among them, the attention mechanism built based on the second encoded feature means that the model will adjust the attention to the first encoded feature (i.e. the encoded feature of the original input text) according to the information in the second encoded feature. Specifically, the model will learn an attention weight allocation mechanism that can identify the key data or important information in the second encoded feature and increase the attention to the relevant part of the first encoded feature accordingly.
[0061] In addition, a bias term can be added to the attention weight matrix to adjust the degree of attention to the second encoding feature, so that the generated text can learn more about the distribution of structured data, so that it can be more accurately reflected in the output text, thereby playing the role of generating controllable text.
[0062] Next, the first encoding feature and the second encoding feature are fused, and the attention mechanism is used to enhance the attention on the relevant parts of the structured text, thereby obtaining a fused feature that combines the original context information and precise data information. The fused feature contains rich context information and highlights the key data in the structured text, and can therefore more accurately reflect the true intent or meaning of the original input text.
[0063] In the above step S105, the target generated text is the generated text output by the text generation model after generating text based on the fusion features. The target generated text can accurately reflect the true intention or meaning of the original input text, especially accurately express the key information in the structured data. In the application of financial technology, the target generated text can clearly convey the results of financial report analysis, the conclusions of credit assessment or the suggestions of investment strategy, etc., while maintaining the accuracy and consistency with the original input data.
[0064] In summary, through the above-mentioned steps S101 to S105, the embodiment of the present application obtains the initial text, and the structured text corresponding to the key data in the initial text, splices the initial text with the structured text, and obtains the original input text, which can provide a comprehensive information source that contains both rich context and accurate data for the text generation model. Subsequently, the original input text and the structured text are input into the text generation model, the original input text is encoded, the first coding feature is obtained, and the structured text is encoded to obtain the second coding feature, and the attention mechanism constructed based on the second coding feature is used to perform feature fusion processing on the first coding feature and the second coding feature, and the attention of the structured text in the initial text is strengthened, so that the content of the original input text can be accurately reflected. Finally, the fusion feature is obtained by fusion, and then text generation is performed based on the fusion feature, so that the target generated text finally generated can be close to the true intention or meaning of the original input text. Therefore, the embodiment of the present application improves the controllability when outputting the text, thereby improving the quality of text generation.
[0065] In addition to being applied in the field of financial technology, the embodiments of the present application can also be applied to a wider range of scenarios, so that the text generation model can also be widely used in other fields of structured data text generation, thereby improving the efficiency of text writing through structured data such as financial reports and industry research reports. At the same time, the embodiments of the present application can also have strong scalability, and the text generation model can be adjusted according to different tasks and needs, thereby serving as a reference for the generation of controllable text in other fields, and the embodiments of the present application do not make specific restrictions on this.
[0066] See also Figure 2 In some embodiments, step S101 may include but is not limited to steps S201 to S203:
[0067] Step S201, obtaining an initial text, wherein the initial text contains key data;
[0068] Step S202, extracting key data from the initial text to obtain data content of the key data under the corresponding data type;
[0069] Step S203: output structured text based on the data type and the corresponding data content.
[0070] In the above steps, the structured text is obtained after the initial text is extracted. Specifically, the initial text contains key data, which can be divided into different data types, and the key data under the corresponding data type has related data content. Among them, the data type refers to the type of data item or data element, which defines the values that the data can contain and how these values are stored and represented. In the field of financial technology, the data type can be very diverse, including but not limited to numbers (such as amount, percentage, year), text (such as company name, name, address), date / time, enumeration type (such as credit rating, industry classification), etc. Data content refers to specific data values or information. These data values are defined and represented according to the data type. When extracting key data in the initial text, the specific value or information corresponding to the key data, that is, the data content, will be obtained.
[0071] After obtaining the initial text, the embodiment of the present application needs to extract key data from the text, obtain the data content of the key data under the corresponding data type, and need to express these data and their data types in a structured form. Structured text has a clear format and field, which can clearly express the meaning and relationship of the data. Based on this, the unstructured initial text can be converted into structured data, providing an accurate data source for subsequent text generation.
[0072] The following is an example of the process of extracting key data from the initial text and finally obtaining structured text. Figure 3 , Figure 3 It is a schematic diagram of extracting the structured text from the initial text provided by the embodiment of the present application. Among them, taking the initial text as a report text of a financial institution as an example, the initial text includes "Company Name: Bank", "Securities Code (001)" and "The growth rate of scale remains stable. In the first half of 2020, the growth rate of a bank's total assets fell slightly by 0.1% month-on-month, maintaining stability, and the asset-liability structure improved. On the asset side, loans increased by 2.8% month-on-month, and the number of businesses increased steadily. On the liability side, deposits increased by 3.1% month-on-month, and the capital acquisition situation was good. The interest rate spread narrowed as expected. The interest rate spread fell by 5% month-on-month in the first half of 2020, which was in line with expectations. On the asset side, the yield decreased by 0.13% month-on-month, and the interest rates of various assets all fell by 0.02% compared with 2019, and the impact was controllable. On the liability side, the cost rate decreased by 0.08% month-on-month. Among them, the deposit cost decreased by 0.04% compared with 2019, and the current situation of deposit costs is expected to improve."
[0073] In this initial text, the data types of relevant key data include company name, stock code, time, total asset growth rate month-on-month, loan month-on-month, deposit month-on-month, interest rate spread month-on-month, yield month-on-month, asset interest rate difference, cost rate month-on-month, and deposit cost difference rate. The corresponding data content includes the company name is a certain company, the stock code is 001, the time is the first half of 2020, the total asset growth rate is 0.1% month-on-month, the loan is +2.8% month-on-month, the deposit is +3.1% month-on-month, the interest rate spread is -5% month-on-month, the yield is -0.13% month-on-month, the asset interest rate difference is -0.2%, the cost rate is -0.08% month-on-month, and the deposit cost difference rate is -0.04%. After completing the extraction of key data, a structured text as shown in the figure can be obtained, in which the data type of key data and the corresponding data content are embodied and stored in a structured form.
[0074] Exemplarily, there are many ways to extract key data from the initial text. For example, you can write a specific regular expression to match and extract the character string that meets the specific pattern in the initial text to obtain the required key data; or you can use a graph model and PageRank algorithm to identify the key data in the initial text; or you can use a model trained based on machine learning to extract the key data in the initial text.
[0075] Further, the present application embodiment discloses a method for extracting key data through an information extraction model. In some embodiments, step S202 may include but is not limited to the following steps:
[0076] Input the initial text into the preset information extraction model to extract key data and obtain the data content of the key data under the corresponding data type;
[0077] Among them, the information extraction model is jointly trained with the text generation model.
[0078] In the above steps, the information extraction model is a machine learning or deep learning model, which is specifically used to automatically identify and extract structured information from unstructured or semi-structured text data. This information can be entities (such as names of people, places, names of institutions, etc.), relationships (such as relationships between characters, causal relationships between events, etc.), attributes (such as a person's age, gender, occupation, etc.) or any other data useful for a specific task. In the application scenarios in the field of financial technology, the role of information extraction models is particularly important, because the field of financial technology involves a large number of documents, such as financial reports, credit assessment reports, etc. These documents contain a large amount of key structured data, such as amounts, dates, ratios, etc., which are crucial for subsequent tasks such as text generation and decision analysis. Furthermore, the information extraction model can be constructed based on a variety of algorithms, including but not limited to rule matching, statistical learning, deep learning, etc., and this embodiment of the present application does not make specific restrictions on this.
[0079] When the initial text is input into the information extraction model, the information extraction model will traverse the text content and use the knowledge and skills it has learned during the training process to identify the key data in the text, thereby obtaining the data content of the key data under the corresponding data type. For example, the information extraction model will identify the amount as a financial data type, the date as a time data type, and the name as an entity name type. The obtained data content is finally output in a structured form (such as key-value pairs, lists, tables, etc.) to facilitate subsequent text generation or data analysis tasks.
[0080] Furthermore, the information extraction model is jointly trained with the text generation model, that is, the two models influence each other during the training process. Through joint training, the information extraction model can more accurately identify key data useful for text generation, while the text generation model can generate higher quality text based on these accurate structured data. Therefore, the joint training strategy helps to improve the performance and effect of the entire system.
[0081] See also Figure 4 In some embodiments, the process of inputting the original input text and the structured text into the text generation model in step S102 may include but is not limited to steps S301 to S302:
[0082] Step S301, constructing a tag key-value pair using the data type corresponding to the key data in the structured text as the key and the data content under the corresponding data type in the structured text as the value;
[0083] Step S302, inputting the original input text and the tag key-value pair into the text generation model;
[0084] In some embodiments, the process of encoding the structured text to obtain the second encoding feature in step S103 may include but is not limited to the following steps:
[0085] The tag key-value pair is encoded to obtain a second encoding feature.
[0086] In the above steps, for each key data, its data type needs to be determined, and then the data type is used as the key (Key) and the corresponding data content is used as the value (Value) to build a tag key-value pair. In this process, each key data will be converted into a key-value pair containing the data type and data content. For example, for the amount "10,000 yuan", a key-value pair can be built, where the key is "amount" and the value is "10,000 yuan".
[0087] Finally, different label key-value pairs can be constructed based on the key data under different data types. These key-value pairs contain the data types and data contents of all key data in the structured text. Subsequently, the label key-value pairs will be input into the text generation model as part of the model input together with the original input text.
[0088] It should be noted that in the embodiments of the present application, by converting structured data into label key-value pairs, the model can more easily understand and utilize these data, wherein the data type serves as a key, providing the model with additional information about the nature of the data, which helps the model to more accurately reflect the meaning and importance of these data when generating text. In addition, the form of label key-value pairs enables the model to more accurately control the use of structured data when generating text. The model can generate text related to the data type and data content in the key-value pairs, thereby improving the controllability and accuracy of text generation. Ultimately, the text generation model can generate text that contains rich context and accurately reflects structured data, thereby improving the overall quality of text generation.
[0089] In addition, each key data in the structured text (converted into its corresponding embedding vector) can be input into the text generation model together with the original input text. It should be noted that the embedding vector can be obtained through a pre-trained embedding model (such as Word2Vec, G loVe or BERT, etc.), or customized training for data in a specific field, and this embodiment of the present application does not make specific restrictions on this.
[0090] See also Figure 5 In some embodiments, the process of concatenating the initial text with the structured text to obtain the original input text in step S102 may include but is not limited to steps S401 to S402:
[0091] Step S401, serializing the structured text to obtain a serialized text;
[0092] Step S402, concatenating the initial text with the serialized text to obtain the original input text.
[0093] In the above steps, the structured text contains a series of data organized in a specific format or structure, however, the expected input of the text generation model is in a linear, serialized form, such as a string of words or characters. Therefore, in the embodiment of the present application, it is necessary to perform serialization processing on the structured text, flatten the nested structure in the structured text, that is, convert all related data items into a one-dimensional sequence, thereby converting each data item into a text form, forming a long string, which is the serialized text.
[0094] See also Figure 6 , Figure 6 It is a schematic diagram of the text generation process provided by the embodiment of the present application. After obtaining the serialized text, the initial text needs to be combined with the serialized text to form a comprehensive information source that contains both natural language descriptions and precise data, that is, the original input text, so that in the text generation process, the model can consider both contextual information and precise data, thereby generating more accurate and meaningful text.
[0095] In some embodiments, Figure 6 As shown, after the original input text is input into the text generation model, the text generation model encodes the original input text to obtain the first encoding feature. In addition, the label key-value pair is input into the text generation model, and the text generation model encodes the label key-value pair to obtain the second encoding feature. Then, the first encoding feature and the second encoding feature are semantically fused through the label attention mechanism to enhance the attention of the label in the text, and finally generate accurate target generated text.
[0096] For example, the obtained financial report is used as the label text with the initial text, and Figure 3 The structured text in is used as an example to illustrate. Figure 3 The structured text extracted from the data is serialized into a format such as "The company name is a certain bank, and the stock code is 001." The serialized text is then concatenated with the initial text to form a format such as "The company name is a certain bank, and the stock code is 001...The growth rate of a certain bank's total assets in the first half of 2020 was month-on-month...The current situation of deposit costs is expected to improve." After concatenation, the original input text is formed.
[0097] It should be noted that the splicing process can splice the serialized text with the original text. In addition, the serialized text can also be inserted into the corresponding position in the original text, such as near the position of the corresponding key data, but a corresponding mark needs to be added to indicate that it is inserted so as not to affect the coherence of the original text.
[0098] See also Figure 7 , Figure 7 : is a structural diagram of a text generation model provided by an embodiment of the present application. In some embodiments, the text generation model includes an encoder, a decoder, and a label attention layer. The encoder is used to encode the original input text and the structured text, including encoding the original input text to obtain a first encoding feature, and encoding the label key-value pair to obtain a second encoding feature, and then the encoder inputs the first encoding feature and the second encoding feature into the label attention layer. Among them, the label attention layer is a label attention mechanism constructed based on the second encoding feature.
[0099] See also Figure 8 In some embodiments, step S105 may include but is not limited to steps S501 to S502:
[0100] Step S501, inputting the fused features into a decoder, decoding the fused features, and obtaining decoded features;
[0101] Step S502: generate text based on the decoding features to obtain a target generated text.
[0102] In the above steps, the fused features are input into the decoder, which is an important component in the text generation model. Its main function is to convert the encoded features (i.e., fused features) into a text form that is easy to understand and generate. It should be noted that the decoder receives the current input (usually a special starting symbol in the first step, followed by the embedded representation of the word or character generated in the previous step) and the previous hidden state (containing all information up to the current step), and then calculates a new hidden state and output state. In an embodiment of the present application, the decoder decodes the fused features and converts them into decoded features through a series of transformations and calculations. The decoded features contain all the information required to generate the target text and are the basis for the subsequent text generation.
[0103] After obtaining the decoding features, the next step is to generate text based on these features to obtain the final target generated text. Exemplarily, at each time step, the decoder will calculate a vocabulary distribution based on the current decoding features and the previous output, indicating the probability of generating each possible vocabulary at the current step. According to the vocabulary distribution, the vocabulary with the highest probability can be selected as the output of the current step, or sampling can be performed according to the vocabulary distribution (such as sampling using a softmax function) to introduce a certain degree of randomness, and the current hidden state of the output of the current step is used together as the input of the next time step to update the hidden state of the decoder for the next step of text generation. Finally, the above process is repeated until the preset stop condition is reached, and all the generated words are connected in order to obtain the final target generated text, thereby realizing the conversion process from structured data and initial text to target generated text.
[0104] Below, we introduce the training process of the text generation method:
[0105] See also Fig. 9 In some embodiments, the text generation method further includes but is not limited to steps S601 to S605:
[0106] Step S601, obtaining an initial sample text and a sample structured text corresponding to key data in the initial sample text;
[0107] Step S602, concatenating the initial sample text with the sample structured text to obtain a sample input text, and inputting the sample input text and the sample structured text into an initial text generation model;
[0108] Step S603, encoding the sample input text to obtain a first sample encoding feature, and encoding the sample structured text to obtain a second sample encoding feature;
[0109] Step S604, using an attention mechanism constructed based on the second sample coding feature, performing feature fusion processing on the first sample coding feature and the second sample coding feature to obtain a sample fusion feature;
[0110] Step S605, generating text based on the sample fusion features to obtain sample generated text, and training an initial text generation model based on the sample generated text to obtain a trained text generation model.
[0111] In the above steps, the initial sample text is the basic text required for text generation during the training process. Similar to the initial text in the application process, the initial sample text is also a text that provides rich contextual information, which will not be described in detail in the embodiments of the present application.
[0112] Exemplarily, the initial sample text may also be a financial report text, in which the key data are specific financial-related data, indicators or parameters. For example, like the initial text, the initial sample text may also be a report text of a financial institution, and the corresponding key data is explained similarly to the above embodiment, which is not described in detail in the present application embodiment.
[0113] Exemplarily, the initial sample text may be obtained from various channels, such as news reports, market analysis, company announcements or related websites, etc. The initial sample text may also be obtained by user input, which is not specifically limited here.
[0114] The sample structured text refers to data corresponding to the key data in the initial sample text and presented in a structured form (such as a table, a key-value pair, a database record, etc.), that is, data presenting the key data in a structured form. These data are specific values, proportions, percentages, etc. mentioned in the initial sample text. They are extracted and represented in a form that is easier for computers to process. Exemplarily, the sample structured text can be directly extracted from the initial sample text, or it can be obtained by the user selecting or inputting the corresponding key data based on the initial sample text, and no specific restrictions are made here.
[0115] After obtaining the initial sample text and the sample structured text, the initial sample text and the sample structured text need to be spliced. The splicing process is similar to the splicing of the initial text and the structured text in the above embodiment, and will not be repeated here.
[0116] The sample input text is the text obtained after splicing. Therefore, the sample input text is both an extension of the initial sample text and contains the precise data in the sample structured text. This comprehensive information source provides a richer and more comprehensive input for the text generation model in the training process, allowing the subsequent model to consider both contextual information and precise data when generating text, thereby generating more accurate and meaningful text.
[0117] The initial text generation model is an untrained or incompletely trained text generation model. During the training process, the initial text generation model also needs to encode the input data. Among them, the sample input text can be encoded to obtain the first sample encoding feature. For example, the model will encode each word (or character, subword, etc., depending on the specific encoding method) in the sample input text and convert it into a vector representation in a high-dimensional space. These vectors capture the semantics, grammar and other information of the words, so that the model can understand and process these texts. The vector set (or matrix) obtained after encoding is the first sample encoding feature, which reflects the numerical representation of the sample input text and is the basis for the model to generate text later.
[0118] Similar to the sample input text, the sample structured text also needs to be encoded into a numerical form that the model can understand. Similarly, the model will encode each word in the sample structured text and convert it into a vector representation in a high-dimensional space. The numerical representation of the sample structured text obtained after encoding is the second sample encoding feature, which reflects the numerical representation of the precise data in the sample structured text and is an important factor that the model needs to consider when generating text.
[0119] Similar to the application process of the above embodiment, in the embodiment of the present application, an attention mechanism constructed based on the second sample coding feature can also be used to perform feature fusion processing on the first sample coding feature and the second sample coding feature to obtain a sample fusion feature.
[0120] Among them, the attention mechanism built based on the second sample encoding feature means that the model will adjust the attention to the first sample encoding feature (i.e. the encoding feature of the sample input text) according to the information in the second sample encoding feature. Specifically, the model will learn an attention weight allocation mechanism that can identify the key data or important information in the second sample encoding feature and increase the attention to the relevant part of the first sample encoding feature accordingly.
[0121] In addition, a bias term can be added to the attention weight matrix to adjust the degree of attention to the encoding features of the second sample, so that the generated text can learn more about the distribution of structured data, which can be more accurately reflected in the output text, thereby playing the role of generating controllable text.
[0122] Next, feature fusion processing is performed on the first sample coding feature and the second sample coding feature to obtain a sample fusion feature similar to the fusion feature in the above embodiment, and after text generation is performed based on the fusion feature, the sample generated text is output.
[0123] Finally, based on the sample generated text, the back propagation algorithm and optimization techniques such as gradient descent are used to update the parameters of the initial text generation model, so that the model can better reflect the true intention or meaning of the original input when generating text, and improve the quality of the generated text. Specifically, the calculation of the loss function can be introduced in the training process, and the loss value can be minimized. In each iteration, new sample inputs are used to generate sample generated text, and these samples are used to train or optimize the model. As the iterations proceed, the model will gradually learn more knowledge and patterns, thereby improving its generation capabilities under various input conditions. After multiple iterations of training, a trained text generation model will eventually be obtained, which can better meet the text generation needs in specific scenarios.
[0124] See also Fig.10The present application also provides a text generation device that can implement the above-mentioned text generation method. The device includes:
[0125] The text acquisition module 1001 is used to acquire the initial text and the structured text corresponding to the key data in the initial text;
[0126] A concatenation module 1002 is used to concatenate the initial text with the structured text to obtain an original input text, and input the original input text and the structured text into a text generation model;
[0127] The encoding module 1003 is used to encode the original input text to obtain a first encoding feature, and to encode the structured text to obtain a second encoding feature;
[0128] A fusion module 1004 is used to perform feature fusion processing on the first coding feature and the second coding feature by using an attention mechanism constructed based on the second coding feature to obtain a fusion feature;
[0129] The generation module 1005 is used to generate text based on the fusion features to obtain the target generated text.
[0130] The text generation device can realize the text generation method in the above-mentioned embodiment, by obtaining the initial text, and the structured text corresponding to the key data in the initial text, splicing the initial text with the structured text, and obtaining the original input text, which can provide a comprehensive information source that contains both rich context and accurate data for the text generation model. Subsequently, the original input text and the structured text are input into the text generation model, the original input text is encoded, the first coding feature is obtained, and the structured text is encoded to obtain the second coding feature, and the attention mechanism constructed based on the second coding feature is used to perform feature fusion processing on the first coding feature and the second coding feature, and the attention of the structured text in the initial text is strengthened, so that the content of the original input text can be accurately reflected. Finally, the fusion feature is obtained by fusion, and then text generation is performed based on the fusion feature, so that the target generated text finally generated can be close to the true intention or meaning of the original input text. Therefore, the embodiment of the present application improves the controllability when outputting the text, thereby improving the quality of text generation.
[0131] The specific implementation of the text generation device is basically the same as the specific implementation of the above text generation method, and will not be repeated here. On the premise of meeting the requirements of the embodiment of this application, the text generation device can also be provided with other functional modules to implement the text generation method in the above embodiment.
[0132] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above text generation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.
[0133] See also Fig.11 , Fig.11 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0134] The processor 1101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0135] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the text generation method of the embodiment of this application;
[0136] Input / output interface 1103, used to implement information input and output;
[0137] The communication interface 1104 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);
[0138] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);
[0139] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .
[0140] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned text generation method is implemented.
[0141] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0142] The text generation method and device, electronic device and storage medium provided by the embodiment of the present application, by obtaining the initial text, and the structured text corresponding to the key data in the initial text, the initial text and the structured text are spliced to obtain the original input text, which can provide a comprehensive information source containing both rich context and accurate data for the text generation model. Subsequently, the original input text and the structured text are input into the text generation model, the original input text is encoded to obtain the first encoding feature, and the structured text is encoded to obtain the second encoding feature, and the attention mechanism constructed based on the second encoding feature is used to perform feature fusion processing on the first encoding feature and the second encoding feature, so as to strengthen the attention of the structured text in the initial text, so as to accurately reflect the content of the original input text. Finally, the fusion feature is obtained by fusion, and then the text is generated based on the fusion feature, so that the target generated text finally generated can be close to the true intention or meaning of the original input text. Therefore, the embodiment of the present application improves the controllability when outputting the text, thereby improving the quality of text generation.
[0143] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0144] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0145] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0146] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0147] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0148] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0149] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0150] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0151] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0152] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0153] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A text generation method, characterized in that: include: Acquire an initial text and a structured text corresponding to key data in the initial text; The initial text is concatenated with the structured text to obtain an original input text, and the original input text and the structured text are input into a text generation model; Encoding the original input text to obtain a first encoding feature, and encoding the structured text to obtain a second encoding feature; Using an attention mechanism constructed based on the second coding feature, performing feature fusion processing on the first coding feature and the second coding feature to obtain a fused feature; Text generation is performed based on the fusion features to obtain a target generated text.
2. The text generation method according to claim 1, characterized in that: The obtaining of the initial text and the structured text corresponding to the key data in the initial text includes: Acquire an initial text, wherein the initial text contains key data; Extracting the key data in the initial text to obtain data content of the key data under the corresponding data type; Output structured text based on the data type and the corresponding data content.
3. The text generation method according to claim 2, characterized in that: The extracting the key data from the initial text to obtain the data content of the key data under the corresponding data type includes: Input the initial text into a preset information extraction model to extract the key data, and obtain the data content of the key data under the corresponding data type; The information extraction model is obtained by joint training with the text generation model.
4. The text generation method according to claim 1 or 2, characterized in that: The step of inputting the original input text and the structured text into a text generation model comprises: Using the data type corresponding to the key data in the structured text as a key, and using the data content corresponding to the data type in the structured text as a value, to construct a tag key-value pair; Inputting the original input text and the tag key-value pair into a text generation model; The encoding of the structured text to obtain a second encoding feature includes: The tag key-value pair is encoded to obtain a second encoding feature.
5. The text generation method according to claim 1, characterized in that: The step of concatenating the initial text with the structured text to obtain the original input text includes: Serializing the structured text to obtain a serialized text; The initial text is concatenated with the serialized text to obtain the original input text.
6. The text generation method according to claim 1, characterized in that: The text generation model includes an encoder and a decoder, wherein the encoder is used to encode the original input text and the structured text; The step of generating text based on the fusion features to obtain a target generated text includes: Inputting the fused features into the decoder, decoding the fused features to obtain decoded features; Text generation is performed based on the decoding features to obtain a target generated text.
7. The text generation method according to claim 1, characterized in that: The text generation method also includes: Acquire an initial sample text and a sample structured text corresponding to key data in the initial sample text; The initial sample text is concatenated with the sample structured text to obtain a sample input text, and the sample input text and the sample structured text are input into an initial text generation model; Encoding the sample input text to obtain a first sample encoding feature, and encoding the sample structured text to obtain a second sample encoding feature; Using an attention mechanism constructed based on the second sample encoding feature, feature fusion processing is performed on the first sample encoding feature and the second sample encoding feature to obtain a sample fusion feature; Text generation is performed based on the sample fusion features to obtain sample generated text, and the initial text generation model is trained based on the sample generated text to obtain the trained text generation model.
8. A text generation device, characterized in that: include: A text acquisition module, used to acquire an initial text and a structured text corresponding to key data in the initial text; A concatenation module, used for concatenating the initial text with the structured text to obtain an original input text, and inputting the original input text and the structured text into a text generation model; An encoding module, used for encoding the original input text to obtain a first encoding feature, and encoding the structured text to obtain a second encoding feature; A fusion module, configured to perform feature fusion processing on the first coding feature and the second coding feature by using an attention mechanism constructed based on the second coding feature to obtain a fusion feature; A generation module is used to generate text based on the fusion features to obtain a target generated text.
9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the text generation method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text generation method according to any one of claims 1 to 7 is implemented.
