Text generation method and device based on diffusion strategy, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a text generation method based on a diffusion strategy, and the method comprises the steps: defining a target sequence extension length of an initial diffusion model through obtaining a training text data set, and determining a total diffusion time step length; in the generation process, a sentence-level time step length is selected in a preset range, a starting point of a generation sequence is calculated, and a corresponding token-level time step length is distributed for each token. The generation sequence and step length of each token can be dynamically adjusted through a time step length model so as to adapt to the sequence dependence characteristic of a natural language. The model defines and optimizes a loss function by calculating the generation probability of each token, and updates model parameters through optimization methods such as gradient descent to obtain a target diffusion model. According to the method, the input data can be accepted, and the target text sequence conforming to the context logic is generated, so that the problem of sequence dependence in the text generation process is effectively solved, and the logic continuity and semantic reasonability of the generated content are ensured.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a text generation method, device, equipment and storage medium based on a diffusion strategy. Background Art

[0002] In the financial sector, with the widespread application of big data, artificial intelligence and cloud computing technologies, financial institutions are increasingly relying on intelligent technologies to improve operational efficiency and service quality. Intelligent text generation technology has gradually been integrated into all aspects of financial services, such as customer service, market analysis, investment advice, etc., providing strong technical support for financial institutions.

[0003] Financial institutions receive a large number of customer inquiries every day, involving account information, transaction details, product recommendations, and other aspects. Through intelligent text generation technology, financial institutions can automatically generate personalized customer responses, greatly improving service efficiency. With natural language processing technology, the system can understand the semantics of customer questions and generate accurate and personalized answers based on a predefined knowledge base.

[0004] Financial institutions need to frequently generate market analysis reports to guide business decisions and customer investments. Traditional report writing methods usually require a lot of manpower and are time-consuming. Using a text generation model based on a diffusion strategy, the latest market data can be automatically extracted from financial databases and structured analysis reports can be generated. This automatically generated report can cover multiple dimensions such as market trends and product performance, helping financial analysts quickly complete report writing tasks.

[0005] In the field of investment consulting, intelligent text generation technology is also playing an increasingly important role. By analyzing the customer's historical investment behavior, market conditions and the latest financial product information, the intelligent system can automatically generate personalized investment advice. These suggestions are usually based on the customer's risk preference, expected returns and investment goals, and are presented to the customer in a clear and understandable way through text generation technology, so that the customer can make better investment decisions.

[0006] Financial products often have complex terms and conditions and need to be frequently updated based on market changes. With intelligent text generation technology, financial product description documents can be automatically written and updated. This process not only improves work efficiency, but also reduces human errors and ensures the accuracy and timeliness of financial product information. Intelligently generated product descriptions can quickly respond to market changes and provide customers with the latest product information.

[0007] Although intelligent text generation is widely used in the financial field, it still faces some challenges. First, the professional terminology and complex context in the financial field require that the generated text must not only be accurate, but also conform to the standards and norms of the financial industry. Second, how to effectively integrate and process a large amount of financial data so that the generated text is both professional and easy to understand is also an important issue that needs to be solved. Summary of the invention

[0008] The main purpose of the present invention is to provide a text generation method, device, equipment and storage medium based on a diffusion strategy, aiming to solve the technical problem of how to process the sequential dependency of natural language in a text sequence generated by a diffusion model.

[0009] To achieve the above object, the present invention provides a text generation method based on a diffusion strategy, comprising:

[0010] Obtain a training text dataset, define the target sequence extension length of the initial diffusion model, and determine the total diffusion time step of the initial diffusion model;

[0011] Selecting a sentence-level time step within a preset range, calculating a starting point of a generated sequence according to the sentence-level time step, and assigning a token-level time step to each token in the training text dataset;

[0012] Calculating the generation probability of each token by using the initial diffusion model and the token-level time step, defining a loss function by the generation probability, and updating the parameters of the initial diffusion model by using the loss function to obtain a target diffusion model;

[0013] Input data is acquired, and the input data is input into the target diffusion model to obtain a target text sequence.

[0014] In one embodiment, the input data is input into the target diffusion model to obtain a target text sequence, including:

[0015] Inputting the input data into the target diffusion model, and using the input data as a starting point for generating a target text sequence;

[0016] Determining a decreasing time step sequence based on the target sequence extension length and the total diffusion time step;

[0017] Sampling an initial latent variable from a standard Gaussian distribution, and initializing the initial latent variable to obtain a current latent variable;

[0018] The target text sequence is generated according to the generation starting point of the target text sequence, the current latent variable and the decreasing time step sequence.

[0019] In one embodiment, the input data is input into the target diffusion model to obtain a target text sequence, including:

[0020] Determine the initial generation stage, the middle generation stage and the late generation stage of the target diffusion model in the generation process according to the total diffusion time step and the preset diffusion ratio;

[0021] In the initial generation stage, text sequence generation is performed based on the generation strategy of preset grammar rules;

[0022] In the mid-term generation stage, text sequence generation is performed based on the generation strategy of context-dependent rules and intermediate semantic verification rules;

[0023] In the later generation stage, the text sequence generated in the early generation stage and the mid-generation stage is corrected based on the generation strategy of grammar and spelling check rules to obtain the target text sequence.

[0024] In one embodiment, assigning a token-level time step to each token in the training text dataset comprises:

[0025] Use a pre-trained language model to predict the importance of each token in context;

[0026] Count the word frequency of each token in the training text dataset, and record the position information of each token in the sentence;

[0027] The token weight of each token is determined according to the importance, word frequency and position information of each token, and the token-level time step of each token is determined according to the token weight of each token and the sentence-level time step.

[0028] In one embodiment, calculating the starting point of the generated sequence according to the sentence-level time step includes:

[0029] Subtract the sentence-level time step value from the target sequence length to obtain a starting point difference, and determine whether the starting point difference is less than zero;

[0030] If the starting point difference is not less than zero, the starting point difference is directly used as the starting point position of the generated sequence;

[0031] If the starting point difference is less than zero, the starting point difference is assigned zero and used as the starting point position of the generated sequence.

[0032] In one embodiment, a loss function is defined by generating probabilities, and the parameters of the initial diffusion model are updated by the loss function to obtain a target diffusion model, including:

[0033] Determining the task requirements of the initial diffusion model, and determining a plurality of preset loss functions and the weight of each loss function according to the task requirements;

[0034] A total loss function is defined by the weights of each loss function and the generation probability;

[0035] The generation probability is calculated and the loss is accumulated through the initial diffusion model, the partial derivative of the loss function with respect to each parameter of the initial diffusion model is calculated through the chain rule, and the parameters of the initial diffusion model are updated using the gradient descent algorithm to obtain the target diffusion model.

[0036] In one embodiment, defining a target sequence extension length of an initial diffusion model includes:

[0037] Dividing the training text data set into multiple text sequences, and counting the actual length of each text sequence;

[0038] Calculate the average length, maximum length, minimum length and standard deviation of the text sequences in the training text data set according to the actual length of each text sequence, and determine the actual length of the target sequence in combination with the text generation task requirements;

[0039] Set the target sequence extension length to a preset multiple of the actual length of the target sequence.

[0040] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a text generation device based on a diffusion strategy, comprising:

[0041] The model initialization module is used to obtain the training text dataset, define the target sequence extension length of the initial diffusion model, and determine the total diffusion time step of the initial diffusion model;

[0042] Generate a starting point determination module, which is used to select a sentence-level time step within a preset range, calculate the starting point of the generated sequence according to the sentence-level time step, and assign a token-level time step to each token in the training text dataset;

[0043] A model training and parameter optimization module, used for calculating the generation probability of each token through the initial diffusion model and the token-level time step, defining a loss function through the generation probability, updating the parameters of the initial diffusion model through the loss function, and obtaining a target diffusion model;

[0044] The text generation module is used to obtain input data, input the input data into the target diffusion model, and obtain a target text sequence.

[0045] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a text generation device based on a diffusion strategy, wherein the text generation device based on a diffusion strategy comprises a memory, a processor, and a text generation program based on a diffusion strategy stored in the memory and executable on the processor, wherein the text generation program based on a diffusion strategy implements the steps of the text generation method based on a diffusion strategy as described above when executed by the processor.

[0046] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer storage medium, on which a text generation program based on a diffusion strategy is stored, and when the text generation program based on a diffusion strategy is executed by a processor, the steps of the text generation method based on a diffusion strategy as described above are implemented.

[0047] Beneficial effects: The present invention relates to a text generation method based on a diffusion strategy, which obtains a training text data set, defines the target sequence extension length of the initial diffusion model, and determines the total diffusion time step. During the generation process, the sentence-level time step is first selected within a preset range, the starting point of the generation sequence is calculated, and a corresponding token-level time step is assigned to each token. Through these time steps, the model can dynamically adjust the generation order and step size of each token to adapt to the sequential dependency characteristics of natural language. On this basis, the model calculates the generation probability of each token, defines and optimizes the loss function, and finally updates the model parameters through optimization methods such as gradient descent to obtain a target diffusion model. The present invention can accept input data and generate a target text sequence that conforms to the context logic, thereby effectively solving the problem of sequential dependency in the text generation process and ensuring the logical coherence and semantic rationality of the generated content. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0049] Figure 1 It is a flow chart of an embodiment of a text generation method based on a diffusion strategy of the present invention;

[0050] Figure 2 It is a functional module diagram of a preferred embodiment of a text generation device based on a diffusion strategy of the present invention;

[0051] Figure 3 It is a structural schematic diagram of the device hardware operating environment involved in the embodiment scheme of the text generation device based on the diffusion strategy of the present invention. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0053] It should be noted that in the financial field, with the widespread application of big data, artificial intelligence and cloud computing technologies, financial institutions are increasingly relying on intelligent technologies to improve operational efficiency and service quality. Intelligent text generation technology has gradually been integrated into various aspects of financial services, such as customer service, market analysis, investment advice, etc., providing strong technical support for financial institutions.

[0054] Financial institutions receive a large number of customer inquiries every day, involving account information, transaction details, product recommendations, and other aspects. Through intelligent text generation technology, financial institutions can automatically generate personalized customer responses, greatly improving service efficiency. With natural language processing technology, the system can understand the semantics of customer questions and generate accurate and personalized answers based on a predefined knowledge base.

[0055] Financial institutions need to frequently generate market analysis reports to guide business decisions and customer investments. Traditional report writing methods usually require a lot of manpower and are time-consuming. Using a text generation model based on a diffusion strategy, the latest market data can be automatically extracted from financial databases and structured analysis reports can be generated. This automatically generated report can cover multiple dimensions such as market trends and product performance, helping financial analysts quickly complete report writing tasks.

[0056] In the field of investment consulting, intelligent text generation technology is also playing an increasingly important role. By analyzing the customer's historical investment behavior, market conditions and the latest financial product information, the intelligent system can automatically generate personalized investment advice. These suggestions are usually based on the customer's risk preference, expected returns and investment goals, and are presented to the customer in a clear and understandable way through text generation technology, so that the customer can make better investment decisions.

[0057] Financial products often have complex terms and conditions and need to be frequently updated based on market changes. With intelligent text generation technology, financial product description documents can be automatically written and updated. This process not only improves work efficiency, but also reduces human errors and ensures the accuracy and timeliness of financial product information. Intelligently generated product descriptions can quickly respond to market changes and provide customers with the latest product information.

[0058] Although intelligent text generation is widely used in the financial field, it still faces some challenges. First, the professional terminology and complex context in the financial field require that the generated text must not only be accurate, but also conform to the standards and norms of the financial industry. Second, how to effectively integrate and process a large amount of financial data so that the generated text is both professional and easy to understand is also an important issue that needs to be solved.

[0059] See also Figure 1 , Figure 1This is a flow chart of an embodiment of a method for text generation based on a diffusion strategy provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0060] like Figure 1 As shown, the text generation method based on the diffusion strategy proposed in the present invention includes the following steps:

[0061] S10, obtaining a training text data set, defining a target sequence extension length of an initial diffusion model, and determining a total diffusion time step of the initial diffusion model;

[0062] In this embodiment, a large amount of text data is obtained from various sources as the basis for training the machine learning model. These data sets can include texts in various forms from structured to unstructured, such as articles, reports, conversation records, documents, etc. The quality and diversity of the training data set are crucial to the performance of the model.

[0063] In addition, the dataset can be preprocessed, such as denoising, deduplication, word segmentation, labeling, etc., to improve the learning effect and generation quality of the model. By expanding the scale and diversity of the dataset, the model can better adapt to text generation tasks in different scenarios.

[0064] The target sequence extension length determines the maximum length range of the generated text. This length is defined to ensure that the generated text is long enough to contain all necessary information, but not too long to cause redundancy and information overload. In different application scenarios, the target sequence extension length can be adjusted based on the statistical results of historical data. For example, the extension length can be determined based on the average length of historical generated text or by analyzing the importance of the text content. In addition, the extension length can be adjusted dynamically to change the length of the generated text in real time according to the specific needs of the task and the actual content generation during the text generation process.

[0065] Total diffusion time steps refers to the total number of time steps used by the model during the generation process. This parameter determines the level of sophistication of the generation process and the complexity of the generated content. Longer time steps allow the model to generate and adjust at a more detailed level, while shorter time steps may be faster but less accurate.

[0066] The time step of the initial diffusion model can be set according to the specific text generation task. For example, when generating complex and logical text, a longer time step can be selected to ensure the quality and consistency of the generated text. When generating simpler or templated text, a shorter time step can be selected to increase the generation speed. The determination of the time step can also be dynamically adjusted based on the real-time feedback of the generated content to achieve more efficient and accurate text generation.

[0067] In one specific implementation, training text data sets are first extracted from historical financial reports, market analysis data, and macroeconomic indicators. These data sets include a large amount of text content, such as stock market analysis, economic trend forecasts, and industry research reports. Based on these data analyses, the system derives the length and content structure of typical financial reports, and defines the target sequence extension length of the initial diffusion model to ensure that the generated text can cover all necessary market information and analytical insights. Next, the system sets a longer time step for the diffusion model to ensure that each data point and analysis conclusion is fully processed during the generation process. During the generation process, the diffusion model can be dynamically adjusted according to the real-time changes in market data to ensure that the generated analysis report can accurately reflect the current market situation and forecast trends. The final generated report is detailed and logically clear, providing a reliable basis for investment decisions.

[0068] Through the above steps, the target sequence extension length and time step are dynamically adjusted, and high-quality, personalized, real-time and accurate text output is achieved in text generation. It is suitable for various scenarios such as financial analysis reports, product recommendations and customer service responses, greatly improving the flexibility and adaptability of text generation.

[0069] S20, selecting a sentence-level time step within a preset range, calculating a starting point of a generated sequence according to the sentence-level time step, and assigning a token-level time step to each token in the training text dataset;

[0070] In this embodiment, when generating text, the sentence-level time step determines the step size during the generation process. The preset range is usually determined by the application scenario and the characteristics of the text. The time step can be set to a fixed value or a value within a dynamic range. For example, when generating complex financial reports, the sentence-level time step can be set to a shorter step (such as 1 to 3 time steps) to ensure the meticulousness and accuracy of the generated content. When generating simpler texts (such as customer notifications), the step size can be set to a longer range (such as 5 to 10 time steps) to speed up the generation speed.

[0071] The range of time steps can be dynamically adjusted according to the needs of different text generation tasks. For example, for descriptive text, the range of time steps can be set between 3 and 5 to ensure that the generated text has good fluency and coherence. For highly specialized texts (such as legal or technical documents), the range of time steps may need to be smaller (such as 1 to 2) to ensure that each part of the generated content is carefully processed.

[0072] The starting point of the generated sequence is the initial position of the generated text, which is determined by the selected sentence-level time step. This starting point determines the position where the generated sequence begins, which affects the order and logic of the subsequent generated content. Usually, the position of the starting point is combined with the logical order of the text. For example, when generating a market analysis report, the starting point of the generated sequence can be set at the analysis part of the market opening data.

[0073] The starting point of the generated sequence can be determined by comparing the sentence-level time step with the target sequence length. For example, if the target sequence length is 100 and the time step is set to 2, the generation starting point may start from the 20th position. This setting allows the generation process to start from the key part of the content, thereby highlighting the key information and ensuring the logical flow of the text.

[0074] The token-level time step determines the generation stride of each token during the generation process. The token-level time step can be assigned based on factors such as the contextual importance of the token, word frequency, etc. For example, for key terms or high-frequency words, a shorter time step (such as 0.1 to 0.5 time steps) is assigned to ensure the accuracy of the generated content. On the contrary, for less important tokens (such as conjunctions, rarely used words), the time step can be set to be longer (such as 1 to 2 time steps) to improve the generation efficiency. The allocation of token-level time steps can be dynamic and adjusted according to changes in context. For example, when generating legal documents, words involving core legal terms may be assigned shorter time steps (such as 0.05 to 0.2) to ensure accurate description of the legal terms. In more ordinary descriptive paragraphs, the token-level time step may be set to be longer (such as 0.5 to 1.0) to speed up the generation process.

[0075] In one specific implementation, the system is used to generate a financial market analysis report. First, within a preset range, the system selects a shorter sentence-level time step (such as 1 to 3 time steps) to ensure that the analysis and description of each market data point during the generation process can be meticulous. Subsequently, the system calculates the starting point of the generated sequence based on the selected sentence-level time step, and chooses to start the analysis from the market opening data to ensure the logic and coherence of the report. The system assigns a dynamic token-level time step to each token in the training text dataset. For example, for key market terms and indicators (such as "stock index", "yield"), a shorter time step (such as 0.2 to 0.5 time step) is assigned to ensure its accurate description when generating the report; for auxiliary information (such as time, place), a longer time step (such as 1.5 to 2 time steps) is assigned to speed up the generation. Finally, the system generates a market analysis report with a clear structure and rigorous logic to help investors make wise decisions.

[0076] Through the above steps, the sentence-level and token-level time steps are dynamically adjusted, which effectively balances the accuracy and efficiency of text generation, ensuring that the generated content has both logical coherence and semantic accuracy, and can quickly generate high-quality text in different scenarios. It is particularly suitable for complex text generation tasks such as financial reports and product recommendations.

[0077] S30, calculating the generation probability of each token through the initial diffusion model and the token-level time step, defining a loss function through the generation probability, updating the parameters of the initial diffusion model through the loss function, and obtaining a target diffusion model;

[0078] In this embodiment, during the text generation process, the model needs to assign a generation probability to each generated token (such as a word or phrase). This probability is calculated by the initial diffusion model combined with the token-level time step. The generation probability reflects the probability of a token appearing under the current context. This step ensures that the generated text is not only grammatical and logical, but also has high semantic coherence.

[0079] The calculation of generation probability is usually based on deep learning models (such as Transformer or RNN), combined with the content generated by the previous text and the set time step. The time step determines the model's reliance on the previous text and the flexibility of the current generation. For example, in financial texts, the generation probability of important terms needs to be accurately calculated to ensure the rigor and logic of the content.

[0080] The loss function is a key component in model training that measures the gap between the predicted output and the actual target. By defining the loss function through the generation probability, the quality of the generated text can be quantified. The closer the generation probability is to the target distribution, the lower the value of the loss function.

[0081] The loss function can be in the form of cross entropy loss or KL divergence, depending on the design of the model and the requirements of the training task. The cross entropy loss function is more common. It adjusts the model parameters by comparing the difference between the probability distribution of the model output and the actual distribution, making the generated text more accurate and coherent.

[0082] The ultimate goal of model training is to minimize the loss function by continuously adjusting the parameters, thereby obtaining an optimized target diffusion model. By calculating the gradient of the loss function and using an optimization algorithm (such as gradient descent), the model parameters are gradually updated and the generation effect is improved accordingly.

[0083] The parameter update process usually uses optimization algorithms such as stochastic gradient descent (SGD) or Adam optimizer. These optimizers adjust the weights of the model based on the gradient information of the loss function so that the generated text gradually approaches the target output. The resulting target diffusion model can generate high-quality text in practical applications.

[0084] In one specific implementation, the system is used to generate financial product description documents. The system accurately calculates the generation probability of each product feature description through an initial diffusion model and a shorter token-level time step. For example, the generation probability of terms such as "annual rate of return" and "minimum investment amount" are calculated in detail to ensure the accuracy of the description document. Next, the system uses a cross-entropy-based loss function to compare the probability generated by the model with the probability distribution of the actual document, and updates the model parameters by calculating the gradient of the loss function. The optimization algorithm used is Adam to ensure fast convergence in a high-dimensional parameter space. Ultimately, the generated target diffusion model can automatically generate detailed and accurate product description documents, which helps customers fully understand financial products.

[0085] Through the above steps, the accuracy and semantic coherence of text generation can be effectively improved. The target diffusion model finally obtained can generate logically rigorous and accurate texts in financial scenarios, such as financial analysis reports, investment proposals, and product description documents, which greatly improves the quality and practicality of text generation.

[0086] S40, acquiring input data, and inputting the input data into the target diffusion model to obtain a target text sequence.

[0087] In this embodiment, during the text generation process, input data refers to the initial information used to drive the target diffusion model to generate text. The input data can be text fragments, keywords, structured data or other forms of information, which provide the basic content framework and context for generating the target text sequence.

[0088] Input data can be obtained in many ways, such as user input, real-time data automatically obtained by the system, data obtained from a database or API, etc. The form and content of the input data will directly affect the theme, style and content of the generated text, so it may need to be preprocessed before use, such as denoising, formatting or extracting key information.

[0089] The acquired input data is provided to the trained target diffusion model. The model uses this data as the starting point and context for generating the target text sequence, and starts the generation process through the preset generation algorithm. The input data can be used as the initial condition of the model to determine the direction of the generation process. Different forms of input data (such as sequence text, numerical parameters, etc.) may trigger different generation paths and strategies in the model. In some advanced applications, the input data can also dynamically adjust the generation rules of the model, making the generation process more flexible and personalized.

[0090] The target text sequence is the final text content generated by the model based on the input data. These contents can be in the form of complete articles, paragraphs, conversations, reports, etc., depending on the requirements of the generation task. The generated text sequence should conform to the subject of the input data and be semantically consistent with the input content. The generated target text sequence can be directly used for display, storage, or further processing. For example, when generating a financial report, the target text sequence can include detailed content such as market analysis, forecast data, etc., and can be formatted or adjusted according to specific requirements to meet the standards for publication or transmission.

[0091] Through the above steps, high-quality, personalized text content can be generated based on specific input data. This generation method is suitable for scenarios such as financial reports, investment proposals, and product introductions, which significantly improves the efficiency and accuracy of text generation and ensures that the generated content meets the expected topics and needs.

[0092] The present invention relates to a text generation method based on a diffusion strategy, which obtains a training text data set, defines the target sequence extension length of an initial diffusion model, and determines the total diffusion time step. In the generation process, a sentence-level time step is first selected within a preset range, the starting point of the generation sequence is calculated, and a corresponding token-level time step is assigned to each token. Through these time steps, the model can dynamically adjust the generation order and step length of each token to adapt to the sequential dependency characteristics of natural language. On this basis, the model calculates the generation probability of each token, defines and optimizes the loss function, and finally updates the model parameters through optimization methods such as gradient descent to obtain a target diffusion model. The present invention can accept input data and generate a target text sequence that conforms to the context logic, thereby effectively solving the problem of sequential dependency in the text generation process and ensuring the logical coherence and semantic rationality of the generated content.

[0093] In one embodiment, in the above S40, inputting the input data into the target diffusion model to obtain a target text sequence includes:

[0094] S401, inputting the input data into the target diffusion model, and using the input data as a starting point for generating a target text sequence;

[0095] S402, determining a decreasing time step sequence based on the target sequence extension length and the total diffusion time step;

[0096] S403, sampling an initial latent variable from a standard Gaussian distribution, and initializing the initial latent variable to obtain a current latent variable;

[0097] S404, generating the target text sequence according to the generation starting point of the target text sequence, the current latent variable and the decreasing time step sequence.

[0098] In this embodiment, input data (such as text snippets, keywords, or numerical data provided by the user) are input into the trained target diffusion model. The input data, as the starting point of the generation process, determines the initial content and context of the generated text. For example, when generating a financial report, the input data may be recent market data or key economic indicators, and the model will generate a complete analysis report starting from these data. When generating a financial report, the input data may include current market prices, stock trends, etc. After these data are input into the model, the model will start with a description of the market overview and gradually generate a complete report containing detailed analysis and predictions. This ensures that the generated text is closely centered around the input data and meets actual needs.

[0099] In the text generation process, the time step controls the progress of each generation step. A decreasing time step sequence means that as the generation process progresses, the time step gradually decreases, so that the generation process gradually approaches the target sequence length from coarse to fine. For example, when generating a detailed investment report, the initial time step is long to quickly generate the framework content; as the generation proceeds, the time step gradually shortens to ensure that every detail in the report is processed in detail. For example, when generating a financial product description, a decreasing time step sequence may initially allow the model to generate a broad description of the product category and market positioning. As the generation progresses, the time step shortens, and the model will describe the product's rate of return, investment threshold and other specific details in detail.

[0100] Latent variables are used to represent potential content structures or topics during the generation process. By sampling from a standard Gaussian distribution, the model obtains a random initial latent variable, which introduces diversity and randomness to the generation process. The initialized current latent variable combines the input data to ensure that the generated content is both novel and in line with the context of the input data. For example, when generating a market forecast report, the sampling and initialization of the latent variables may combine the current market data and economic forecasts to ensure that the generated report has practical reference value.

[0101] When generating customized investment proposals for customers, the initialization of latent variables may combine the customer's risk preferences and the dynamic changes in the market, so that the generated recommendations are both in line with customer needs and reflect the latest market trends.

[0102] The final target text sequence is gradually generated by integrating the generation starting point, latent variables, and decreasing time step sequence. The generation starting point provides the initial content, the latent variables introduce diversity and potential structure, and the time step sequence controls the sophistication of the generation process. The final generated text should be logically coherent, semantically reasonable, and consistent with the theme of the initial input data. For example, when generating a financial analysis report, the generation starting point may be market opening data, the latent variables reflect the potential structure of market fluctuations, and the decreasing time step ensures that each generated analysis paragraph is detailed and accurate. For example, when generating a financial product introduction document, the starting point may be the basic information of the product, the latent variables are initialized in combination with market data, and the decreasing time step ensures that the document content is gradually expanded, and finally a complete introduction is generated, covering product features, market positioning, and investment return analysis.

[0103] In one specific implementation, it is used to generate a financial market analysis report. The system first obtains the latest market data (such as stock prices, market indices) from the database as input data, and inputs it into the target diffusion model as the starting point for generating the report. Then, based on the set target sequence length and diffusion time step, the system determines the decreasing time step sequence to ensure precise control of the generation process. The model then samples the initial latent variables from the standard Gaussian distribution, and initializes them in combination with the market data to generate the current latent variables. During the generation process, the system combines the generation starting point, the current latent variables and the decreasing time step sequence to gradually generate a market analysis report. The final report is detailed, logically clear, and accurately reflects market dynamics.

[0104] Through the above steps, this embodiment ensures the coherence, semantic rationality and personalization of the generated text, is suitable for scenarios such as financial reports, investment advice and product descriptions, and significantly improves the accuracy of text generation.

[0105] In one embodiment, in the above S40, inputting the input data into the target diffusion model to obtain a target text sequence includes:

[0106] S405, determining an initial generation stage, a mid-generation stage, and a late generation stage of the target diffusion model in the generation process according to the total diffusion time step and the preset diffusion ratio;

[0107] S406, in the initial generation stage, generating a text sequence based on a generation strategy of preset grammar rules;

[0108] S407, in the mid-term generation stage, generating a text sequence based on a generation strategy of context-dependent rules and intermediate semantic verification rules;

[0109] S408, in the later generation stage, the text sequence generated in the initial generation stage and the mid-generation stage is corrected based on the generation strategy of grammar and spelling check rules to obtain the target text sequence.

[0110] In the present embodiment, according to the total time step of the whole text generation process and the preset diffusion ratio, the generation process is divided into three stages: initial stage, mid-term and late stage. Each stage corresponds to a different generation strategy to gradually improve the generated text sequence. The initial stage is usually used to generate the skeleton and main structure of the text, the mid-term stage further refines the content, and the late stage is corrected and polished. For example, when generating a financial report, the total diffusion time step can be set to 1000 steps. By presetting the diffusion ratio, the initial generation stage may occupy the first 30% of the step length (i.e. 300 steps), the mid-term generation stage occupies the middle 50% (i.e. 500 steps), and the late generation stage uses the remaining 20% ​​(i.e. 200 steps). This segmentation process helps to ensure that the text is gradually refined from structure to content, and finally obtains high-quality generated text.

[0111] The initial generation stage is the basic stage of text generation, and its main task is to build the overall framework and basic content of the text. The generation strategy at this stage relies on preset grammatical rules to ensure that the generated text has basic grammatical correctness and structural rationality. When generating financial reports, the initial stage may generate a preliminary description of the market profile, an overview of core data, etc., which usually follow fixed grammatical and structural rules. For example, the beginning of the report may contain market background, major economic indicators, etc., which need to be accurately expressed and consistent with the expected text structure.

[0112] The mid-term generation phase is to further refine and expand the text content based on the initial framework. The generation strategy in this phase is based on context-dependent rules and semantic validation rules to ensure that the generated content is semantically consistent with the previous text and logically coherent. In the process of generating financial reports, the mid-term phase may generate more detailed market analysis, data interpretation, and trend forecasts. Context-dependent rules ensure that each generated text is closely related to the previous and next content, while semantic validation rules ensure that the generated content is logically and meaningfully consistent. For example, if it is mentioned in the initial stage that a market is rising, the analysis content in the mid-term generation phase should further support this view.

[0113] The post-generation phase is mainly used to correct and polish the generated text. The generation strategy at this stage relies on grammar checking and spelling checking rules to ensure that the final generated text is linguistically error-free and the content is fluent. In the late stage of financial report generation, the system will check the previously generated text, correct possible grammatical errors, spelling errors or irregularities in format, and perform language polishing. For example, the system may refine the end of the report to make it more in line with the format requirements of a formal report and ensure that the wording is accurate and the expression is clear.

[0114] This embodiment divides the text generation process into three stages: early, middle and late stages, and adopts different generation strategies in each stage, thereby effectively improving the structural rationality, content coherence and language accuracy of the generated text. It is suitable for text generation in complex scenarios, such as financial reports, investment advice and product descriptions, and significantly improves the quality and practicality of text generation.

[0115] In one embodiment, in the above S20, assigning a token-level time step to each token in the training text dataset includes:

[0116] S201, predict the importance of each token in the context using a pre-trained language model;

[0117] Count the word frequency of each token in the training text dataset, and record the position information of each token in the sentence;

[0118] S202, determining a token weight of each token according to the importance, word frequency and position information of each token, and determining a token-level time step of each token according to the token weight of each token and the sentence-level time step.

[0119] In this embodiment, a pre-trained language model is used to evaluate the importance of each token in context. Importance can be represented by a value between 0 and 1, with the larger the value, the more important the token is in context. For example, in a financial report, the word "market" may be given a weight of 0.8, while a conjunction such as "and" may only have a weight of 0.2. The system will guide the subsequent generation process based on the value of the importance weight. For example, "market" with an importance weight of 0.8 may be processed with a higher priority during the generation process to ensure that it occupies an important position in the text.

[0120] The system counts how often each token appears in the training dataset, quantified by a frequency value (e.g., number of occurrences). The system also records the position of the token in the sentence, represented by a ratio between 0 and 1. For example, the word "market" appears 2,500 times in 5,000 sentences, and the frequency can be recorded as 2,500 times, while its probability of appearing at the beginning of the sentence may be 0.7. For example, if the word "market" appears frequently at the beginning of sentences in financial reports, with a frequency of 2,500 times and a probability of appearing at the beginning of a sentence of 0.7, the system may assign a higher weight to the word to ensure that it is fully reflected in the generated report.

[0121] The above values ​​(importance, word frequency, position ratio) of each token are combined into a comprehensive weight, usually calculated by weighted average or other methods. For example, the final weight of the word "market" may be calculated as follows: token weight = 0.5 × importance weight + 0.3 × word frequency weight + 0.2 × position weight.

[0122] If the importance weight is 0.8, the frequency weight is 0.5 (normalized value), and the position weight is 0.7, then the final weight of the word "market" may be: token weight = 0.5 × 0.8 + 0.3 × 0.5 + 0.2 × 0.7 = 0.65.

[0123] Based on the obtained token weight, the system can further determine the token-level time step. For example, if the sentence-level time step is 5 step units and the token weight is 0.65, the token-level time step of the word "market" can be calculated as: token-level time step = sentence-level time step × (1-token weight). For "market", this will be: token-level time step = 5 × (1-0.65) = 1.75. This means that the word will be generated in a shorter time step, emphasizing its importance.

[0124] Through the above steps, this embodiment effectively improves the ability to identify and prioritize key content during the text generation process.

[0125] In one embodiment, in the above S20, calculating the starting point of the generated sequence according to the sentence-level time step includes:

[0126] S203, using the value of the target sequence length minus the value of the sentence-level time step to obtain a starting point difference, and determining whether the starting point difference is less than zero;

[0127] S204, if the starting point difference is not less than zero, directly using the starting point difference as the position of the starting point of the generated sequence;

[0128] S205: If the starting point difference is less than zero, the starting point difference is assigned zero and used as the starting point position of the generated sequence.

[0129] In the present embodiment, when generating a text sequence, the calculation of the starting point difference is to determine the starting position of the generated sequence. By deducting the value of the sentence-level time step from the total length of the target sequence, the system calculates the starting point difference of the generated sequence. This difference determines the starting position of text generation, ensuring that the generation process starts from the correct position. For example, if the target sequence length is 100 units and the sentence-level time step is 20 units, then the starting point difference is 80 units.

[0130] The starting point difference is calculated to control the starting point of the generated text. If the goal is to generate a sentence fragment, when the sentence-level time step is short, the starting point difference will be closer to the end of the target sequence, so that the generated content is more focused on the second half of the sequence. For longer time steps, the starting point difference will be closer to the beginning of the sequence, and the generation process will cover a larger range of the sequence.

[0131] After calculating the starting point difference, the system needs to determine whether the value is negative. If the starting point difference is less than zero, it means that the sentence-level time step exceeds the length of the target sequence. In this case, the starting point of the generated sequence cannot be negative, so it needs to be adjusted.

[0132] If the calculated starting point difference is non-negative (i.e. not less than zero), the difference can be directly used as the starting point of the generated sequence. This means that the generation process will start from this starting position and cover the remaining sequence part. This method of directly determining the starting point is suitable for text generation in most cases. For example, if the target sequence length is 80, the sentence-level time step is 30, and the starting point difference is 50, the system will generate the sequence from position 50 to ensure that the generated content covers the last 30 units of the target sequence.

[0133] When the starting point difference is less than zero, the system adjusts the starting point position to 0, that is, the generation process will start from the beginning of the target sequence. This adjustment ensures that the generated sequence is within the valid range of the target sequence and avoids generating invalid or out-of-bounds content. For example, the target sequence length is 40, the sentence-level time step is 50, and the starting point difference is -10. At this time, the system adjusts the starting point to 0, and the generation process will start from the first position of the sequence, covering the entire target sequence.

[0134] In one embodiment, in this embodiment, the system is used to generate a financial market analysis report. The target sequence length is 100 data points, and the sentence-level time step is set to 30 units. The system first calculates the starting difference, that is, 100 minus 30, which is 70 units. Since the starting difference is greater than zero, the system directly uses 70 as the starting position of the generated sequence. The generation process starts from the second half of the market analysis report to ensure that the generated content focuses on the recent market dynamics analysis.

[0135] Through the above steps, this embodiment can flexibly determine the starting position of text generation and ensure that the generated content covers the valid part of the target sequence, which not only improves the accuracy and coherence of text generation, but also avoids the generation of out-of-bounds content.

[0136] In one embodiment, in the above S30, a loss function is defined by generating probability, and the parameters of the initial diffusion model are updated by the loss function to obtain a target diffusion model, including:

[0137] S301, determining the task requirements of the initial diffusion model, and determining a plurality of preset loss functions and the weight of each loss function according to the task requirements;

[0138] S302, defining a total loss function by weights of the loss functions and the generation probability;

[0139] S303, calculating the generation probability and accumulating the loss through the initial diffusion model, calculating the partial derivative of the loss function with respect to each parameter of the initial diffusion model through the chain rule, updating the parameters of the initial diffusion model using the gradient descent algorithm, and obtaining the target diffusion model.

[0140] In this embodiment, during the model training process, the task requirements determine the goals that the model needs to optimize. Based on these requirements, the system will set multiple loss functions that measure the deviation between the generated results and the target output. The weight of each loss function reflects its relative importance in the overall optimization goal. For example, in a text generation task, there may be a loss function that measures the grammatical correctness of the text and another that measures the coherence of the generated content. The system will assign weights according to specific task requirements, such as a grammatical correctness weight of 0.6 and a coherence weight of 0.4. For example, in financial report generation, task requirements may include the accuracy, logical coherence, and data consistency of the text. The system may set different loss functions for each requirement and assign corresponding weights. For example, the weight of the accuracy loss function is set to 0.5, the logical coherence is set to 0.3, and the data consistency is set to 0.2. These weights will determine the degree of attention the model pays to each goal during the training process.

[0141] The total loss function is the weighted sum of all preset loss functions, calculated in combination with the generation probability. The generation probability reflects the model's confidence in the prediction of each possible result during the generation process. By calculating the weighted sum of each loss function, the system derives a comprehensive indicator that measures the overall deviation between the current output of the model and the target output. When generating text, assume that the grammatical correctness loss of the text generated by the model is 0.3, the coherence loss is 0.4, the data consistency loss is 0.5, and the generation probability is 0.7. The total loss function can be calculated by weighted sum: total loss function = 0.6×0.3+0.3×0.4+0.2×0.5=0.39. This comprehensive loss value is used to guide the model on how to adjust parameters in the next iteration.

[0142] The model calculates the generation probability at each step of text generation and accumulates the overall loss according to the previously defined loss function. Through the chain rule, the system can calculate the impact of each loss function on the model parameters (i.e., partial derivatives). These partial derivatives indicate how the various parameters should be adjusted to reduce the total loss value. The gradient descent algorithm adjusts the model parameters through backpropagation, gradually approaching the optimal solution, thereby optimizing the model performance. For example, during the training process, the model calculates the generation probability of a certain text sequence to be 0.65, and accumulates an overall loss value according to the defined loss function. Through the chain rule, the system calculates the partial derivatives of each parameter and uses the gradient descent algorithm to adjust the parameters. If the partial derivative of a parameter is large, it means that the parameter has a greater impact on the loss, and the system will adjust this parameter accordingly. After many iterations, the target diffusion model obtained in the end can perform well in text generation tasks.

[0143] This embodiment effectively improves the performance of the model in complex text generation tasks by defining multiple loss functions and weightedly calculating the total loss function. Combining the chain rule and gradient descent algorithm, the system can accurately optimize the model parameters. The target diffusion model generated in the end performs well in terms of content accuracy, logical coherence, and semantic consistency, and is suitable for scenarios such as financial reports, investment proposals, and product descriptions.

[0144] In one embodiment, in the above S10, defining the target sequence extension length of the initial diffusion model includes:

[0145] S101, dividing the training text data set into multiple text sequences, and counting the actual length of each text sequence;

[0146] S102, calculating the average length, maximum length, minimum length and standard deviation of the text sequences in the training text data set according to the actual length of each text sequence, and determining the actual length of the target sequence in combination with the text generation task requirements;

[0147] S103, setting the target sequence extension length to a preset multiple of the actual length of the target sequence.

[0148] In this embodiment, the entire training text data set is divided into several smaller text sequences, and the length of each text sequence is a key parameter that the system needs to count. The length of a text sequence refers to the number of characters or words contained in each text segment. By counting the actual length of each text sequence, the system can obtain basic information about each sequence in the training data, such as the length distribution of different text sequences. For example, in financial report generation, the system may divide an entire report into several parts, such as a market overview, data analysis, conclusions, etc. The actual length of each part (such as the number of characters or words) will be recorded for use in subsequent steps.

[0149] After counting the actual lengths of each text sequence, the system will further calculate statistical indicators of these lengths, such as average length, maximum length, minimum length, and standard deviation. These indicators help the system understand the overall distribution of the lengths of each sequence in the training data set. Combined with the requirements of the text generation task, the system can determine a reasonable target sequence length from these statistical indicators. Suppose the average length obtained by the system is 200 words, the maximum length is 300 words, the minimum length is 100 words, and the standard deviation is 50 words. Based on these indicators, the system can judge the task requirements of text generation. For example, if the generated text needs to cover a large amount of information, the system may choose a target sequence length close to the maximum length; if a higher generation speed is required, the system may choose a target sequence length close to the average length.

[0150] The extended length of the target sequence refers to the extent to which the target length of the generated sequence is extended relative to the actual length. By setting a preset multiple, the system can ensure that the generated text sequence can meet the task requirements. The preset multiple can be 1.5 times, 2 times, or higher, depending on the complexity of the generation task and the expected level of detail of the generated text. For example, in financial report generation, if the actual length of the report is usually 200 words, and the generated content needs to be more detailed, the system can select an expansion multiple of 2 and set the target sequence extension length to 400 words. This ensures that the generated report covers more details and analysis content, meeting higher accuracy requirements.

[0151] In one specific implementation, the system is used to generate a financial market analysis report. First, the system divides the entire training text data set into several text sequences and counts the actual length of each sequence. For example, the system divides different parts of the report (such as market overview, data analysis, etc.) into multiple sequences and records their actual number of words. Next, the system calculates the average length, maximum length, minimum length, and standard deviation of these sequences, assuming that the average length is 250 words, the maximum length is 350 words, and the minimum length is 150 words. According to the requirements of the generation task, the system decides to set the actual length of the target sequence to 300 words. Finally, the system sets the target sequence extension length to 2 times, that is, 600 words, to ensure that the generated market analysis report is detailed and rich in content.

[0152] By dividing the training text data set into multiple text sequences and counting the actual length of each sequence, the system can accurately calculate and set the extended length of the target sequence to ensure that the generated text sequence meets the task requirements. This not only improves the completeness and detail of the generated content, but also can flexibly adapt to the requirements of different generation tasks, such as the generation of financial reports, investment proposals, and product description documents.

[0153] The present invention also provides a text generation device based on a diffusion strategy, referring to Figure 2 , Figure 2 The functional module diagram of a preferred embodiment of a text generation device based on a diffusion strategy of the present invention is shown in FIG. The text generation device based on a diffusion strategy comprises:

[0154] The model initialization module is used to obtain the training text dataset, define the target sequence extension length of the initial diffusion model, and determine the total diffusion time step of the initial diffusion model;

[0155] Generate a starting point determination module, which is used to select a sentence-level time step within a preset range, calculate the starting point of the generated sequence according to the sentence-level time step, and assign a token-level time step to each token in the training text dataset;

[0156] A model training and parameter optimization module, used for calculating the generation probability of each token through the initial diffusion model and the token-level time step, defining a loss function through the generation probability, updating the parameters of the initial diffusion model through the loss function, and obtaining a target diffusion model;

[0157] The text generation module is used to obtain input data, input the input data into the target diffusion model, and obtain a target text sequence.

[0158] The specific implementation of the text generation device based on the diffusion strategy of the present invention is basically the same as the various embodiments of the text generation method based on the diffusion strategy described above, and will not be repeated here.

[0159] The present invention also provides a text generation device based on a diffusion strategy, such as Figure 3 As shown, the text generation device based on the diffusion strategy may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0160] Those skilled in the art will understand that Figure 3 The hardware structure of the text generation device based on the diffusion strategy shown in the figure does not constitute a limitation of the text generation device based on the diffusion strategy, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0161] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a text generation program based on a diffusion strategy. The operating system is a program for managing and controlling a text generation device and software resources based on a diffusion strategy, and supports the operation of the network communication module, the user interface module, the text generation program based on a diffusion strategy, and other programs or software; the network communication module is used to manage and control the network interface 1004; and the user interface module is used to manage and control the user interface 1003.

[0162] exist Figure 3 In the hardware structure of the text generation device based on the diffusion strategy shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; the processor 1001 can call the text generation program based on the diffusion strategy stored in the memory 1005, and perform the same operation as the text generation method based on the diffusion strategy.

[0163] The specific implementation of the text generation device based on the diffusion strategy of the present invention is basically the same as the various embodiments of the text generation method based on the diffusion strategy described above, and will not be repeated here.

[0164] In addition, an embodiment of the present invention further proposes a computer storage medium, on which a text generation program based on a diffusion strategy is stored. When the text generation program based on a diffusion strategy is executed by a processor, the steps of the text generation method based on a diffusion strategy as described above are implemented.

[0165] The specific implementation of the computer storage medium of the present invention is basically the same as the above-mentioned embodiments of the text generation method based on the diffusion strategy, and will not be repeated here.

[0166] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.

[0167] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0168] In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, RandomAccess Memory), a disk or an optical disk.

[0169] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0170] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0171] It should be noted that if software tools or components other than those of our company appear in the embodiments of the present application, they are only used for illustrative purposes and do not represent actual use.

Claims

1. A text generation method based on diffusion strategy, characterized in that: The following steps are involved: Obtain a training text dataset, define the target sequence extension length of the initial diffusion model, and determine the total diffusion time step of the initial diffusion model; Selecting a sentence-level time step within a preset range, calculating a starting point of a generated sequence according to the sentence-level time step, and assigning a token-level time step to each token in the training text dataset; Calculating the generation probability of each token by using the initial diffusion model and the token-level time step, defining a loss function by the generation probability, and updating the parameters of the initial diffusion model by using the loss function to obtain a target diffusion model; Input data is acquired, and the input data is input into the target diffusion model to obtain a target text sequence.

2. The text generation method based on diffusion strategy as claimed in claim 1, characterized in that: Inputting the input data into the target diffusion model to obtain a target text sequence includes: Inputting the input data into the target diffusion model, and using the input data as a starting point for generating a target text sequence; Determining a decreasing time step sequence based on the target sequence extension length and the total diffusion time step; Sampling an initial latent variable from a standard Gaussian distribution, and initializing the initial latent variable to obtain a current latent variable; The target text sequence is generated according to the generation starting point of the target text sequence, the current latent variable and the decreasing time step sequence.

3. The text generation method based on diffusion strategy as claimed in claim 1, characterized in that: Inputting the input data into the target diffusion model to obtain a target text sequence includes: Determine the initial generation stage, the middle generation stage and the late generation stage of the target diffusion model in the generation process according to the total diffusion time step and the preset diffusion ratio; In the initial generation stage, text sequence generation is performed based on the generation strategy of preset grammatical rules; In the mid-term generation stage, text sequence generation is performed based on the generation strategy of context-dependent rules and intermediate semantic verification rules; In the later generation stage, the text sequence generated in the initial generation stage and the mid-generation stage is corrected based on the generation strategy of grammar and spelling check rules to obtain the target text sequence.

4. The text generation method based on diffusion strategy as claimed in claim 1, characterized in that: And assign a token-level time step to each token in the training text dataset, including: Use a pre-trained language model to predict the importance of each token in context; Count the word frequency of each token in the training text dataset, and record the position information of each token in the sentence; The token weight of each token is determined according to the importance, word frequency and position information of each token, and the token-level time step of each token is determined according to the token weight of each token and the sentence-level time step.

5. The text generation method based on diffusion strategy as described in claim 1, characterized in that: The starting point of the generated sequence is calculated according to the sentence-level time step, including: Subtract the sentence-level time step value from the target sequence length to obtain a starting point difference, and determine whether the starting point difference is less than zero; If the starting point difference is not less than zero, the starting point difference is directly used as the starting point position of the generated sequence; If the starting point difference is less than zero, the starting point difference is assigned zero and used as the position of the starting point of the generated sequence.

6. The text generation method based on diffusion strategy as claimed in claim 1, characterized in that: The loss function is defined by generating probability, and the parameters of the initial diffusion model are updated by the loss function to obtain the target diffusion model, including: Determining the task requirements of the initial diffusion model, and determining a plurality of preset loss functions and the weight of each loss function according to the task requirements; A total loss function is defined by the weights of each loss function and the generation probability; The generation probability is calculated and the loss is accumulated through the initial diffusion model, the partial derivative of the loss function with respect to each parameter of the initial diffusion model is calculated through the chain rule, and the parameters of the initial diffusion model are updated using the gradient descent algorithm to obtain the target diffusion model.

7. The text generation method based on diffusion strategy as claimed in claim 1, characterized in that: Define the target sequence extension length for the initial diffusion model, including: Dividing the training text data set into multiple text sequences, and counting the actual length of each text sequence; Calculate the average length, maximum length, minimum length and standard deviation of the text sequences in the training text data set according to the actual length of each text sequence, and determine the actual length of the target sequence in combination with the text generation task requirements; Set the target sequence extension length to a preset multiple of the actual length of the target sequence.

8. A text generation device based on diffusion strategy, characterized in that: The text generation device based on the diffusion strategy includes: The model initialization module is used to obtain the training text dataset, define the target sequence extension length of the initial diffusion model, and determine the total diffusion time step of the initial diffusion model; Generate a starting point determination module, which is used to select a sentence-level time step within a preset range, calculate the starting point of the generated sequence according to the sentence-level time step, and assign a token-level time step to each token in the training text dataset; A model training and parameter optimization module, used for calculating the generation probability of each token through the initial diffusion model and the token-level time step, defining a loss function through the generation probability, updating the parameters of the initial diffusion model through the loss function, and obtaining a target diffusion model; The text generation module is used to obtain input data, input the input data into the target diffusion model, and obtain a target text sequence.

9. A text generation device based on diffusion strategy, characterized in that: The diffusion strategy-based text generation device includes a memory, a processor, and a diffusion strategy-based text generation program stored in the memory and executable on the processor. When the diffusion strategy-based text generation program is executed by the processor, the steps of the diffusion strategy-based text generation method described in any one of claims 1 to 7 are implemented.

10. A computer storage medium, characterized in that: The storage medium stores a text generation program based on a diffusion strategy, and when the text generation program based on a diffusion strategy is executed by a processor, the steps of the text generation method based on a diffusion strategy as described in any one of claims 1 to 7 are implemented.

Images