A paraphrase sentence generation method based on a controllable latent space diffusion model
By adopting a paraphrase generation method based on a controllable latent space diffusion model, the problems of low efficiency and discontinuous embedding space in the existing technology are solved, and high-quality and efficient paraphrase generation is achieved, which improves the generation efficiency of the model and the performance of downstream tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING UNIV
- Filing Date
- 2024-01-12
- Publication Date
- 2026-07-03
AI Technical Summary
Existing paraphrase generation methods based on diffusion models suffer from inefficiency, truncation errors, and discontinuous text representations in the embedding space, resulting in limitations on generation quality and speed.
By adopting a controllable latent space diffusion model, and constructing a latent space encoding and decoding model and a diffusion model, combined with a semantic fragment controller, the continuous diffusion of text in the latent space is achieved, reducing rounding matching operations and improving generation quality and efficiency.
It achieves high-quality and highly diverse paraphrased sentence generation, reduces training and inference overhead, and improves the model's generation efficiency and the performance of downstream tasks.
Smart Images
Figure CN118070899B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for generating paraphrased statements, and more particularly to a method for generating paraphrased statements based on a controllable latent space diffusion model. Background Technology
[0002] With the development of deep learning and the advancement of natural language processing technology, research on language understanding and generation has attracted increasing attention from researchers. Paraphrasing, consisting of different sentences containing the same semantic information, is a common phenomenon in natural language. Paraphrase generation, as an important data augmentation technique, has played a significant role in fields such as neural machine translation, automatic question answering, information extraction, semantic parsing, and automatic summarization.
[0003] The diffusion model is a relatively new generative model that progressively removes noise from pure Gaussian noise to generate data, playing a significant role in image, video, and audio generation. Retelling generation tasks can be implemented using the diffusion model.
[0004] Currently, mainstream neural network-based paraphrase generation methods are divided into deterministic and non-deterministic generation. The deterministic paraphrase generation paradigm, which utilizes an encoder-decoder architecture, introduces diverse constraints by leveraging external features (such as grammar, example sentences, and keywords). This type of method focuses on high-quality generation but lacks theoretical guarantees for diversity. Another approach, utilizing variational autoencoders, models hidden layer representations with different contextual distributions. This method, based on sampling from Gaussian noise, provides theoretical guarantees for diversity, but still falls short in modeling complex and diverse natural language.
[0005] Diffusion models, as a recent and powerful generative model, have achieved a good balance between modeling generation quality and diversity. Currently, mainstream paraphrasing generation methods based on diffusion models are implemented using embedding space-based diffusion models. This method first encodes the text into the embedding space, then transforms it into high-quality data through multiple rounds of Markov sampling. In each round of transformation, additional rounding matching operations are needed to limit the diffusion transformation to valid lexical encodings, ensuring the meaningfulness of the generated data.
[0006] In existing technical solutions, a method for rehearsing and generating text by embedding spatial diffusion models has been proposed, which generates diverse texts through a partial diffusion process of text sequence data (Reference: Gong S, Li M, Feng J, et al. Diffuseq: Sequence to sequence text generation with diffusion models[J].2022.).
[0007] The existing rounding matching method suffers from two problems: inefficiency and truncation error. Nearest neighbor matching in rounding has significant training and inference overhead, greatly limiting the model's scalability and ease of use. Furthermore, rounding matching introduces truncation error, which accumulates during inference and ultimately reduces the quality of the generated model.
[0008] Other existing technical solutions have analyzed the phenomenon of discontinuous text representation in the embedded space when using the embedded space diffusion model for paraphrasing generation, and proposed a noise scaling strategy to manipulate the training and inference of the text diffusion model to reduce the truncation error in the rounding matching operation and improve the generation quality (Reference: Ye J, Zheng Z, Bao Y, et al. Dinoiser: Diffused conditional sequence learning by manipulating noises[J].2023.).
[0009] However, none of the above-mentioned existing technical solutions have fundamentally solved the problem of discreteness in the text diffusion model, and the resulting problems of inference speed and control truncation have not been solved either. Summary of the Invention
[0010] Purpose of the invention: The technical problem to be solved by the present invention is to provide a paraphrase generation method based on a controllable latent space diffusion model, which addresses the shortcomings of the existing technology.
[0011] To address the aforementioned technical problems, this invention discloses a method for generating restatements based on a controllable latent space diffusion model, comprising the following steps:
[0012] Step 1: Construct a paraphrase model based on the latent space diffusion model. Train the paraphrase model using an existing paraphrase text dataset. The existing paraphrase text dataset contains the original sentence and its paraphrase.
[0013] Step 2: Divide the paraphrased source sentence into key information and non-key information, and construct enhanced semantic fragment information;
[0014] Step 3, build the controller;
[0015] Step 4, train the controller;
[0016] Step 5: Combine the paraphrase model trained in Step 1 based on the latent space diffusion model and the controller trained in Step 4 to perform collaborative reasoning to generate paraphrased statements, thus completing the paraphrased statement generation task based on the controllable latent space diffusion model.
[0017] Furthermore, the restatement model based on the latent space diffusion model described in step 1 includes: a latent space encoding / decoding model and a latent space diffusion model, wherein,
[0018] The latent space encoding and decoding model includes an encoder and a decoder, wherein the encoder is used to encode text into a latent space representation, and the decoder is used to decode the latent space representation into text;
[0019] The aforementioned latent space diffusion model is used for iterative denoising of noisy latent space representations.
[0020] Furthermore, the restatement model based on the latent space diffusion model described in step 1 specifically includes:
[0021] Step 1-1: Use the BART pre-trained model as the latent space encoding / decoding model to provide latent space encoding and decoding, as detailed below:
[0022] Given a text sequence x = x1, x2, ..., x l , where x l This represents the l-th text in the text sequence x;
[0023] The encoder encodes the text sequence x into a d-dimensional latent space representation z, that is:
[0024] z = E(x)
[0025] Where E() represents the encoding;
[0026] Decoder D reconstructs the input information based on the aforementioned latent space representation z as follows:
[0027]
[0028] in, This represents the reconstructed text;
[0029] Steps 1-2: Construct the latent space diffusion model, as follows:
[0030] Step 1-2-1: A Transformer model with normalized pre-sequence layers is used as the backbone network; each layer consists of a self-attention module, a cross-attention module, and a feedforward layer.
[0031] Step 1-2-2: Introduce an adaptive layer normalization module into the Transformer model. Each input time step t is first transformed into a d-dimensional vector and then passed into the Transformer model through this adaptive layer normalization module.
[0032] Steps 1-2-3: Use GeGLU as the activation function for the latent space diffusion model;
[0033] Steps 1-3 involve generating a restatement, as detailed below:
[0034] Step 1-3-1: Use the encoder in the latent space encoding / decoding model to perform latent space encoding on the input text sequence;
[0035] Step 1-3-2: Iteratively call the latent space diffusion model to denoise the above latent space encoding;
[0036] Step 1-3-3: Use the decoder in the latent space encoding and decoding model to decode the denoised latent space encoding and complete the retelling generation.
[0037] Furthermore, the training of the restatement model based on the latent space diffusion model described in step 1 specifically includes:
[0038] Suppose a paraphrase pair in an existing paraphrase text dataset is...<a,b> Where 'a' represents the source text sequence, i.e., the original sentence or source sentence, and 'b' represents the restatement of sentence 'a'; the training process includes a forward process and a backward process, as detailed below:
[0039] During the forward pass, the latent space representation z0 = E(y) of the target restatement is derived from the cosine noise sequence β. t Add noise gradually until it becomes pure Gaussian noise. T That is, the above process is divided into several sub-processes, and for the t-th intermediate state z t , means as follows:
[0040]
[0041] Among them, the parameters are defined. q(z t |z0) represents the sampling process with added noise, ∈ represents a noise that follows a normal distribution, β i This represents the noise intensity at the current time step i;
[0042] During the reverse process, the latent space diffusion model is guided by the source restatement conditional encoding information c = E(x) to gradually remove noise, including the Gaussian noise z. T Restore to original data For each intermediate state zt The latent space diffusion model is trained to predict the original data through this intermediate state;
[0043] The loss function used during training is as follows:
[0044]
[0045] Where t represents the number of noise steps sampled during model training; z0 is the original representation of the data; z t This is the representation of the data after adding noise; z θ It is a learnable latent space diffusion model.
[0046] Furthermore, the construction of enhanced semantic fragment information described in step 2 specifically includes:
[0047] Step 2-1: For the source sentence in the paraphrased sentence text dataset mentioned in Step 1, sample the longest preset number of words from the sentence as key information, and the other words as non-key information.
[0048] Step 2-2: Lexicalization of the sentence, that is, replacing the lexical units corresponding to non-key information in the sentence with special lexical units. <mask>Key information and corresponding word units are retained;
[0049] Step 2-3: Using the encoder described in Step 1-1, the source sentence after lexicalization in Step 2-2 is encoded into a continuous latent space representation as enhanced semantic fragment information.
[0050] Furthermore, the controller described in step 3 includes the same diffusion model as the restatement model based on the latent space diffusion model in step 1, i.e., a copy of the diffusion model. Zero-initialized convolutional modules are added before and after the copy of the diffusion model to fuse the input and output information of the two diffusion models.
[0051] Furthermore, the training controller mentioned in step 4, namely:
[0052] Step 4-1: Fix the paraphrasing model based on the latent space diffusion model trained in Step 1, and input the original sentence into the paraphrasing model above;
[0053] Step 4-2: Input the enhanced semantic fragment constructed in step 2 into the controller;
[0054] Step 4-3: Using the zero-initialization convolution module in the controller, information is fused between the controller and the restatement model based on the latent space diffusion model at the input and output stages to obtain the final output. Specifically as follows:
[0055]
[0056] Among them, z θ This represents the restatement model trained in step 1 based on the latent space diffusion model, z. θ ' indicates a copy of the diffusion model in the controller; zero_conv indicates a zero-initialized convolutional module with weights and offsets initialized to zero, z t This represents a noisy intermediate state, where c represents conditional information, t represents the current time step of the controller, and c kw This represents the latent space representation of the enhanced semantic fragment information constructed in step 2;
[0057] Step 4-4: Define the loss function to train the controller during the above process.
[0058] Furthermore, the loss function described in step 4-4 is:
[0059]
[0060] Furthermore, the collaborative reasoning described in step 5 specifically includes:
[0061] Step 5-1: Using the source sentence in the paraphrased sentence pair as a semantic condition, and using the enhanced semantic fragment as the input to the controller, from the complete noise sequence z... T Initially, an iterative denoising process is performed under the guidance of conditional information, ultimately generating the latent space representation of the target restatement.
[0062] Step 5-2: Use decoder D to transform the latent space representation into the target paraphrase.
[0063] Furthermore, the denoising process described in step 5-1 is accelerated iteratively using a diffusion model sampler based on ordinary differential equations.
[0064] Beneficial effects:
[0065] From a technical perspective, the technical solution of this invention improves the quality and efficiency of paraphrasing generation by using the latent space encoded by the pre-trained model in the text diffusion model; by introducing a semantic fragment controller, the quality of paraphrasing generation is further improved by receiving additional key semantic information, thus achieving a balance between semantic preservation and diversity.
[0066] From an application perspective, the technical solution of this invention can automatically generate high-quality and highly diverse paraphrased sentence pairs for natural language text, which can significantly reduce the manual cost required to collect data during model training; using high-quality and highly diverse paraphrased text can enhance the performance of the model in downstream tasks and improve the model's generalization and stability. Attached Figure Description
[0067] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.
[0068] Figure 1 This is a schematic diagram of the training, inference process, and overall architecture of the latent space diffusion model.
[0069] Figure 2 This is a schematic diagram of the latent space diffusion model enhanced by the semantic fragment controller.
[0070] Figure 3 This is a schematic diagram of the process of a controllable latent space retelling model for semantic fragment enhancement.
[0071] Figure 4 This is a schematic diagram of the process of constructing key semantic fragments. Detailed Implementation
[0072] This invention aims to address the problem of diffusion space discontinuity in text diffusion models based on embedding space, as well as the resulting high training and inference overhead, low generation quality, and truncation of additional control information caused by the necessary rounding matching operations.
[0073] This invention improves the usage space of the text diffusion model by replacing the text embedding space with the latent space corresponding to the encoder-decoder, effectively enhancing the continuity of the diffusion model's corresponding space and eliminating the "discreteness trap." It generates high-quality text results without discrete rounding matching operations. Furthermore, this invention injects semantic control information into the diffusion step, further improving the generation quality of paraphrased sentence pairs. Specifically, the technical solution of this invention is as follows:
[0074] This invention proposes a paraphrase generation method based on a controllable latent space diffusion model. By utilizing the continuity of text in the latent space, a diffusion model training and generation strategy that does not rely on rounding matching can be implemented. This not only eliminates rounding errors and improves generation quality, but also significantly improves the training and inference speed of the model. Furthermore, it can receive key semantic segments through an external controller to control the latent space denoising process, thereby achieving a stronger paraphrase generation effect.
[0075] The overall process of the paraphrase generation method based on the controllable latent space diffusion model is as follows: Figure 3 As shown.
[0076] Step S1: Train a paraphrase model based on a latent space diffusion model based on existing paraphrased sentences. This invention first requires constructing a paraphrase model based on a latent space diffusion model capable of generating target paraphrased sentences from source sentences. This model consists of two main parts: a latent space constructor and a diffusion model. The overall architecture and process of the model are as follows: Figure 1 As shown.
[0077] like Figure 1 As shown on the left side of section a, this invention uses a BART pre-trained model as the encoder-decoder structure to provide the latent space for the diffusion model. Given a text sequence x = x1, x2, ..., x... l The encoder of this invention encodes the input information into a d-dimensional latent space representation, i.e., z = E(x), and the decoder D reconstructs the input information using the latent space encoding according to this invention. During the training and inference processes of the model, both the encoder and decoder are frozen and not trained.
[0078] The training and inference process of the restatement model based on the latent space diffusion model is as follows: Figure 1 As shown on the right side of part a, for a restatement sentence...<x,y> This invention trains its paraphrase model based on a latent space diffusion model in an end-to-end manner. Specifically, during the forward pass, this invention progressively adds noise to the latent layer representation z of the target paraphrase until it becomes completely Gaussian noise z. T In the reverse process, the present invention trains the diffusion model z. θ (z, c, t) constructs the condition c = E(x) based on the source restatement and gradually removes noise to recover the original data. This invention uses cross-attention to input control information into the model.
[0079] The diffusion model z of the present invention θ Model architecture such as Figure 1 As shown in section b, this invention employs a Transformer model with pre-sequence layer normalization as its backbone network. Each layer consists of a self-attention module, a cross-attention module, and a feedforward layer. The time step t is first embedded as a d-dimensional vector and then fed into the model through an Adaptive-Layer Normalization module (Ada-LN). The model in this invention uses GeGLU as the activation function.
[0080] During training, this invention trains the diffusion model by having the model reconstruct the latent space representation of the restatement, with the specific loss function being: In this invention, it is necessary to base the noise sequence β t The data representation z with added noise is constructed using the corresponding noise time step t. t The specific formula is as follows This invention defines In this project, the present invention uses a cosine noise sequence to represent the noise sequence β of the present invention. t Furthermore, to ensure the model can generate fluent sentences, this invention employs sentence-level probabilistic conditional missingness to guarantee the model's ability to generate sentences independently of conditions. Specifically, this invention sets a conditional probability p, under which the original conditional sentence is replaced with a series of learnable empty semantic units. The model has a certain probability of generating paraphrased sentences instead of original conditional sentences, based on empty semantic sentences.
[0081] Step S2 involves dividing the sentence into key information and non-key information, constructing enhanced semantic fragment information. The latent space paraphrasing model based on controller semantic enhancement can enhance its paraphrasing ability by receiving key semantic fragments. The process of constructing the dataset in this invention is as follows: Figure 4 As shown.
[0082] Step S201: For an existing source sentence, the present invention samples the longest 15% of words in the sentence as key information.
[0083] Step S202: In this invention, the sentence is lexicalized, and the lexical units corresponding to non-keywords are replaced with special lexical units. <mask>This indicates that the encoder model needs to fill in semantic information here; the corresponding lexical units of the keywords are retained.
[0084] Step S203: The present invention uses an encoder model to encode the replaced sentence into a continuous latent space representation as a key semantic segment.
[0085] Step S3: Fix the existing latent space paraphrasing model, feed the original sentence into the original model, and feed the semantic fragments into the controller to train the controller. The controller model architecture and training / inference process are as follows: Figure 2 As shown.
[0086] Paraphrasing can be enhanced by learning key semantic fragments through the controller to achieve better generation results. This invention uses the same architecture as the original diffusion model as the controller. Specifically, this invention fixes the weights z of the pre-trained latent space diffusion model. θ Fine-tuning a trainable copy z of the model weights θ The inputs to the original model and the controller are fused through a zero-initialized convolutional layer, and the denoised representation of the output is also fused through a zero-initialized convolutional layer. This process can be defined as:
[0087]
[0088] Where c kw This refers to the controller input, specifically the semantic fragment latent space representation constructed in step S2. During fine-tuning, this invention uses key semantic fragments from the target end as the controller input.
[0089] Step S4: Combine the original paraphrase model and the controller for collaborative reasoning of the paraphrase sentence. During the reasoning process, this invention uses the source sentence as a semantic condition and key semantic fragments from the source as input to the controller, from the complete noise sequence z... T Initially, an iterative denoising process is performed under the guidance of conditional information, ultimately generating the latent space representation of the target restatement. Then, decoder D is used to transform the latent space representation into the target paraphrase. This invention uses the DPM-Solver++ sampler to generate the corresponding target paraphrase.
[0090] Example:
[0091] For the source sentence of a restatement, "how is black money going off with no longer the use of the same 500 and 1000 notes?", this invention constructs key semantic segments for it in step S2. <m> <m>black money <m> <m> <m> <m> <m> <m>longer <m> <m> <m> <m> <m> <m> 1000 <m> <m>",in <m>express <mask>; Collaborative reasoning is performed in step S4 to obtain the target restatement "how does banning 500 1000 rupeenotes solve black money problem?".
[0092] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a method for generating paraphrased statements based on a controllable latent space diffusion model, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0093] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MCU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.
[0094] This invention provides a method for generating paraphrased statements based on a controllable latent space diffusion model. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.< / mask> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / m> < / mask> < / mask>
Claims
1. A paraphrase sentence generation method based on a controllable latent space diffusion model, characterized in that, Includes the following steps: Step 1: Construct a paraphrase model based on the latent space diffusion model. Train the paraphrase model using an existing paraphrase text dataset. The existing paraphrase text dataset contains the original sentence and its paraphrase. Step 2: Divide the paraphrased source sentence into key information and non-key information, and construct enhanced semantic fragment information; Step 3, build the controller; Step 4, train the controller; Step 5: Combine the paraphrase model trained in Step 1 based on the latent space diffusion model and the controller trained in Step 4 to perform collaborative reasoning to generate paraphrased statements, thus completing the paraphrased statement generation task based on the controllable latent space diffusion model. The restatement model based on the latent space diffusion model mentioned in step 1 specifically includes: Step 1-1: Use the BART pre-trained model as the latent space encoding / decoding model to provide latent space encoding and decoding, as detailed below: Given a text sequence ,in, Represents a text sequence The first in One text; The encoder will output the text sequence Encoded as Latent space representation of dimensionality ,Right now: ； in, Indicates the encoding process; Decoder D is based on the above hidden space representation The reconstructed input information is as follows: ； in, This represents the reconstructed text; Steps 1-2: Construct the latent space diffusion model, as follows: Step 1-2-1: A Transformer model with normalized pre-sequence layers is used as the backbone network; each layer consists of a self-attention module, a cross-attention module, and a feedforward layer. Step 1-2-2: Introduce an adaptive layer normalization module into the Transformer model. Each input time step t is first transformed into a d-dimensional vector and then passed into the Transformer model through this adaptive layer normalization module. Steps 1-2-3: Use GeGLU as the activation function for the latent space diffusion model; Steps 1-3 involve generating a restatement, as detailed below: Step 1-3-1: Use the encoder in the latent space encoding / decoding model to perform latent space encoding on the input text sequence; Step 1-3-2: Iteratively call the latent space diffusion model to denoise the above latent space encoding; Step 1-3-3: Use the decoder in the latent space encoding and decoding model to decode the denoised latent space encoding and complete the paraphrase generation; The controller described in step 3 includes the same diffusion model as the restatement model based on the latent space diffusion model in step 1, i.e., a copy of the diffusion model. Zero-initialized convolutional modules are added before and after the copy of the diffusion model to fuse the input and output information of the two diffusion models.
2. The method for generating restatements based on a controllable latent space diffusion model according to claim 1, characterized in that, The restatement model based on the latent space diffusion model described in step 1 includes: a latent space encoding / decoding model and a latent space diffusion model, wherein, The latent space encoding and decoding model includes an encoder and a decoder, wherein the encoder is used to encode text into a latent space representation, and the decoder is used to decode the latent space representation into text; The aforementioned latent space diffusion model is used for iterative denoising of noisy latent space representations.
3. The method for generating restatements based on a controllable latent space diffusion model according to claim 2, characterized in that, The training of the restatement model based on the latent space diffusion model in step 1 specifically includes: Suppose a paraphrase pair in an existing paraphrase text dataset is... in, This represents the source text sequence, i.e., the original sentence or source sentence. express The restatement of the sentence; the training process includes forward and backward processes, as detailed below: During the forward pass, the latent space representation of the target restatement is... Based on cosine noise sequence Add noise gradually until it becomes pure Gaussian noise. That is, the above process is divided into several sub-processes, for the first... intermediate states , means as follows: ； Among them, the parameters are defined. , This indicates the sampling process with added noise. This represents noise that follows a normal distribution. Indicates the current time step The corresponding noise intensity; During the reverse process, the conditional encoding information of the source paraphrase is used. During the guided reverse process, the latent space diffusion model gradually removes noise, including Gaussian noise. Restore the original data to its latent space representation For each intermediate state The latent space diffusion model is trained to predict the original data through this intermediate state; The loss function used during training is as follows: ； in, This represents the number of noise steps sampled during model training; It is the raw representation of the data; It is a representation of the data after adding noise; It is a learnable latent space diffusion model.
4. The method for generating paraphrased statements based on a controllable latent space diffusion model according to claim 3, characterized in that, The construction of enhanced semantic fragment information described in step 2 specifically includes: Step 2-1: For the source sentence in the paraphrased sentence text dataset mentioned in Step 1, sample the longest preset number of words from the sentence as key information, and the other words as non-key information. Step 2-2: Lexicalization of the sentence, that is, replacing the lexical units corresponding to non-key information in the sentence with special lexical units. <mask>Key information and corresponding word units are retained;< / mask> Step 2-3: Using the encoder described in Step 1-1, the source sentence after lexicalization in Step 2-2 is encoded into a continuous latent space representation as enhanced semantic fragment information.
5. The method for generating restatements based on a controllable latent space diffusion model according to claim 4, characterized in that, The training controller mentioned in step 4, namely: Step 4-1: Fix the paraphrasing model based on the latent space diffusion model trained in Step 1, and input the original sentence into the paraphrasing model above; Step 4-2: Input the enhanced semantic fragment constructed in step 2 into the controller; Step 4-3: Using the zero-initialization convolution module in the controller, information is fused between the controller and the restatement model based on the latent space diffusion model at the input and output stages to obtain the final output. The details are as follows: ； in, This represents the restatement model trained in step 1 based on the latent space diffusion model. This indicates a copy of the diffusion model in the controller; This indicates a zero-initialized convolutional module where both weights and offsets are initialized to zero. This indicates a noisy intermediate state. This represents the conditional encoding information of the source paraphrase. Indicates the current time step of the controller. This represents the latent space representation of the enhanced semantic fragment information constructed in step 2; Step 4-4: Define the loss function to train the controller during the above process.
6. The method for generating restatements based on a controllable latent space diffusion model according to claim 5, characterized in that, The loss function described in step 4-4 is: 。 7. The method for generating restatements based on a controllable latent space diffusion model according to claim 6, characterized in that, The collaborative reasoning described in step 5 specifically includes: Step 5-1: Using the source sentence in the paraphrased sentence pair as a semantic condition, and using the enhanced semantic fragment as the input to the controller, from the complete noise sequence... Initially, an iterative denoising process is performed under the guidance of conditional information, ultimately generating the latent space representation of the target restatement. ; Step 5-2: Use decoder D to transform the latent space representation into the target paraphrase.
8. The method for generating paraphrased statements based on a controllable latent space diffusion model according to claim 7, characterized in that, The denoising process described in step 5-1 is accelerated iteratively using a diffusion model sampler based on ordinary differential equations.