Artificial intelligence-based copywriting generation method, device, equipment and medium

Abstract

The present application relates to the fields of artificial intelligence, finance, and medical technology, and discloses a method, apparatus, device, and medium for generating text based on artificial intelligence, including: receiving an original text theme input by a user; generating a first feature vector of m tokens in the original text theme based on the original text theme, where m is a positive integer; generating an nth generated text based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text using a segmented recurrent cross-attention mechanism, where the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m; and concatenating the m-th generated text and the m-1th concatenated text, and outputting the result as a target text. The method saves storage hardware costs and achieves text generation without sacrificing model effect and model accuracy.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence technology, financial technology, and medical health, and in particular to an artificial intelligence-based copywriting generation method, device, equipment, and medium. Background Art

[0002] With the rapid development of artificial intelligence (AI), copywriting generation has become a hot topic. This technology uses natural language processing (NLP) and machine learning algorithms to automatically generate grammatically correct content. Currently, copywriting generation technology is widely used in advertising, marketing, insurance, finance, healthcare, and other fields.

[0003] However, the accuracy of current copywriting generation models is generally not high and requires a large number of model parameters. Summary of the Invention

[0004] The present application provides an artificial intelligence-based copywriting generation method, apparatus, computer equipment, and medium to solve the technical problems of low accuracy of copywriting generation models and a large number of model parameters.

[0005] In a first aspect, a copy generation method based on artificial intelligence is provided, which is applied to a copy generation model, and the method includes: receiving an original copy theme input by a user; generating a first feature vector of m tokens in the original copy theme based on the original copy theme, where m is a positive integer; based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth spliced ​​text, a segmented recurrent cross-attention mechanism is used to generate an nth generated text, where the nth spliced ​​text is determined based on the n-1th generated text and the n-1th spliced ​​text, where n is a positive integer less than or equal to m; splicing the mth generated text and the m-1th spliced ​​text, and outputting them as a target copy.

[0006] In a second aspect, a copywriting generation device based on artificial intelligence is provided, comprising:

[0007] A receiving module, used to receive the original copy theme input by the user;

[0008] A generating module, configured to generate first feature vectors of m tokens in the original copy subject based on the original copy subject, where m is a positive integer;

[0009] The generation module is further configured to generate an n-th generated text based on the first feature vector of the n-th token in the m tokens and the second feature vector of the n-th concatenated text using a segmented recurrent cross attention mechanism, wherein the n-th concatenated text is determined based on the n-1-th generated text and the n-1-th concatenated text, where n is a positive integer less than or equal to m;

[0010] The output module is used to splice the mth generated text and the m-1th spliced ​​text, and output them as the target text.

[0011] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned artificial intelligence-based copywriting generation method when executing the computer program.

[0012] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned artificial intelligence-based copywriting generation method are implemented.

[0013] In the solution implemented by the above-mentioned artificial intelligence-based copywriting generation method, device, computer equipment and storage medium, the user inputs the original copywriting theme into the copywriting generation model. When the copywriting generation model generates the n-th generated text, it uses a segmented cyclic cross-attention mechanism based on the feature vector of the token in the copywriting theme and the feature vector of the n-th spliced ​​text to generate the generated text this time. The n-th spliced ​​text is determined based on the n-1-th generated text and the n-1-th spliced ​​text, and the cycle is repeated until the copywriting is generated. The embodiment of the present application saves the cost of storage hardware without losing the model effect and model accuracy by adopting the segmented cyclic cross-attention mechanism to achieve copywriting generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 This is a schematic diagram of an application environment of a method for generating text based on artificial intelligence in an embodiment of the present application;

[0016] Figure 2 This is a flow chart of a method for generating text based on artificial intelligence in one embodiment of the present application;

[0017] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S20;

[0018] Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S30;

[0019] Figure 5 1 is a schematic diagram of the calculation flow of the RAF module in one embodiment of the present application;

[0020] Figure 6 This is a flow chart of a training method based on a copywriting generation model in one embodiment of the present application;

[0021] Figure 7 yes Figure 6 A schematic flow chart of a specific implementation of step S70;

[0022] Figure 8 This is a structural diagram of a text generation device based on artificial intelligence in one embodiment of the present application;

[0023] Figure 9 is a structural diagram of a computer device in one embodiment of the present application;

[0024] Figure 10 It is another structural diagram of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] The artificial intelligence-based copywriting generation method provided in the embodiment of the present application can be applied in Figure 1In an application environment, the client communicates with the server through a network. A copy generation model is provided in the server, and the server can receive the copy theme input by the user through the client, and input the copy theme into the copy generation model. The copy generation model generates the first feature vector of the m tokens in the copy theme based on the copy theme, and based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth spliced ​​text, a segmented cyclic cross-attention mechanism is used to generate the nth generated text, and the mth generated text and the m-1th spliced ​​text are spliced ​​and output as the target copy, so that the user can obtain the target copy corresponding to the copy theme through the client. In this application, it can be applied to the generation of marketing copy for life insurance and property insurance. The user inputs a theme related to marketing, and the model generates marketing copy within the scope of the currently input theme, which can improve the office efficiency of marketing personnel and save mechanical expression of copy. At the same time, the content of the copy can be innovated through a large model to achieve marketing effects. In addition, this application has low requirements for the hardware platform for model operation, which can reduce related investment costs. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server can be implemented as a standalone server or a server cluster consisting of multiple servers. The present application is described in detail below using specific embodiments.

[0027] The industry uses the transformer architecture as the codec for dialogue generation, primarily utilizing the multi-head self-attention (MHA) mechanism. With the widespread adoption of large language models, such as ChatGPT, models with varying parameter counts ranging from over 6 billion to over 170 billion are being used. The majority of these parameters are set during the MHA process. The number of parameters is strongly correlated with the number of heads in the MHA: the greater the number of heads, the greater the number of parameters in the model. When training large language models, graphics processing units (GPUs) are often used as computing resources rather than standard central processing units (CPUs) to accelerate training cycles. GPUs offer superior matrix and floating-point computing capabilities. However, GPU memory is not as expandable as CPU memory. For example, a CPU host may have 128GB of memory, while a GPU typically has 8GB or 16GB. The latest GPUs even have 64GB of memory, but the price is several times higher. GPU memory usage is primarily driven by the number of model parameters. The larger the number of model parameters, the more memory is consumed. A large model with a massive number of parameters can learn more complex tasks than a smaller model, and its performance is also superior. This is particularly true for dialogue generation tasks, such as chatGPT, which can handle a variety of generation tasks, including dialogue, reasoning, reading comprehension, and sorting, using a single model. Therefore, simply reducing the number of parameters in a large model to reduce hardware costs is not an option; ensuring model performance is paramount.

[0028] The embodiments of the present application can greatly save the cost of storage hardware. The copywriting generation model involved in this application is based on a segmented recursive transformer model structure, which realizes the generation of copywriting without losing the model effect and model accuracy.

[0029] Among them, the Segmented Recurrent Transformer (SRformer) model. SRformer aims to overcome the high computational cost of traditional transformers when processing long sequences. The computational complexity of traditional transformers is quadratic with the length of the sequence, which limits its use in resource-constrained applications. SRformer achieves efficient processing of sequence information by dividing the entire input sequence into multiple segments and using recurrent neurons to aggregate information between segments. This approach not only retains the ability to process sequences but also significantly reduces computational and memory requirements. Specifically, SRformer uses a local attention mechanism to process each segment, combined with recurrent attention, and a recurrent accumulation-and-fire (RAF) layer to handle the flow of information between consecutive segments. The RAF layer updates the key-value product to compensate for the information loss caused by reducing the length of the attention window.

[0030] Segmented attention is a key component of SRformer, designed to reduce the computational complexity of sequence-to-sequence models. SRformer also proposes recurrent attention. Because directly computing the remaining partial attention results in the same computational complexity as a non-segmented model, recurrent attention first computes the product of the key and value, and then multiplies it by the query. The RAF layer in SRformer bridges the gap between segmented and recurrent attention to enable efficient cross-segment information propagation. The RAF layer's main function is to accumulate and filter the partial products of key-value pairs from different segments, helping to approximate the softmax operation and exclude unnecessary information. It is designed to mimic the global information integration capabilities of traditional attention mechanisms in segmented sequence processing while reducing computational effort and memory usage.

[0031] This application proposes a Transformer model based on segmented recurrent cross-attention to replace the traditional Transformer multi-head self-attention method. While maintaining the accuracy of the original model, it significantly reduces the number of model parameters and reduces the cost of GPU training. The RAF module is also introduced, using an approximate attention mechanism instead of a global attention mechanism to achieve the goal of reducing parameters.

[0032] See also Figure 2 As shown, Figure 2 A flowchart of the method for generating text based on artificial intelligence provided in an embodiment of the present application includes the following steps:

[0033] S10: Receive the original copy theme input by the user.

[0034] The AI-based copywriting generation method provided in this application can be applied to copywriting in various application scenarios. Copywriting generation is typically implemented through a server that can receive user-entered copywriting topics in real time. For example, in the insurance marketing application field, users can enter marketing topics, and the model will generate marketing copy within the scope of the currently entered topic. This can improve the work efficiency of marketers and reduce the mechanical expression of copywriting.

[0035] For example, in the field of insurance marketing, for life insurance, the copywriting theme can be "long-term protection for the family"; for car insurance, the copywriting theme can be "a safe driving companion"; in the field of medical health, the copywriting theme can be "life and hope", "health and care", "how to prevent diabetes", etc.

[0036] S20: Generate first feature vectors of m tokens in the original copy subject based on the original copy subject, where m is a positive integer.

[0037] A token is the smallest semantic unit in a text. Word segmentation is used to break text into tokens, which can be words, punctuation marks, or phrases. For example, "long-term security for the family" can be broken down into three phrases: "family," "long-term," and "security."

[0038] After obtaining m tokens in the original copy theme, the feature vector of each token is obtained by encoding the m tokens. Specifically, in some embodiments of the present application, such as Figure 3 As shown, a specific feature vector generation solution is provided, including the following steps S21-S24:

[0039] S21: Segment the original copy topic to obtain m tokens.

[0040] Among them, there are many word segmentation methods, such as defining word segmentation rules based on regular expressions, word segmentation based on dictionaries, word segmentation based on machine learning or deep learning, etc., or word segmentation based on word segmentation tools, which are not limited in the embodiments of this application.

[0041] S22: Input m tokens into the Transformer encoder and output the initial feature vector corresponding to each token.

[0042] In the Transformer encoder, the input token is first converted into the corresponding embedding vector, and the embedding vector is input into the encoder layer. After processing by the encoder layer, each token is converted into a feature vector.

[0043] S23: Based on the initial feature vector corresponding to each token, determine the first feature vector of each token, where the first feature vector includes a first sub-vector and a second sub-vector.

[0044] Specifically, the tokens after word segmentation are input into the Transformer encoder for feature encoding. Each token corresponds to an initial feature vector of dimension d. This feature vector is replicated three times as Q, K, and V, with K as the first subvector and V as the second subvector. The first subvector is also called the first key vector, and the second subvector is also called the first value vector.

[0045] S30: Based on the first feature vector of the nth token among the m tokens and the second feature vector of the nth concatenated text, a segmented recurrent cross-attention mechanism is used to generate the nth generated text.

[0046] The nth concatenated text is determined based on the generated text from the n-1th time and the concatenated text from the n-1th time, where n is a positive integer less than or equal to m. It is worth noting that when n is 1, since no text is generated, the first concatenated text is a special character token, indicating the beginning. The special character token can be customized by the technician.

[0047] For example, the second concatenated text is determined based on the first generated text and the first concatenated text. If the first concatenated text is a special character token, the second concatenated text is obtained by concatenating the first generated text and the special character token.

[0048] In some embodiments, since the concatenated text is also composed of tokens, the feature vector of the concatenated text can be determined by determining the feature vectors of the tokens in the concatenated text. Specifically, the second feature vector of the n-th concatenated text is determined based on the following method: determining the initial feature vectors of the tokens in the n-th concatenated text; and calculating the second feature vector of the n-th concatenated text based on the initial feature vectors of the tokens in the n-th concatenated text and the self-attention mechanism.

[0049] Specifically, in the self-attention mechanism, the initial feature vector of each token is replicated three times (or linearly transformed) to form Q, K, and V. For each token in the sequence, the dot product between its Q vector and the K vectors of all other tokens is calculated to obtain an attention score matrix. The attention scores are normalized using the softmax function so that the sum of each row is 1. The normalized attention weights are multiplied by the corresponding V vector, and the weighted V vectors of all tokens are summed to obtain the final output vector. The self-attention machine is usually implemented in a multi-head form, that is, the above process is replicated multiple times (usually 8 heads or more), and each head has its own Q, K, and V matrices. The representation learned by each head is different. Finally, the outputs of all heads are concatenated, processed by a linear layer, and residually connected with the input initial feature vector. Then, layer normalization is performed to obtain the second feature vector of the concatenated text.

[0050] In the embodiment of the present application, the second feature vector of the n-th concatenated text can also be called a Q vector. That is, step S30 can be described as generating the n-th generated text based on the K vector and V vector of the n-th token in the m tokens and the Q vector of the n-th concatenated text. Specifically, in some embodiments of the present application, such as Figure 4 As shown, a method for generating a generated text is provided, comprising the following steps S31-S33:

[0051] S31: Input the first sub-vector, second sub-vector and second eigenvector of the nth token into the segmented recurrent cross attention mechanism and output the score matrix.

[0052] S32: Input the score matrix into the feedforward network and output the probability of the associated words.

[0053] S33: Select the associated word with the highest probability as the generated text for the nth time.

[0054] In an embodiment of the present application, attention is calculated based on the second feature vector (Q), the first sub-vector (K) and the second sub-vector (V) of the input through a segmented recurrent cross-attention mechanism, and the traditional attention calculation is replaced by approximate attention (RAF).

[0055] Specifically, after multiplying the K and V vectors, a new vector Ht is obtained as the input of RAF. The output of RAF is calculated with Q and divided by the square root of the dimension of vector K, which serves as the softmax normalization factor, ultimately resulting in the score matrix QR[t]. QR[t] is then used as the input of the feedforward network. The output of the feedforward network is the probability of all words, and the word with the highest probability is taken as the token for the nth generated text.

[0056] The calculation process of RAF is as follows: Figure 5 As shown, RAF includes the following key components: Linear layer: The input is first projected through a linear layer, which is needed to adjust the scale of the input. Leaky memory: RAF does not apply weights to hidden memories. Instead, it uses a learnable leakage coefficient and threshold. The leakage coefficient controls the speed at which information "leaks out", while the threshold acts as a bias to activate values ​​above this threshold. Soft-reset: Once the hidden memory triggers "excitation", it is soft-reset by subtracting the threshold value, which is similar to the reset mechanism of the Integrate-and-Fire neuron. ReLU activation: The output finally passes through the ReLU activation function to ensure that only positive values ​​pass through.

[0057] Specifically, the RAF calculation process is as follows: the input Ht is projected to Qt through a linear layer. mem is the memory state, which is updated to leak * mem + Qt. The output y is calculated as mem / threshold - 1.0. If y is greater than 0, mem is subtracted by the threshold value to implement a soft reset. The final output of the RAF is obtained through the ReLU activation function.

[0058] S40: Splicing the m-th generated text and the m-1-th spliced ​​text, and outputting the result as the target text.

[0059] In the embodiment of the present application, after step 30, the nth generated text is obtained, the nth generated text is spliced ​​with the n-th spliced ​​text to obtain the n+1th spliced ​​text, and the n+1th generated text is generated based on the K vector and V vector of the n+1th token and the Q vector of the n+1th spliced ​​text. This cycle is repeated until the copy generation model generates the mth generated text based on the m tokens in the original copy theme, and the mth generated text and the m-1th spliced ​​text are spliced ​​and output as the target copy.

[0060] Based on the above embodiments, in order to more clearly illustrate the copywriting generation method provided by the embodiments of the present application, as shown in FIG. Figure 6The figure shows the architecture of the copywriting generation model, which includes a Transformer encoder, a segmented recurrent cross-attention mechanism, a feedforward network, a self-attention mechanism, and a text generation module. Specifically, the user inputs the original copywriting topic into the copywriting generation model. After tokenization, the Transformer encoder generates the first feature vector for each token. Based on this first feature vector, a first subvector (K) and a second subvector (V) are generated. The generated text token output by the feedforward network is concatenated with the generated text output from the past to obtain the second feature vector (Q) of the concatenated text. The first subvector (K), second subvector (V), and second feature vector (Q) are input into the segmented recurrent cross-attention mechanism to obtain a score matrix QR[t]. The score matrix QR[t] is processed by the feedforward network and the probabilities of all words are output. The text generation module selects the word with the highest probability as the generated text for this time. The generated text tokrn is then concatenated with the concatenated text from the past. The feature vector of the concatenated text serves as the second feature vector (Q) for the next segmented recurrent cross-attention mechanism. Repeat this cycle until the text generation is complete.

[0061] In this application, a Transformer model based on segmented recurrent cross-attention is proposed to replace the traditional Transformer multi-head self-attention method. While maintaining the accuracy of the original model, it significantly reduces the number of model parameters and reduces the cost of GPU training. At the same time, the RAF module is introduced to replace the global attention mechanism with an approximate attention mechanism to achieve the goal of reducing parameters.

[0062] In some embodiments of the present application, a method for training a copywriting generation model is provided, such as Figure 6 As shown, the following steps are included:

[0063] Step S50: training the copywriting generation model based on the sample set to obtain a trained initial copywriting generation model.

[0064] Step S60: Copy the initial copy generation model to obtain a copy generation model. The copy generation model includes a Transformer encoder, a self-attention mechanism, a segmented recurrent cross-attention mechanism, a feedforward network, and a Deep Q-Networks (DQN) network.

[0065] Because it's a generative model, the quality of the generated copy is subject to a certain degree of randomness, and cumulative errors can occur. For example, if a word is generated incorrectly, subsequent generation processes will continue based on the current error information, resulting in cumulative errors in subsequent generated copy. To address this issue, the present embodiment adds a reinforcement learning module (DQN network) that guides the generated results and improves the quality of the copy generated by the model by only changing the objective function at the end of the model.

[0066] Step S70: training the copywriting generation and replication model using the sample set, and updating the model parameters of the copywriting generation and replication model.

[0067] In some embodiments of the present application, Figure 7 As shown, a specific method for updating model parameters is provided, including the following steps S71-S74:

[0068] S71: The word vectors of the words generated by the feedforward network at each moment are combined into a word vector matrix.

[0069] S72: Input the word vector matrix into the deep Q network and output the quality score.

[0070] S73: Obtain manual scoring results for the generated words.

[0071] S74: Update the model parameters of the replication model based on the manual scoring results and the quality score.

[0072] Specifically, the DQN network input is the vector before the model generates a word (not the word that has already been generated). The model records each word generated at a given moment and matrices the corresponding vector (not the word) to form a word vector matrix N*d, where N is the number of generated words and d is the dimension of the current word vector. This N*d matrix is ​​used as the input to the DQN network, which outputs a quality score. This requires manual intervention, that is, manually scoring the output quality score, with high quality being 1 and low quality being 0. Because the model records all generated words, a human can review the results of the recorded words. If the results meet expectations, the intervention manually scores them as high quality; otherwise, they score them as low quality.

[0073] Furthermore, after the DQN network obtains the manual scoring results, it calculates the loss value based on the manual scoring results and the quality score, such as the MSE loss, and backpropagates the gradient value of the loss value to modify the model parameters of the copy model.

[0074] Step S80: copying the model parameters of the copy copy model to the initial copy generation model to obtain a target copy generation model.

[0075] In some embodiments, after iterations of reinforcement learning, the model parameters of the copy generation model and the initial copy generation model differ. In this case, the model parameters of the copy generation model are copied to the initial copy generation model to update the initial copy generation model. After the update, the initial copy generation model is used to generate new results by inputting training data. This new model is then cloned, and the results from this batch of data are used for manual training to train the clone model. This process is repeated until data quality meets requirements.

[0076] In an embodiment of the present application, a text generation model is copied, a DQN network is added to the copied model, and the copied model is updated through the DQN network. When the model quality meets certain requirements, the parameters of the copied model are synchronized to the text generation model, thereby improving the quality of the text generated by the model.

[0077] It is worth noting that the above embodiment introduces the introduction of the DQN network in the training process of the text generation model. In other embodiments of the present application, the DQN network can also be introduced in the actual application process of the text generation model. When the text is actually generated, the text generation model is updated in real time to generate more accurate text.

[0078] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0079] In one embodiment, a text generation device based on artificial intelligence is provided, and the text generation device based on artificial intelligence corresponds to the text generation method based on artificial intelligence in the above embodiment. Figure 8 As shown, the artificial intelligence-based copywriting generation device includes a receiving module 101, a generating module 102, and an output module 103. The detailed description of each functional module is as follows:

[0080] Receiving module 101, for receiving the original copy theme input by the user;

[0081] A generating module 102 is configured to generate first feature vectors of m tokens in the original copy subject based on the original copy subject, where m is a positive integer;

[0082] The generating module 102 is further configured to generate an n-th generated text based on the first feature vector of the n-th token among the m tokens and the second feature vector of the n-th concatenated text, wherein the n-th concatenated text is determined based on the n-1-th generated text and the n-1-th concatenated text, where n is a positive integer less than or equal to m;

[0083] The output module 103 is used to splice the m-th generated text and the m-1-th spliced ​​text, and output them as the target text.

[0084] In some embodiments, the generation module 102 is specifically used to segment the original text topic to obtain m tokens; input the m tokens into the Transformer encoder and output the initial feature vector corresponding to each token; based on the initial feature vector corresponding to each token, determine the first feature vector of each token, and the first feature vector includes a first sub-vector and a second sub-vector.

[0085] In some embodiments, the second feature vector of the nth spliced ​​text is determined based on the following method: determining the initial feature vector of the token in the nth spliced ​​text; calculating the second feature vector of the nth spliced ​​text based on the initial feature vector of the token in the nth spliced ​​text and the self-attention mechanism.

[0086] In some embodiments, the generation module 102 is specifically used to input the first sub-vector, the second sub-vector and the second eigenvector of the nth token into the segmented recurrent cross-attention mechanism to output a score matrix; input the score matrix into the feedforward network to output the probability of the associated words; and select the associated word with the highest probability as the generated text for the nth time.

[0087] In some embodiments, the copy generation model is trained and generated based on the following method: the copy generation model is trained based on the sample set to obtain a trained initial copy generation model; the initial copy generation model is copied to obtain a copy generation model, and the copy generation model includes a Transformer encoder, a self-attention mechanism, a segmented recurrent cross-attention mechanism, a feedforward network, and a deep Q network; the copy generation model is trained through the sample set to update the model parameters of the copy generation model; the model parameters of the copy generation model are copied to the initial copy generation model to obtain a target copy generation model.

[0088] In some embodiments, the update module 104 is used to form a word vector matrix by combining the word vectors of the words generated by the feedforward network at each moment; input the word vector matrix into the deep Q network and output a quality score; obtain manual scoring results of the generated words; and update the model parameters of the replication model based on the manual scoring results and the quality score.

[0089] In some embodiments, the updating module 104 is specifically configured to calculate a loss value based on the manual scoring result and the quality score; and modify the model parameters of the replication model based on the gradient value of the loss value.

[0090] The present application provides an artificial intelligence-based copywriting generation device. The user inputs the original copywriting theme into the copywriting generation model. When the copywriting generation model generates the nth generated text, it uses a segmented cyclic cross-attention mechanism based on the feature vector of the token in the copywriting theme and the feature vector of the nth spliced ​​text to generate the generated text this time. The nth spliced ​​text is determined based on the n-1th generated text and the n-1th spliced ​​text, and the cycle repeats until the copywriting is generated. By adopting the segmented cyclic cross-attention mechanism, the embodiment of the present application saves the cost of storage hardware without sacrificing the model effect and model accuracy, and realizes the generation of copywriting.

[0091] For the specific definition of the artificial intelligence-based copy generation device, please refer to the definition of the artificial intelligence-based copy generation method above, which will not be repeated here. The various modules in the above-mentioned artificial intelligence-based copy generation device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0092] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a text generation method based on artificial intelligence.

[0093] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 10As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the client side of a method for generating text based on artificial intelligence.

[0094] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0095] Receive the original copy topic input by the user;

[0096] Generate first feature vectors of m tokens in the original copy subject based on the original copy subject, where m is a positive integer;

[0097] Generate an nth generated text based on the first feature vector of the nth token among the m tokens and the second feature vector of the nth concatenated text, wherein the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m;

[0098] The generated text of the mth time and the spliced ​​text of the m-1th time are spliced ​​together and output as the target text.

[0099] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0100] Receive the original copy topic input by the user;

[0101] Generate first feature vectors of m tokens in the original copy subject based on the original copy subject, where m is a positive integer;

[0102] Generate an nth generated text based on the first feature vector of the nth token among the m tokens and the second feature vector of the nth concatenated text, wherein the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m;

[0103] The generated text of the mth time and the spliced ​​text of the m-1th time are spliced ​​together and output as the target text.

[0104] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0105] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0106] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0107] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

[0108] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A copywriting generation method based on artificial intelligence, characterized in that: Applied to a copywriting generation model, the method includes: Receive the original copy topic input by the user; Generate first feature vectors of m tokens in the original copy subject based on the original copy subject, where m is a positive integer; Based on the first feature vector of the nth token among the m tokens and the second feature vector of the nth concatenated text, a segmented recurrent cross attention mechanism is used to generate the nth generated text, wherein the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m; The generated text of the mth time and the spliced ​​text of the m-1th time are spliced ​​together and output as the target text; The step of generating first feature vectors of m tokens in the original copy subject based on the original copy subject includes: Segment the original copy topic to obtain m tokens; Input m tokens into the Transformer encoder and output the initial feature vector corresponding to each token; Determine a first eigenvector for each token based on an initial eigenvector corresponding to each token, where the first eigenvector includes a first subvector and a second subvector; The method of generating the nth generated text based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text by using a segmented recurrent cross attention mechanism includes: Input the first subvector, the second subvector, and the second eigenvector of the nth token into the segmented recurrent cross attention mechanism and output a score matrix; Input the score matrix into a feedforward network and output the probability of the associated words; Select the associated word with the highest probability as the generated text for the nth time.

2. The method according to claim 1, characterized in that The second feature vector of the n-th concatenated text is determined based on the following method: Determine the initial feature vector of the token in the n-th concatenated text; Based on the initial feature vector of the token in the n-th concatenated text and the self-attention mechanism, a second feature vector of the n-th concatenated text is calculated.

3. The method according to claim 1, characterized in that The copywriting generation model is trained and generated based on the following method: The copywriting generation model is trained based on the sample set to obtain a trained initial copywriting generation model; The initial copy generation model is copied to obtain a copy generation model, wherein the copy generation model includes a Transformer encoder, a self-attention mechanism, a segmented recurrent cross attention mechanism, a feedforward network, and a deep Q network; Training the copywriting generation and replication model using a sample set, and updating model parameters of the copywriting generation and replication model; The model parameters of the copywriting generation model are copied to the initial copywriting generation model to obtain a target copywriting generation model.

4. The method according to claim 3, characterized in that The step of training the copywriting generation and replication model using the sample set and updating the model parameters of the copywriting generation and replication model includes: The word vectors of the words generated by the feedforward network at each moment are combined into a word vector matrix; Input the word vector matrix into the deep Q network and output the quality score; Obtain manual scoring results for generated words; The model parameters of the replication model are updated based on the manual scoring results and the quality score.

5. The method according to claim 4, characterized in that The updating of the model parameters of the replication model based on the manual scoring results and the quality score includes: Calculating a loss value based on the manual scoring result and the quality score; The model parameters of the copied model are modified based on the gradient value of the loss value.

6. A copywriting generation device based on artificial intelligence, characterized in that: include: A receiving module is used to receive the original copy theme input by the user; A generating module, configured to generate first feature vectors of m tokens in the original copy subject based on the original copy subject, where m is a positive integer; The generation module is further configured to generate an n-th generated text based on the first feature vector of the n-th token in the m tokens and the second feature vector of the n-th concatenated text using a segmented recurrent cross attention mechanism, wherein the n-th concatenated text is determined based on the n-1-th generated text and the n-1-th concatenated text, where n is a positive integer less than or equal to m; The output module is used to splice the generated text of the mth time and the spliced ​​text of the m-1th time, and output it as the target copy; A generation module is specifically used to segment the original copy topic and obtain m tokens; Input m tokens into the Transformer encoder and output the initial feature vector corresponding to each token; Determine a first eigenvector for each token based on an initial eigenvector corresponding to each token, where the first eigenvector includes a first subvector and a second subvector; Input the first subvector, the second subvector, and the second eigenvector of the nth token into the segmented recurrent cross attention mechanism and output a score matrix; Input the score matrix into a feedforward network and output the probability of the associated words; Select the associated word with the highest probability as the generated text for the nth time.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the artificial intelligence-based copywriting generation method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the artificial intelligence-based copywriting generation method according to any one of claims 1 to 5 are implemented.

Images