Artificial intelligence-based copywriting generation method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, finance and medical treatment, and discloses a copywriting generation method and device based on artificial intelligence, equipment and a medium, and the method comprises the steps: receiving an original copywriting theme input by a user; generating a first feature vector of m tokens in the original copywriting theme based on the original copywriting theme, wherein m is a positive integer; based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth spliced text, an nth generated text is generated by adopting a segmented loop cross attention mechanism, the nth spliced text is determined based on the (n-1) th generated text and the (n-1) th spliced text, and n is a positive integer smaller than or equal to m; and splicing the mth generated text and the (m-1) th spliced text, and outputting the spliced text as a target copywriting. On the premise that the model effect and the model precision are not lost, the cost of hardware storage is saved, and copywriting generation is achieved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence technology, financial technology, and medical health, and in particular to a method, device, equipment, and medium for generating text based on artificial intelligence. Background Art

[0002] With the rapid development of artificial intelligence technology, copywriting generation has gradually become a focus of attention. Copywriting generation refers to the use of natural language processing technology and machine learning algorithms to allow computers to automatically generate grammatically correct copywriting content. At present, copywriting generation technology is widely used in advertising, marketing, insurance, finance, medical care, health and other fields.

[0003] However, the accuracy of current copywriting generation models is generally not high and requires a large number of model parameters. Summary of the invention

[0004] The present application provides a method, device, computer equipment and medium for generating text based on artificial intelligence to solve the technical problems of low accuracy of text generation model and large number of model parameters.

[0005] In a first aspect, a copy generation method based on artificial intelligence is provided, which is applied to a copy generation model, and the method includes: receiving an original copy theme input by a user; generating a first feature vector of m tokens in the original copy theme based on the original copy theme, where m is a positive integer; based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text, using a segmented recurrent cross-attention mechanism to generate an nth generated text, the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m; concatenating the mth generated text and the m-1th concatenated text, and outputting them as a target copy.

[0006] In a second aspect, a copywriting generation device based on artificial intelligence is provided, comprising:

[0007] A receiving module, used for receiving the original copy theme input by the user;

[0008] A generating module, used for generating first feature vectors of m tokens in the original copy theme based on the original copy theme, where m is a positive integer;

[0009] The generation module is further used to generate an nth generated text based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text by using a segmented recurrent cross attention mechanism, wherein the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m;

[0010] The output module is used to splice the m-th generated text and the m-1-th spliced ​​text, and output them as the target text.

[0011] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned artificial intelligence-based copywriting generation method when executing the computer program.

[0012] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned artificial intelligence-based copywriting generation method are implemented.

[0013] In the scheme implemented by the above-mentioned artificial intelligence-based copywriting generation method, device, computer equipment and storage medium, the user inputs the original copywriting theme into the copywriting generation model. When the copywriting generation model generates the nth generated text, it uses a segmented cyclic cross-attention mechanism based on the feature vector of the token in the copywriting theme and the feature vector of the nth spliced ​​text to generate the generated text this time. The nth spliced ​​text is determined based on the n-1th generated text and the n-1th spliced ​​text, and the cycle is repeated until the copywriting is generated. The embodiment of the present application saves the cost of storage hardware and realizes the generation of copywriting by adopting a segmented cyclic cross-attention mechanism without losing the model effect and model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0015] Figure 1 It is a schematic diagram of an application environment of a method for generating text based on artificial intelligence in one embodiment of the present application;

[0016] Figure 2 It is a flow chart of a method for generating text based on artificial intelligence in one embodiment of the present application;

[0017] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S20;

[0018] Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S30;

[0019] Figure 5 is a schematic diagram of the calculation flow of the RAF module in one embodiment of the present application;

[0020] Figure 6 It is a flow chart of a training method based on a copywriting generation model in one embodiment of the present application;

[0021] Figure 7 yes Figure 6 A schematic flow chart of a specific implementation of step S70;

[0022] Figure 8 It is a structural schematic diagram of a text generation device based on artificial intelligence in one embodiment of the present application;

[0023] Fig. 9 is a structural schematic diagram of a computer device in one embodiment of the present application;

[0024] Fig.10 It is another structural schematic diagram of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The method for generating text based on artificial intelligence provided in the embodiment of the present application can be applied in the following aspects: Figure 1In an application environment, the client communicates with the server through a network. A copy generation model is provided in the server, and the server can receive the copy theme input by the user through the client, and input the copy theme into the copy generation model. The copy generation model generates the first feature vector of m tokens in the copy theme based on the copy theme, and based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text, a segmented cyclic cross-attention mechanism is used to generate the nth generated text, and the mth generated text and the m-1th concatenated text are concatenated and output as the target copy, so that the user can obtain the target copy corresponding to the copy theme through the client. In this application, it can be applied to the generation of marketing copy for life insurance and property insurance. The user inputs the theme related to marketing, and the model generates marketing copy within the scope of the currently input theme, which can improve the office efficiency of marketing personnel and save the mechanical expression of the copy. At the same time, the content of the copy can be innovated through the large model to achieve marketing effects. In addition, this application has low requirements on the hardware platform for the operation of the model, which can reduce related investment costs. The client may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server may be implemented by an independent server or a server cluster consisting of multiple servers. The present application is described in detail below through specific embodiments.

[0027] The industry uses the transformer structure as the codec for dialogue generation, and the main mechanism used in the structure is the multi-head self-attention mechanism (MHA). With the widespread application of large language models, such as chatGPT, various versions of a model with parameters ranging from more than 6 billion to more than 170 billion are being used, and the main parameter quantities are set during the MHA process. The number of parameters is strongly related to the number of heads in MHA. The more heads there are, the more parameters a model has. When training a large language model, in order to speed up the training cycle, a graphics processing unit (GPU) is used as a computing resource instead of an ordinary central processing unit (CPU), because the GPU has stronger matrix and floating-point computing capabilities. However, the GPU video memory is not like the CPU memory that can be expanded at will. For example, the memory of a CPU host can have 128G, but the video memory of a GPU is generally 8G or 16G. The latest GPU also has 64G video memory, but the price is dozens of times that of the memory. The GPU memory usage is mainly due to the number of parameters in the model. The larger the number of model parameters, the larger the memory usage. A large model with a large number of parameters can learn more complex tasks than an ordinary small model, and the effect of use will be better, especially in the task of dialogue generation, such as chatGPT, which can complete various generation tasks such as dialogue, reasoning, reading comprehension, sorting, etc. with only one model. Therefore, the number of parameters of a large model cannot be arbitrarily reduced to reduce hardware costs, but the main premise is to ensure the model effect.

[0028] The embodiments of the present application can greatly save the cost of storage hardware. The copywriting generation model involved in the present application is based on a piecewise recursive transformer model structure, which realizes the generation of copywriting without losing the model effect and model accuracy.

[0029] Among them, the Segmented Recurrent Transformer (SRformer) model. SRformer aims to overcome the problem of high computational cost of traditional transformers when processing long sequences. The computational complexity of traditional transformers is quadratic with the length of the sequence, which limits its use in resource-constrained application scenarios. SRformer achieves efficient processing of sequence information by dividing the entire input sequence into multiple segments and using recurrent neurons to aggregate information between segments. This method not only retains the ability to process sequences, but also greatly reduces computational and memory requirements. Specifically, SRformer uses a local attention mechanism to process each segment, combined with recurrent attention, and uses a recurrent accumulation-and-fire (RAF) layer to process the flow of information between consecutive segments. The RAF layer updates the key-value product to compensate for the information loss caused by reducing the length of the attention window.

[0030] Segmented Attention is a key component in SRformer, which is designed to reduce the computational complexity of sequence-to-sequence models. At the same time, SRformer also proposes recurrent attention. Since directly calculating the remaining partial attention will result in the same computational complexity as the non-segmented model, the product of the key and value is first calculated through recurrent attention, and then multiplied with the query. The RAF layer in SRformer is used to build a bridge between segmented attention and recurrent attention to achieve effective propagation of cross-segment information. The main function of the RAF layer is to accumulate and filter partial products of key-value pairs from different segments, help approximate softmax operations, and exclude unnecessary information. It is designed to mimic the global information integration capabilities of the traditional attention mechanism in segmented sequence processing, while reducing computation and memory usage.

[0031] The embodiment of the present application proposes a transformer model based on segmented recurrent cross attention to replace the traditional transformer multi-head self-attention method, which greatly reduces the number of model parameters while ensuring the accuracy of the original model, and reduces the cost dependence on GPU during model training. At the same time, the RAF module is introduced to use the approximate attention mechanism instead of the global attention mechanism to achieve the purpose of reducing parameters.

[0032] See also Figure 2 As shown, Figure 2 A flowchart of a method for generating text based on artificial intelligence provided in an embodiment of the present application includes the following steps:

[0033] S10: receiving the original copy theme input by the user.

[0034] The artificial intelligence-based copywriting generation method provided in this application can be applied to copywriting generation in various application scenarios. Copywriting generation is usually implemented through a server, which can receive the copywriting topic input by the user in real time. For example, in the field of insurance marketing applications, users input marketing-related topics, and the model generates marketing copywriting within the scope of the currently input topic, which can improve the office efficiency of marketing personnel and save mechanical expression of copywriting.

[0035] For example, in the field of insurance marketing, for life insurance, the copywriting theme can be "long-term protection for the family"; for car insurance, the copywriting theme can be "a safe driving companion"; in the field of medical and health care, the copywriting theme can be "life and hope", "health and care", "how to prevent diabetes" and so on.

[0036] S20: Generate first feature vectors of m tokens in the original copy theme based on the original copy theme, where m is a positive integer.

[0037] Token refers to the smallest semantic unit in the text. The text is divided into tokens through word segmentation. Tokens can be words, punctuation marks or phrases. For example, "long-term security of the family" can be divided into three phrases: "family", "long-term" and "security".

[0038] After obtaining m tokens in the original copy theme, the feature vector of each token is obtained by encoding the m tokens. Specifically, in some embodiments of the present application, Figure 3 As shown, a specific feature vector generation solution is provided, including the following steps S21-S24:

[0039] S21: Segment the original copy topic to obtain m tokens.

[0040] Among them, there are many word segmentation methods, such as defining word segmentation rules based on regular expressions, word segmentation based on dictionaries, word segmentation based on machine learning or deep learning, etc., or word segmentation based on word segmentation tools, which are not limited in the embodiments of the present application.

[0041] S22: Input m tokens into the Transformer encoder and output the initial feature vector corresponding to each token.

[0042] In the Transformer encoder, the input token is first converted into the corresponding embedding vector, and the embedding vector is input into the encoder layer. After being processed by the encoder layer, each token is converted into a feature vector.

[0043] S23: Based on the initial feature vector corresponding to each token, determine the first feature vector of each token, where the first feature vector includes a first sub-vector and a second sub-vector.

[0044] Specifically, the token after word segmentation is input into the Transformer encoder for feature encoding. Each token corresponds to an initial feature vector of dimension d. The feature vector is copied three times as Q, K, and V, and K is taken as the first sub-vector and V as the second sub-vector. The first sub-vector can also be called the first key vector, and the second sub-vector can also be called the first value vector.

[0045] S30: Based on the first feature vector of the nth token among the m tokens and the second feature vector of the nth concatenated text, a segmented recurrent cross-attention mechanism is used to generate the nth generated text.

[0046] The nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, and n is a positive integer less than or equal to m. It is worth noting that when n is 1, because no text is generated, the first concatenated text is a special character token, indicating the beginning, and the special character token can be customized by the technician.

[0047] For example, the second concatenated text is determined based on the first generated text and the first concatenated text, and the first concatenated text is a special character token, then the second concatenated text is obtained by concatenating the first generated text and the special character token.

[0048] In some embodiments, since the concatenated text is also composed of tokens, the feature vector of the concatenated text can be determined by determining the feature vector of the token in the concatenated text. Specifically, the second feature vector of the nth concatenated text is determined based on the following method: determining the initial feature vector of the token in the nth concatenated text; and calculating the second feature vector of the nth concatenated text based on the initial feature vector of the token in the nth concatenated text and the self-attention mechanism.

[0049] Specifically, in the self-attention mechanism, the initial feature vector of each token is copied three times (or transformed linearly) to form Q, K, V; for each token in the sequence, the dot product between its Q vector and the K vectors of all other tokens is calculated to obtain an attention score matrix. The attention scores are normalized using the softmax function so that the sum of each row is 1. The normalized attention weights are multiplied by the corresponding V vectors, and the weighted V vectors of all tokens are summed to obtain the final output vector. The self-attention machine is usually implemented in a multi-head form, that is, the above process is replicated multiple times (usually 8 heads or more), and each head has its own Q, K, V matrices. The representation learned by each head is different. Finally, the outputs of all heads are concatenated, processed by a linear layer, and residually connected with the input initial feature vector, and then the second feature vector of the concatenated text is obtained by layer normalization.

[0050] In the embodiments of the present application, the second feature vector of the n-th concatenated text may also be referred to as a Q vector. That is, step S30 may be described as generating the n-th generated text based on the K vector and V vector of the n-th token among the m tokens and the Q vector of the n-th concatenated text. Specifically, in some embodiments of the present application, such as Figure 4 As shown, a method for generating a generated text is provided, comprising the following steps S31-S33:

[0051] S31: Input the first sub-vector, the second sub-vector and the second feature vector of the nth token into the segmented recurrent cross attention mechanism and output the score matrix.

[0052] S32: Input the score matrix into the feedforward network and output the probability of the associated words.

[0053] S33: Select the associated word with the highest probability as the generated text for the nth time.

[0054] In an embodiment of the present application, attention is calculated based on the input second feature vector (Q), the first sub-vector (K) and the second sub-vector (V) through a segmented recurrent cross-attention mechanism, and the traditional attention calculation is replaced by approximate attention (RAF).

[0055] Specifically, after multiplying the K and V vectors, a new vector Ht is obtained as the input of RAF. The output of RAF is calculated with Q and divided by the square root of the dimension of vector K as the normalization factor of softmax, and finally the score matrix QR[t] is obtained. QR[t] is then used as the input of the feedforward network. The output of the feedforward network is the probability of all words, and the word with the highest probability is taken as the token of the generated text for the nth time.

[0056] The calculation process of RAF is as follows: Figure 5 As shown, RAF includes the following key components: Linear layer: The input is first projected through a linear layer, which is needed to adjust the scale of the input. Leaky memory Leak: RAF does not apply weights to hidden memories, but uses a learnable leakage coefficient and threshold. The leakage coefficient controls the speed at which information "leaks", while the threshold acts as a bias to activate values ​​above this threshold. Soft-reset: Once the hidden memory triggers "excitation", it is soft-reset by subtracting the value of the threshold, which is similar to the reset mechanism of the Integrate-and-Fire neuron. ReLU activation: The output finally passes through the ReLU activation function to ensure that only positive values ​​pass through.

[0057] Specifically, the calculation process of RAF is as follows: the input Ht is projected to Qt through the linear layer, mem is the memory state, which is updated to leak*mem+Qt, and the output y is calculated as mem / threshold-1.0. If y is greater than 0, mem will subtract the value of threshold to achieve a soft reset. The final output of RAF is obtained through the ReLU activation function.

[0058] S40: splicing the m-th generated text and the m-1-th spliced ​​text, and outputting them as the target text.

[0059] In the embodiment of the present application, the nth generated text is obtained through step 30, and the nth generated text is spliced ​​with the n-th spliced ​​text to obtain the n+1th spliced ​​text. The n+1th generated text is generated based on the K vector and V vector of the n+1th token and the Q vector of the n+1th spliced ​​text. This cycle is repeated until the copy generation model generates the mth generated text based on the m tokens in the original copy theme, and the mth generated text and the m-1th spliced ​​text are spliced ​​and output as the target copy.

[0060] Based on the above embodiments, in order to more clearly illustrate the copywriting generation method provided in the embodiments of the present application, as shown in FIG. Figure 6As shown, the architecture of the copywriting generation model is shown, including a Transformer encoder, a segmented cyclic cross attention mechanism, a feedforward network, a self-attention mechanism, and a text generation module. Specifically, the user inputs the original copywriting theme into the copywriting generation model, and after the original copywriting theme is segmented, the first feature vector of each token is generated by the Transformer encoder, and the first subvector (K) and the second subvector (V) are generated based on the first feature vector. The generated text token output by the feedforward network each time is spliced ​​with the generated text output by the history to obtain the second feature vector (Q) of the spliced ​​text, and the first subvector (K), the second subvector (V), and the second feature vector (Q) are input into the segmented cyclic cross attention mechanism to obtain a score matrix QR[t]. The score matrix QR[t] is processed by the feedforward network, and the probability of all words is output. The text generation module selects the word with the highest probability as the text generated this time. After that, the text tokrn generated this time is spliced ​​with the historical spliced ​​text, and the feature vector of the spliced ​​text is used as the second feature vector (Q) of the next segmented cyclic cross attention mechanism. Repeat this cycle until the text generation is complete.

[0061] In the embodiment of the present application, a transformer model based on segmented recurrent cross attention is proposed to replace the traditional transformer multi-head self-attention method, which greatly reduces the number of model parameters while ensuring the accuracy of the original model, and reduces the cost dependence on GPU during model training. At the same time, the RAF module is introduced to use the approximate attention mechanism instead of the global attention mechanism to achieve the purpose of reducing parameters.

[0062] In some embodiments of the present application, a training method for a copywriting generation model is provided, such as Figure 6 As shown, the following steps are included:

[0063] Step S50: training the text generation model based on the sample set to obtain a trained initial text generation model.

[0064] Step S60: Copy the initial copy generation model to obtain a copy copy model, where the copy generation model includes a Transformer encoder, a self-attention mechanism, a segmented recurrent cross-attention mechanism, a feedforward network, and a deep Q (Deep Q-Networks, DQN) network.

[0065] Because it is a generative model, the quality of the generated copy will have a certain degree of randomness, and there will be cumulative errors. For example, if a word is generated incorrectly, the subsequent generation process will continue to generate based on the current error information, resulting in cumulative errors in the subsequent generated copy. In order to solve this problem, the embodiment of the present application adds a reinforcement learning module (DQN network) to guide the generated results and improve the quality of the copy generated by the model by only changing the objective function at the end of the model.

[0066] Step S70: training the copy generation model using the sample set, and updating the model parameters of the copy generation model.

[0067] In some embodiments of the present application, Figure 7 As shown, a specific method for updating model parameters is provided, including the following steps S71-S74:

[0068] S71: The word vectors of the words generated by the feedforward network at each moment are combined into a word vector matrix.

[0069] S72: Input the word vector matrix into the deep Q network and output the quality score.

[0070] S73: Obtain manual scoring results for the generated words.

[0071] S74: Update the model parameters of the replication model based on the manual scoring results and the quality score.

[0072] Specifically, the input of the DQN network is the vector before the model generates a word (not the word that has been generated). The model records the words generated at each moment, matrices the corresponding vectors (not words), and forms a word vector matrix N*d, where N is the number of generated words and d is the dimension of the current word vector. This N*d matrix is ​​used as the input of the DQN network, and the DQN network will output a quality score. Manual intervention is required here, that is, manual scoring of the output quality score, high quality is 1, and low quality is 0. Because the model has recorded all the generated words, humans can check the results of the recorded generated words. If the results meet expectations, the manual scoring is high quality during intervention, otherwise it is low quality.

[0073] Furthermore, after the DQN network obtains the manual scoring results, it calculates the loss value, such as the MSE loss, based on the manual scoring results and the quality score, and back-propagates the gradient value of the loss value to modify the model parameters of the copy model.

[0074] Step S80: copying the model parameters of the copy text generation model to the initial copy text generation model to obtain a target copy text generation model.

[0075] In some embodiments, after the reinforcement learning iteration, the model parameters of the copy generation model and the initial copy generation model are different. At this time, the model parameters of the copy generation model are copied to the initial copy generation model to update the initial copy generation model. After the update, the initial copy generation model is used to continue to input training data to generate new results, and then the new model is copied. This batch of data results are used for manual intervention to train the copy model. Repeated iterations are performed until the data quality meets the requirements.

[0076] In an embodiment of the present application, a text generation model is copied, a DQN network is added to the copied model, and the copied model is updated through the DQN network. When the model quality meets certain requirements, the parameters of the copied model are synchronized to the text generation model, thereby improving the quality of the text generated by the model.

[0077] It is worth noting that the above-mentioned embodiment introduces the introduction of the DQN network in the training process of the text generation model. In other embodiments of the present application, the DQN network can also be introduced in the actual application process of the text generation model. When the text is actually generated, the text generation model is updated in real time to generate more accurate text.

[0078] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0079] In one embodiment, a text generation device based on artificial intelligence is provided, and the text generation device based on artificial intelligence corresponds to the text generation method based on artificial intelligence in the above embodiment. Figure 8 As shown, the text generation device based on artificial intelligence includes a receiving module 101, a generating module 102, and an output module 103. The detailed description of each functional module is as follows:

[0080] The receiving module 101 is used to receive the original copy theme input by the user;

[0081] A generating module 102, configured to generate first feature vectors of m tokens in the original copy subject based on the original copy subject, where m is a positive integer;

[0082] The generating module 102 is further used to generate an nth generated text based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text, wherein the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m;

[0083] The output module 103 is used to splice the m-th generated text and the m-1-th spliced ​​text, and output them as the target text.

[0084] In some embodiments, the generation module 102 is specifically used to segment the original text topic to obtain m tokens; input the m tokens into the Transformer encoder and output the initial feature vector corresponding to each token; based on the initial feature vector corresponding to each token, determine the first feature vector of each token, and the first feature vector includes a first sub-vector and a second sub-vector.

[0085] In some embodiments, the second feature vector of the nth concatenated text is determined based on the following method: determining the initial feature vector of the token in the nth concatenated text; calculating the second feature vector of the nth concatenated text based on the initial feature vector of the token in the nth concatenated text and the self-attention mechanism.

[0086] In some embodiments, the generation module 102 is specifically used to input the first sub-vector, the second sub-vector and the second feature vector of the nth token into the segmented recurrent cross-attention mechanism, and output a score matrix; input the score matrix into the feedforward network, and output the probability of the associated words; select the associated word with the highest probability as the generated text for the nth time.

[0087] In some embodiments, the copy generation model is trained and generated based on the following method: the copy generation model is trained based on a sample set to obtain a trained initial copy generation model; the initial copy generation model is copied to obtain a copy copy model, the copy generation model includes a Transformer encoder, a self-attention mechanism, a segmented recurrent cross-attention mechanism, a feedforward network and a deep Q network; the copy generation model is trained through a sample set to update the model parameters of the copy generation model; the model parameters of the copy generation model are copied to the initial copy generation model to obtain a target copy generation model.

[0088] In some embodiments, the update module 104 is used to form a word vector matrix from the word vectors generated by the feedforward network at each moment; input the word vector matrix into the deep Q network and output a quality score; obtain manual scoring results for the generated words; and update the model parameters of the replication model based on the manual scoring results and the quality score.

[0089] In some embodiments, the update module 104 is specifically used to calculate the loss value based on the manual scoring result and the quality score; and to modify the model parameters of the copy model based on the gradient value of the loss value.

[0090] The present application provides a copywriting generation device based on artificial intelligence. The user inputs the original copywriting theme into the copywriting generation model. When the copywriting generation model generates the nth generated text, the generated text is generated based on the feature vector of the token in the copywriting theme and the feature vector of the nth spliced ​​text using a segmented cyclic cross-attention mechanism. The nth spliced ​​text is determined based on the n-1th generated text and the n-1th spliced ​​text, and the cycle is repeated until the copywriting is generated. The embodiment of the present application saves the cost of storage hardware and realizes the generation of copywriting by adopting a segmented cyclic cross-attention mechanism without losing the model effect and model accuracy.

[0091] For the specific definition of the artificial intelligence-based copy generation device, please refer to the definition of the artificial intelligence-based copy generation method above, which will not be repeated here. Each module in the above-mentioned artificial intelligence-based copy generation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0092] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a text generation method based on artificial intelligence.

[0093] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the client side of a method for generating text based on artificial intelligence

[0094] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0095] Receive the original copy topic input by the user;

[0096] Generate a first feature vector of m tokens in the original copy theme based on the original copy theme, where m is a positive integer;

[0097] Generate an nth generated text based on a first feature vector of an nth token among m tokens and a second feature vector of an nth concatenated text, wherein the nth concatenated text is determined based on an n-1th generated text and an n-1th concatenated text, where n is a positive integer less than or equal to m;

[0098] The generated text of the mth time and the concatenated text of the m-1th time are concatenated and output as the target text.

[0099] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0100] Receive the original copy topic input by the user;

[0101] Generate a first feature vector of m tokens in the original copy theme based on the original copy theme, where m is a positive integer;

[0102] Generate an nth generated text based on a first feature vector of an nth token among m tokens and a second feature vector of an nth concatenated text, wherein the nth concatenated text is determined based on an n-1th generated text and an n-1th concatenated text, where n is a positive integer less than or equal to m;

[0103] The generated text of the mth time and the concatenated text of the m-1th time are concatenated and output as the target text.

[0104] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0106] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0107] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

[0108] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A method for generating text based on artificial intelligence, characterized in that: Applied to a copywriting generation model, the method includes: Receive the original copy topic input by the user; Generate a first feature vector of m tokens in the original copy theme based on the original copy theme, where m is a positive integer; Based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text, a segmented recurrent cross attention mechanism is used to generate an nth generated text, wherein the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m; The generated text of the mth time and the concatenated text of the m-1th time are concatenated and output as the target text.

2. The method according to claim 1, characterized in that The step of generating first feature vectors of m tokens in the original copy theme based on the original copy theme includes: Segment the original copy topic to obtain m tokens; Input m tokens into the Transformer encoder and output the initial feature vector corresponding to each token; Based on the initial feature vector corresponding to each token, a first feature vector of each token is determined, where the first feature vector includes a first sub-vector and a second sub-vector.

3. The method according to claim 2, characterized in that The second feature vector of the n-th concatenated text is determined based on the following method: Determine the initial feature vector of the token in the nth concatenated text; Based on the initial feature vector of the token in the n-th concatenated text and the self-attention mechanism, a second feature vector of the n-th concatenated text is calculated.

4. The method according to claim 2, characterized in that: The method of generating the nth generated text based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text by using a segmented recurrent cross attention mechanism includes: Input the first subvector, the second subvector and the second feature vector of the nth token into a segmented recurrent cross attention mechanism, and output a score matrix; Input the score matrix into a feedforward network and output the probability of the associated words; Select the associated word with the highest probability as the generated text for the nth time.

5. The method according to claim 1, characterized in that The copywriting generation model is trained and generated based on the following method: The copywriting generation model is trained based on the sample set to obtain a trained initial copywriting generation model; The initial copy generation model is copied to obtain a copy generation model, wherein the copy generation model includes a Transformer encoder, a self-attention mechanism, a segmented recurrent cross attention mechanism, a feedforward network, and a deep Q network; Training the copywriting generation and replication model through the sample set, and updating the model parameters of the copywriting generation and replication model; The model parameters of the copy text generation model are copied to the initial text generation model to obtain a target text generation model.

6. The method according to claim 5, characterized in that The step of training the copy generation model using the sample set and updating the model parameters of the copy generation model includes: The word vectors of the words generated by the feedforward network at each moment are combined into a word vector matrix; Input the word vector matrix into the deep Q network and output the quality score; Obtain manual scoring results for generated words; The model parameters of the replication model are updated based on the manual scoring results and the quality score.

7. The method according to claim 7, characterized in that: The updating of the model parameters of the replication model based on the manual scoring result and the quality score includes: Calculate the loss value based on the manual scoring result and the quality score; The model parameters of the copy model are modified by backpropagation based on the gradient value of the loss value.

8. A copywriting generation device based on artificial intelligence, characterized in that: include: A receiving module, used for receiving the original copy theme input by the user; A generating module, used for generating first feature vectors of m tokens in the original copy theme based on the original copy theme, where m is a positive integer; The generation module is further used to generate an nth generated text based on the first feature vector of the nth token in the m tokens and the second feature vector of the nth concatenated text by using a segmented recurrent cross attention mechanism, wherein the nth concatenated text is determined based on the n-1th generated text and the n-1th concatenated text, where n is a positive integer less than or equal to m; The output module is used to splice the m-th generated text and the m-1-th spliced ​​text, and output them as the target text.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for generating text based on artificial intelligence as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the artificial intelligence-based copywriting generation method as described in any one of claims 1 to 7 are implemented.

Images