Text expansion method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a text expansion method and device, electronic equipment and a storage medium, and belongs to the technical field of text processing. The method comprises the following steps: acquiring an original text sample and a reference text sample; word embedding is carried out on the original text sample based on the initial text expansion model to obtain word embedding features; performing position random embedding on the original text sample based on the initial text expansion model to obtain position embedding features; performing style attention coding on the word embedding features and the position embedding features based on an initial text expansion model to obtain style text coding features; decoding the style text coding features based on the initial text expansion model to obtain an original expansion text; training the initial text expanding-writing model based on the original expanding-writing text and the reference text sample to obtain a target text expanding-writing model; and performing text expansion on the target original text based on the target text expansion model to obtain a target expansion text. According to the embodiment of the invention, the readability of the text generated by expanding writing can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of text processing, and in particular to a text expansion method and device, an electronic device and a storage medium. Background Art

[0002] Text expansion is to expand the content based on given text. For example, when generating marketing content for insurance business, marketing text can be generated based on specific text. However, in the existing text expansion technology, there is a problem that the semantic logic of the text obtained by expansion is incoherent and the text style is not uniform, resulting in poor readability of the generated text. Therefore, how to improve the readability of the text generated by expansion has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a text expansion method and device, an electronic device and a storage medium, aiming to improve the readability of the text generated by the expansion.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a text expansion method, the method comprising:

[0005] Acquire an original text sample and a reference text sample of the original text sample; wherein the reference text sample is a text obtained by expanding the original text sample;

[0006] Based on a preset initial text expansion model, the original text sample is subjected to text word embedding to obtain word embedding features;

[0007] Based on the initial text expansion model, the original text sample is randomly embedded in position to obtain a position embedding feature;

[0008] Performing style attention encoding on the word embedding feature and the position embedding feature based on the initial text expansion model to obtain a style text encoding feature;

[0009] Based on the initial text expansion model, text decoding is performed on the style text encoding feature to obtain the original expanded text;

[0010] Training the initial text expansion model based on the original expanded text and the reference text sample to obtain a target text expansion model;

[0011] A target original text is obtained, and text expansion is performed on the target original text based on the target text expansion model to obtain a target expanded text.

[0012] In some embodiments, the initial text expansion model includes a position random embedder and a position encoder; the step of performing position random embedding on the original text sample based on the initial text expansion model to obtain a position embedding feature includes:

[0013] Randomly embedding the original text sample according to the position random embedder to obtain a position random embedding;

[0014] Performing position encoding on the original text sample according to the position encoder to obtain a position encoding embedding;

[0015] Feature aggregation is performed according to the position random embedding and the position coded embedding to obtain the position embedding feature.

[0016] In some embodiments, the randomly embedding the original text sample according to the position random embedder to obtain the position random embedding includes:

[0017] Performing word segmentation on the original text sample according to the position random embedder to obtain a word sequence; wherein the word sequence includes a target word and a target sequence number of the target word in the word sequence;

[0018] Generating disturbances according to the position random embedder to obtain a target disturbance value;

[0019] If the target sequence number is an odd number, a sinusoidal aggregation calculation is performed on the target disturbance value and the target sequence number according to a position random embedder to obtain the position random embedding;

[0020] If the target serial number is an even number, the position random embedding is performed by performing cosine aggregation calculation on the target disturbance value and the target serial number to obtain the position random embedding.

[0021] In some embodiments, the initial text expansion model includes random embedding weights and encoding embedding weights; and performing feature aggregation according to the position random embedding and the position encoding embedding to obtain the position embedding feature includes:

[0022] Performing aggregation calculation based on the random embedding weight and the position random embedding to obtain a position random feature;

[0023] Performing aggregation calculation based on the encoding embedding weight and the position encoding embedding to obtain a position encoding feature;

[0024] Feature aggregation is performed based on the position random feature and the position random feature to obtain the position embedded feature.

[0025] In some embodiments, the initial text expansion model includes a self-attention layer, a convolution layer, and a style embedding layer; the performing style attention encoding on the word embedding feature and the position embedding feature based on the initial text expansion model to obtain a style text encoding feature includes:

[0026] Summing the word embedding feature and the position embedding feature to obtain a text embedding feature;

[0027] Performing self-attention encoding on the text embedding feature according to the self-attention layer to obtain a self-attention encoding feature;

[0028] Performing convolution encoding on the text embedding feature according to the convolution layer to obtain a convolution encoding feature;

[0029] Performing style encoding on the word embedding feature according to the style embedding layer to obtain a style encoding feature;

[0030] Feature aggregation is performed according to the self-attention coding feature, the convolution coding feature and the style coding feature to obtain the style text coding feature.

[0031] In some embodiments, the training of the initial text expansion model based on the original expanded text and the reference text sample to obtain a target text expansion model includes:

[0032] Performing semantic loss calculation based on the original expanded text and the reference text sample to obtain semantic loss data;

[0033] Performing style loss calculation based on the original expanded text and the reference text sample to obtain style loss data;

[0034] Parameters of the initial text expansion model are optimized based on the semantic loss data and the style loss data to obtain the target text expansion model.

[0035] In some embodiments, the performing semantic loss calculation based on the original expanded text and the reference text sample to obtain semantic loss data includes:

[0036] Performing semantic extraction based on the original expanded text to obtain original semantic features;

[0037] Performing semantic extraction based on the reference text sample to obtain reference semantic features;

[0038] The semantic loss data is obtained by performing cosine similarity aggregation calculation based on the original semantic features and the reference semantic features.

[0039] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a text expansion device, the device comprising:

[0040] A data acquisition module is used to acquire an original text sample and a reference text sample of the original text sample; wherein the reference text sample is a text obtained by expanding the original text sample;

[0041] A word embedding module, used to perform text word embedding on the original text sample based on a preset initial text expansion model to obtain word embedding features;

[0042] A position embedding module, used for randomly embedding the original text sample based on the initial text expansion model to obtain a position embedding feature;

[0043] A style encoding module, used for performing style attention encoding on the word embedding feature and the position embedding feature based on the initial text expansion model to obtain a style text encoding feature;

[0044] A text decoding module, used for decoding the style text encoding features based on the initial text expansion model to obtain the original expanded text;

[0045] A model training module, used for training the initial text expansion model based on the original expanded text and the reference text sample to obtain a target text expansion model;

[0046] The model expansion module is used to obtain a target original text, and perform text expansion on the target original text based on the target text expansion model to obtain a target expanded text.

[0047] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0048] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0049] The present application proposes a text expansion method and device, an electronic device and a storage medium. The method obtains an original text sample and a reference text sample thereof, and then uses a preset initial text expansion model to perform word embedding on the original text sample to obtain word embedding features. At the same time, the original text sample is randomly embedded in position and noise is introduced into the position embedding features to enhance the robustness of the model when processing texts with different word orders and ensure the readability of the generated text. Furthermore, the word embedding features and the position embedding features are style-attention encoded to generate style text encoding features to ensure the semantic coherence and style consistency of the final expanded text. Furthermore, the initial text expansion model is further optimized by comparing with the reference text sample to obtain a target text expansion model. Finally, the target original text is obtained, and the target original text is expanded based on the target text expansion model to generate a target expanded text with high readability, semantic coherence and style consistency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flowchart of a text expansion method provided by an embodiment of the present application;

[0051] Figure 2 yes Figure 1 Flow chart of step S103 in FIG.

[0052] Figure 3 yes Figure 2 Flow chart of step S201 in FIG.

[0053] Figure 4 yes Figure 2 Flow chart of step S203 in FIG.

[0054] Figure 5 yes Figure 1 Flow chart of step S104 in FIG.

[0055] Figure 6 yes Figure 1 Flow chart of step S106 in FIG.

[0056] Figure 7 yes Figure 6 Flow chart of step S601 in FIG.

[0057] Figure 8 is a structural schematic diagram of a text expansion device provided in an embodiment of the present application;

[0058] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0062] First, some nouns involved in this application are analyzed:

[0063] Word embedding: It is a technique in natural language processing that converts words into dense real number vectors to represent them, so that these vectors can capture the semantic and grammatical relationships between words. Word embedding is an important branch of artificial intelligence and an interdisciplinary subject between machine learning and computational linguistics. Through word embedding, computers can understand and process human languages, such as Chinese and English. Word embedding technology is commonly used in various natural language processing applications, including text classification, sentiment analysis, machine translation, and question-answering systems.

[0064] Positional embedding: It is a technique in natural language processing that encodes the position of a given element in a sequence by assigning a unique vector to each element in the sequence. This method is to supplement word embedding so that the model can not only understand the meaning of words, but also capture the order and context of words in the text. Positional embedding is part of the field of artificial intelligence and is an interdisciplinary subject of machine learning and computational linguistics.

[0065] Text style: refers to a unique way of expression displayed through language usage patterns, reflecting the writing intention, emotional color and target audience of the text. Text style analysis plays a key role in understanding and generating language data, and is often used in content creation, text editing, advertising generation, etc. In addition, text style analysis technology is often used in technical fields such as machine translation, speech synthesis, text classification, author identification, public opinion analysis, etc. These applications involve data mining, machine learning, knowledge acquisition, knowledge engineering, and linguistic research related to language computing. The purpose is to process, understand and use human language through computers, and enhance the humanization and personalized expression capabilities of language technology systems.

[0066] Text semantics: It is a technique in natural language processing that studies the meaning of words, phrases, sentences, and entire documents in texts. It involves interpreting and reasoning about concepts and relationships expressed in texts, including word meaning parsing, sentence meaning construction, and document meaning understanding. The main tasks of text semantic analysis include semantic analysis, paragraph understanding, and intent recognition, with the goal of enabling computers to understand and generate language expressions with clear meanings.

[0067] Text expansion is to expand the content based on given text. For example, when generating marketing content for insurance business, marketing text can be generated based on specific text. However, in the existing text expansion technology, there is a problem that the semantic logic of the text obtained by expansion is incoherent and the text style is not uniform, resulting in poor readability of the generated text. Therefore, how to improve the readability of the text generated by expansion has become an urgent problem to be solved.

[0068] Based on this, the embodiments of the present application provide a text expansion method and device, an electronic device and a storage medium, aiming to improve the readability of the text generated by the expansion.

[0069] A text expansion method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the text expansion method in the embodiments of the present application is described.

[0070] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0071] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0072] The text expansion method provided in the embodiment of the present application relates to the field of text processing technology. The text expansion method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the text expansion method, etc., but is not limited to the above forms.

[0073] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0074] Figure 1 is an optional flowchart of the text expansion method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S107.

[0075] Step S101, obtaining an original text sample and a reference text sample of the original text sample; wherein the reference text sample is a text obtained by expanding the original text sample;

[0076] Step S102, performing text word embedding on the original text sample based on a preset initial text expansion model to obtain word embedding features;

[0077] Step S103, performing position random embedding on the original text sample based on the initial text expansion model to obtain position embedding features;

[0078] Step S104, performing style attention encoding on the word embedding features and the position embedding features based on the initial text expansion model to obtain style text encoding features;

[0079] Step S105, decoding the style text encoding features based on the initial text expansion model to obtain the original expanded text;

[0080] Step S106, training the initial text expansion model based on the original expanded text and the reference text sample to obtain a target text expansion model;

[0081] Step S107, obtaining the target original text, and performing text expansion on the target original text based on the target text expansion model to obtain the target expanded text.

[0082] In steps S101 to S107 shown in the embodiment of the present application, by obtaining an original text sample and a reference text sample thereof, the original text sample is word-embedded using a preset initial text expansion model to obtain word embedding features. At the same time, the original text sample is positionally randomly embedded and noise is introduced into the position embedding features to enhance the robustness of the model when processing texts with different word orders and ensure the readability of the generated text; further, the word embedding features and the position embedding features are style-attention encoded to generate style text encoding features to ensure the semantic coherence and style consistency of the final expanded text; further, the initial text expansion model is trained by comparison with the reference text sample to further optimize and obtain a target text expansion model, and finally the target original text is obtained, and the target original text is expanded based on the target text expansion model to generate a target expanded text with high readability, semantic coherence and style consistency.

[0083] In step S101 of some embodiments, the original text sample can be a short text, a sentence or a paragraph. For example, in the generation of marketing content text, the original text sample is a short description of a specific product. The reference text sample refers to the expanded or modified text associated with the original text sample. For example: the Genshin Impact text sample is "Fresh organic apples", and the reference text sample is "Fresh organic apples directly from the farm, with a sweet taste, crisp taste, rich nutrition, and an indispensable part of a healthy diet."

[0084] In step S102 of some embodiments, the initial text expansion model includes a pre-trained word embedding layer for converting the input text into a vector. In one embodiment, the initial text expansion model is a Transformer neural network, and the Transformer includes an Embed subnetwork, i.e., a word embedding layer, and the original text sample is input into the word embedding layer to convert the text into a vector representation, i.e., a word embedding feature.

[0085] See also Figure 2 In some embodiments, the initial text expansion model includes a position random embedder and a position encoder, and step S103 may include but is not limited to steps S201 to S203:

[0086] Step S201, randomly embedding the original text sample according to the position random embedder to obtain the position random embedding;

[0087] Step S202, position encoding the original text sample according to the position encoder to obtain a position encoding embedding;

[0088] Step S203, performing feature aggregation according to position random embedding and position coded embedding to obtain position embedding features.

[0089] In steps S201 to S203 shown in the embodiment of the present application, the original text sample is randomly embedded according to the position random embedder, noise is introduced into the position embedding feature to obtain the position random embedding, and then the original text sample is positionally encoded according to the position encoder to obtain the position encoded embedding, so as to learn the fixed position features of the words in the text, and finally feature aggregation is performed according to the position random embedding and the position encoded embedding, so as to combine the fixed position features and the position features after the introduction of noise, thereby increasing the position diversity and providing accurate position information, and finally obtaining the position embedding feature, so as to improve the subsequent target text expansion model to generate highly readable expanded text when facing texts with different word orders.

[0090] See also Figure 3 In some embodiments, step S201 may include but is not limited to steps S301 to S304:

[0091] Step S301, performing word segmentation on the original text sample according to the position random embedder to obtain a word sequence; wherein the word sequence includes a target word and a target sequence number of the target word in the word sequence;

[0092] Step S302, generating disturbances according to the position random embedder to obtain a target disturbance value;

[0093] Step S303, if the target sequence number is an odd number, a position random embedder is used to perform a sinusoidal aggregation calculation on the target disturbance value and the target sequence number to obtain a position random embedding;

[0094] Step S304: If the target sequence number is an even number, a cosine aggregation calculation is performed on the target disturbance value and the target sequence number according to the position random embedder to obtain a position random embedding.

[0095] In steps S301 to S304 shown in the embodiment of the present application, the original text sample is segmented into words according to the position random embedder to obtain a word sequence, and then the perturbation is generated according to the position random embedder to obtain a target perturbation value. When the target sequence number is an odd number, the target perturbation value and the target sequence number are sinusoidally aggregated to obtain a position random embedding according to the position random embedder. When the target sequence number is an even number, the target perturbation value and the target sequence number are cosine aggregated to obtain a position random embedding, thereby introducing noise into the word position in the text, so that the subsequent target text expansion model can still generate an expanded text with high readability when facing texts with different word orders.

[0096] In step S301 of some embodiments, word segmentation is to divide the original text sample into a sequence of words. For example, the original text sample is "I eat fresh apples", and the word segmentation obtains a word sequence ("I", "fresh", "apple"), including the target word "I", with a target sequence number of 1, the target word "fresh", with a target sequence number of 2, and the target word "evaluation", with a target sequence number of 3.

[0097] In step S302 of some embodiments, disturbance generation refers to obtaining a target disturbance value from a preset normal distribution, and the preset normal distribution is shown in formula (1):

[0098]

[0099] Among them, ∈ is the target perturbation value, pos is the target sequence number, and i is the preset dimension index.

[0100] In step S303 of some embodiments, when the target sequence number is an odd number, a position random embedder is used to perform a sinusoidal aggregation calculation on the target disturbance value and the target sequence number to obtain a position random embedding. The sinusoidal aggregation calculation is shown in formula (2):

[0101]

[0102] Among them, PE (pos,2i) is the position random embedding, and d is the preset embedding dimension.

[0103] In step S304 of some embodiments, when the target sequence number is an even number, a cosine aggregation calculation is performed on the target disturbance value and the target sequence number according to the position random embedder to obtain a position random embedding. The cosine aggregation calculation is shown in formula (3):

[0104]

[0105] In step S202 of some embodiments, the position encoding is embedded as a trainable position vector, which represents the position of the word in the original text sample, that is, for each position pos, a vector L is introduced pos To represent the position, in the iterative process of the initial text expansion model, the position encoding embedding, i.e., L (pos) Iterative optimization is performed so that the positional encoding embedding can more accurately represent the word position in the original text sample.

[0106] See also Figure 4 In some embodiments, the initial text expansion model includes random embedding weights and encoded embedding weights, and step S203 may include but is not limited to steps S401 to S403:

[0107] Step S401, performing aggregation calculation based on random embedding weights and position random embedding to obtain position random features;

[0108] Step S402, performing aggregation calculation based on the code embedding weight and the position code embedding to obtain the position code feature;

[0109] Step S403, performing feature aggregation based on the position random feature and the position random feature to obtain the position embedded feature.

[0110] In steps S401 to S403 shown in the embodiment of the present application, aggregation calculation is performed based on random embedding weights and position random embedding to obtain position random features, and then aggregation calculation is performed based on coding embedding weights and position coding embedding to obtain position coding features. Finally, feature aggregation is performed based on position random features and position random features to obtain position embedding features, thereby combining fixed position features and position features after the introduction of noise, providing accurate position information while increasing position diversity, and finally obtaining position embedding features, thereby improving the subsequent target text expansion model to generate highly readable expanded texts when facing texts with different word orders.

[0111] In step S401 of some embodiments, an aggregation calculation is performed based on the random embedding weight and the position random embedding to obtain the position random feature, and the aggregation calculation is shown in formula (4):

[0112] α·PE(pos,i) (4),

[0113] Among them, α·PE(pos,i) is the position random embedding, and α is the random embedding weight.

[0114] In step S402 of some embodiments, an aggregation calculation is performed based on the code embedding weight and the position code embedding to obtain the position code feature, and the aggregation calculation is shown in formula (5):

[0115] β·L pos (5)

[0116] Among them, β·L pos is the position embedding feature, and β is the encoding embedding weight.

[0117] In step S403 of some embodiments, feature aggregation is performed based on the position random feature and the position random feature to obtain the position embedded feature, and the feature aggregation is shown in formula (6):

[0118] MPE(pos,i)=α·PE(pos,i)+β·L pos,i (6)

[0119] Among them, MPE (pos, i) is the position embedding feature. It should be noted that during the training process of the initial text expansion model, α and β will be optimized.

[0120] See also Figure 5 In some embodiments, the initial text expansion model includes a self-attention layer, a convolution layer, and a style embedding layer, and step S104 includes but is not limited to steps S501 to S505:

[0121] Step S501, summing up the word embedding features and the position embedding features to obtain text embedding features;

[0122] Step S502, performing self-attention encoding on the text embedding feature according to the self-attention layer to obtain a self-attention encoding feature;

[0123] Step S503, performing convolution encoding on the text embedding feature according to the convolution layer to obtain a convolution encoding feature;

[0124] Step S504, performing style encoding on the word embedding features according to the style embedding layer to obtain style encoding features;

[0125] Step S505, performing feature aggregation based on the self-attention coding feature, the convolutional coding feature and the style coding feature to obtain the style text coding feature.

[0126] In steps S501 to S505 shown in the embodiment of the present application, text embedding features are obtained by summing up the word embedding features and the position embedding features, and then the text embedding features are self-attention encoded according to the self-attention layer to obtain self-attention encoded features, and the text embedding features are convolutionally encoded according to the convolution layer to obtain convolutional encoded features. At the same time, the word embedding features are style encoded according to the style embedding layer to obtain style encoded features, and finally feature aggregation is performed based on the self-attention encoded features, the convolutional encoded features and the style encoded features to obtain the style text encoded features, thereby ensuring the semantic coherence and style consistency of the final expanded text.

[0127] In step S501 of some embodiments, the word embedding feature and the position embedding feature are summed to obtain the text embedding feature, and the summation is shown in formula (7):

[0128] X=MPE(pos,i)+Embed(Y) (7),

[0129] Among them, Embed(Y) is the word embedding feature, Y is the original text sample, and X is the text embedding feature;

[0130] In step S502 of some embodiments, the self-attention encoding layer is a self-attention layer of the Transfromer layer, and the self-attention encoding feature is obtained by inputting the text embedding feature into the self-attention layer, as shown in formula (8):

[0131]

[0132] Among them, S(X) is the self-attention encoding feature, W Q ,W K ,W V are the network parameters of the self-attention layer, Q is the query generated by the self-attention layer, K is the key generated by the self-attention layer, V is the value generated by the self-attention layer, and softmax is the activation function.

[0133] In step S503 of some embodiments, the text embedding feature is convolutionally encoded according to the convolution layer to obtain a convolutional encoding feature, and the convolutional encoding is shown in formula (9):

[0134] C(X) = ReLU(X*W+b) (9),

[0135] Among them, C(X) is the convolutional coding feature, ReLU is the activation function, and W and b are the network parameters of the convolutional layer.

[0136] In step S504 of some embodiments, the word embedding feature is style-encoded according to the style embedding layer to obtain a style coding feature. The style coding is implemented by a preset fully connected layer. The style coding is shown in formula (10):

[0137]

[0138] Among them, M is the style encoding feature, are the network parameters of the style embedding layer.

[0139] In step S505 of some embodiments, feature aggregation is performed based on the self-attention encoding feature, the convolutional encoding feature and the style encoding feature to obtain the style text encoding feature, and the feature aggregation is shown in formula (11):

[0140]

[0141] Among them, B(X) is the style text feature, λ is the preset balance weight, (K+M) T is the transpose of (K+M).

[0142] In step S105 of some embodiments, the initial text expansion model includes a decoder. In one embodiment, the initial text expansion model is a Transformer model. The Transformer includes a decoder. The style text encoding features are input into the decoder to obtain the original expanded text.

[0143] See also Figure 6 In some embodiments, step S106 includes but is not limited to steps S601 to S603:

[0144] Step S601, performing semantic loss calculation based on the original expanded text and the reference text sample to obtain semantic loss data;

[0145] Step S602, calculating the style loss based on the original expanded text and the reference text sample to obtain style loss data;

[0146] Step S603 , optimizing the parameters of the initial text expansion model based on the semantic loss data and the style loss data to obtain a target text expansion model.

[0147] In steps S601 to S603 shown in the embodiment of the present application, semantic loss calculation is performed based on the original expanded text and the reference text sample to obtain semantic loss data, and then style loss calculation is performed based on the original expanded text and the reference text sample to obtain style loss data. Finally, the parameters of the initial text expansion model are optimized based on the semantic loss data and the style loss data to obtain a target text expansion model, so that the target text expansion model has a good perception of text style and text semantics, and can generate expanded text with high readability.

[0148] See also Figure 7In some embodiments, step S601 may include but is not limited to steps S701 to S703:

[0149] Step S701, performing semantic extraction based on the original expanded text to obtain original semantic features;

[0150] Step S702, performing semantic extraction based on the reference text sample to obtain reference semantic features;

[0151] Step S703, performing cosine similarity aggregation calculation based on the original semantic features and the reference semantic features to obtain semantic loss data.

[0152] Steps S701 to S703 shown in the embodiment of the present application perform semantic extraction based on the original expanded text to obtain original semantic features, then perform semantic extraction based on the reference text sample to obtain reference semantic features, and finally perform cosine similarity aggregation calculation based on the original semantic features and the reference semantic features to obtain semantic loss data, thereby accurately calculating the semantic loss gap between the original expanded text and the reference text sample, so that the target text expansion model can generate an expanded text with high semantic readability.

[0153] In step S701 of some embodiments, semantic extraction is performed through a preset BERT model, and the original expanded text is input into the BERT model to obtain original semantic features.

[0154] In step S702 of some embodiments, semantic extraction is performed through a preset BERT model, and a reference text sample is input into the BERT model to obtain a reference semantic feature.

[0155] In step S703 of some embodiments, cosine similarity aggregation calculation is performed based on the original semantic features and the reference semantic features to obtain semantic loss data. The cosine similarity aggregation calculation is shown in formula (12):

[0156] L semantic =1-CosineSim(E(T pred ),E(T target )) (12),

[0157] Among them, L semantic is the semantic loss data, CosineSim is the cosine similarity calculation function, E(T pred ) original semantic features, E(T target ) is the reference semantic feature.

[0158] In step S602 of some embodiments, the style loss is calculated by inputting the original expanded text into a pre-trained style extraction model to obtain the style features of the original expanded text, then inputting the reference text sample into the pre-trained style extraction model to obtain the style features of the reference text sample, and then calculating the cross entropy loss data between the style features of the original expanded text and the style features of the reference text sample. In one embodiment, the pre-trained style extraction model is a RoBERTa model.

[0159] In step S603 of some embodiments, a training loss value is calculated based on the semantic loss data and the style loss data. The training loss value calculation is shown in formula (13):

[0160] L total =γ·L semantic +(1-γ)L style (13)

[0161] Among them, L total is the training loss value, L style is the style loss data, and γ is the preset training balance weight.

[0162] The parameters of the initial text expansion model are optimized based on the training loss value to obtain the target text expansion model.

[0163] In step S107 of some embodiments, the target original text is input into the target text expansion model for text expansion, thereby generating a target expanded text with high readability, semantic coherence and style uniformity.

[0164] See also Figure 8 The present application also provides a text expansion device, which can implement the above-mentioned text expansion method, and the device includes:

[0165] The data acquisition module 801 is used to acquire an original text sample and a reference text sample of the original text sample; wherein the reference text sample is a text obtained by expanding the original text sample;

[0166] The word embedding module 802 is used to perform text word embedding on the original text sample based on a preset initial text expansion model to obtain word embedding features;

[0167] A position embedding module 803 is used to perform position random embedding on the original text sample based on the initial text expansion model to obtain a position embedding feature;

[0168] A style encoding module 804 is used to perform style attention encoding on word embedding features and position embedding features based on the initial text expansion model to obtain style text encoding features;

[0169] A text decoding module 805 is used to perform text decoding on the style text encoding features based on the initial text expansion model to obtain the original expanded text;

[0170] A model training module 806 is used to train the initial text expansion model based on the original expanded text and the reference text sample to obtain a target text expansion model;

[0171] The model expansion module 807 is used to obtain the target original text, and perform text expansion on the target original text based on the target text expansion model to obtain the target expanded text.

[0172] The specific implementation of the text expansion device is basically the same as the specific implementation of the above-mentioned text expansion method, and will not be repeated here.

[0173] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above text expansion method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0174] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0175] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0176] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the text expansion method of the embodiment of this application;

[0177] Input / output interface 903, used to implement information input and output;

[0178] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0179] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0180] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0181] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned text expansion method is implemented.

[0182] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0183] The text expansion method, text expansion device, electronic device and storage medium provided in the embodiments of the present application obtain an original text sample and a reference text sample thereof, and then use a preset initial text expansion model to perform word embedding on the original text sample to obtain word embedding features. At the same time, the original text sample is randomly embedded in position and noise is introduced into the position embedding features to enhance the robustness of the model when processing texts with different word orders and ensure the readability of the generated text; further, the word embedding features and the position embedding features are style-attention encoded to generate style text encoding features to ensure the semantic coherence and style consistency of the final expanded text; further, the initial text expansion model is trained by comparison with the reference text sample to further optimize and obtain a target text expansion model, and finally the target original text is obtained, and the target original text is expanded based on the target text expansion model to generate a target expanded text with high readability, semantic coherence and style consistency.

[0184] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0185] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0186] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0188] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0189] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0190] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0191] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0193] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0194] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A text expansion method, characterized in that: The method comprises: Acquire an original text sample and a reference text sample of the original text sample; wherein the reference text sample is a text obtained by expanding the original text sample; Based on a preset initial text expansion model, the original text sample is subjected to text word embedding to obtain word embedding features; Based on the initial text expansion model, the original text sample is randomly embedded in position to obtain a position embedding feature; Performing style attention encoding on the word embedding feature and the position embedding feature based on the initial text expansion model to obtain a style text encoding feature; Based on the initial text expansion model, text decoding is performed on the style text encoding feature to obtain the original expanded text; Training the initial text expansion model based on the original expanded text and the reference text sample to obtain a target text expansion model; A target original text is obtained, and text expansion is performed on the target original text based on the target text expansion model to obtain a target expanded text.

2. The method according to claim 1, characterized in that The initial text expansion model includes a position random embedder and a position encoder; the position random embedding of the original text sample based on the initial text expansion model to obtain a position embedding feature includes: Randomly embedding the original text sample according to the position random embedder to obtain a position random embedding; Performing position encoding on the original text sample according to the position encoder to obtain a position encoding embedding; Feature aggregation is performed according to the position random embedding and the position coded embedding to obtain the position embedding feature.

3. The method according to claim 2, characterized in that The randomly embedding the original text sample according to the position random embedder to obtain the position random embedding includes: Performing word segmentation on the original text sample according to the position random embedder to obtain a word sequence; wherein the word sequence includes a target word and a target sequence number of the target word in the word sequence; Generating disturbances according to the position random embedder to obtain a target disturbance value; If the target sequence number is an odd number, a sinusoidal aggregation calculation is performed on the target disturbance value and the target sequence number according to a position random embedder to obtain the position random embedding; If the target serial number is an even number, the position random embedding is performed by performing cosine aggregation calculation on the target disturbance value and the target serial number to obtain the position random embedding.

4. The method according to claim 2, characterized in that: The initial text expansion model includes random embedding weights and encoding embedding weights; the feature aggregation is performed according to the position random embedding and the position encoding embedding to obtain the position embedding feature, including: Performing aggregation calculation based on the random embedding weight and the position random embedding to obtain a position random feature; Performing aggregation calculation based on the encoding embedding weight and the position encoding embedding to obtain a position encoding feature; Feature aggregation is performed based on the position random feature and the position random feature to obtain the position embedded feature.

5. The method according to any one of claims 1 to 4, characterized in that: The initial text expansion model includes a self-attention layer, a convolution layer and a style embedding layer; the style attention encoding of the word embedding feature and the position embedding feature based on the initial text expansion model to obtain the style text encoding feature includes: Summing the word embedding feature and the position embedding feature to obtain a text embedding feature; Performing self-attention encoding on the text embedding feature according to the self-attention layer to obtain a self-attention encoding feature; Performing convolution encoding on the text embedding feature according to the convolution layer to obtain a convolution encoding feature; Performing style encoding on the word embedding feature according to the style embedding layer to obtain a style encoding feature; Feature aggregation is performed according to the self-attention coding feature, the convolution coding feature and the style coding feature to obtain the style text coding feature.

6. The method according to any one of claims 1 to 4, characterized in that: The training of the initial text expansion model based on the original expanded text and the reference text sample to obtain a target text expansion model includes: Performing semantic loss calculation based on the original expanded text and the reference text sample to obtain semantic loss data; Performing style loss calculation based on the original expanded text and the reference text sample to obtain style loss data; Parameters of the initial text expansion model are optimized based on the semantic loss data and the style loss data to obtain the target text expansion model.

7. The method according to claim 6, characterized in that The performing semantic loss calculation based on the original expanded text and the reference text sample to obtain semantic loss data includes: Performing semantic extraction based on the original expanded text to obtain original semantic features; Performing semantic extraction based on the reference text sample to obtain reference semantic features; The semantic loss data is obtained by performing cosine similarity aggregation calculation based on the original semantic features and the reference semantic features.

8. A text expansion device, characterized in that: The device comprises: A data acquisition module is used to acquire an original text sample and a reference text sample of the original text sample; wherein the reference text sample is a text obtained by expanding the original text sample; A word embedding module, used to perform text word embedding on the original text sample based on a preset initial text expansion model to obtain word embedding features; A position embedding module, used for randomly embedding the original text sample based on the initial text expansion model to obtain a position embedding feature; A style encoding module, used for performing style attention encoding on the word embedding feature and the position embedding feature based on the initial text expansion model to obtain a style text encoding feature; A text decoding module, used for decoding the style text encoding features based on the initial text expansion model to obtain the original expanded text; A model training module, used for training the initial text expansion model based on the original expanded text and the reference text sample to obtain a target text expansion model; The model expansion module is used to obtain a target original text, and perform text expansion on the target original text based on the target text expansion model to obtain a target expanded text.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the text expansion method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text expansion method according to any one of claims 1 to 7 is implemented.

Images