Text generation method, device, medium and electronic device
By performing vector conversion and multiple encoding and decoding processing on the basic text, the pre-trained text generation model is used to solve the problem of plain text generation and semantic incoherence in the prior art, and the generated text is more fluent and coherent.
Patent Information
- Application Number
- CN202011044316.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-28
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-09-28
AI Technical Summary
Existing text generation technologies such as seq2seq and vae have corpus and model design limitations when generating grown coherent text, resulting in the generated text being too bland and repetitive and inconsistent with semantic incompatibility.
By obtaining the basic text for vector transformation, the pre-trained text generation model performs at least two encoding and decoding processing, capturing deeper semantics, thereby generating smooth target text.
The semantic coherence and richness of the target text is improved, and the generated text is more coherent and diversified.
Smart Images

Figure CN112131835B_ABST
Abstract
Description
Background Art
[0002] At present, with the rapid development and application of intelligent AI technology in modern society, text generation technology is widely used in dialogue robots, intelligent customer service, smart medical care, automatic news writing, novel generation and other fields.
[0003] In the existing technology, there are mainly two deep learning solutions related to text generation technology: seq2seq and vae. However, these two technologies have their defects in generating long coherent texts. Due to the limitations of the corpus and the design of the model itself, the generated texts are generally too bland and repetitive, and there is a problem of insufficient semantic coherence.
[0004] Based on this, how to ensure the generation of fluent target text and enhance the semantic coherence in the target text is a technical problem that needs to be solved urgently. Summary of the invention
[0005] The purpose of this application is to provide a text generation method and device, a computer-readable storage medium, and an electronic device, so as to ensure the generation of a fluent target text. Enhancing the semantic coherence in the target text is a technical problem that needs to be solved urgently.
[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.
[0007] According to one aspect of an embodiment of the present application, a text generation method is provided, the method comprising: obtaining a basic text for generating a target text; performing vector conversion on the basic text to obtain a feature vector corresponding to the basic text; encoding the feature vector at least twice using a pre-trained text generation model to obtain encoded data corresponding to the basic text; and decoding the encoded data corresponding to the basic text at least twice using the text generation model to generate the target text.
[0008] According to one aspect of an embodiment of the present application, a text generation device is provided, comprising: an acquisition unit, used to acquire a basic text for generating a target text; a conversion unit, used to perform vector conversion on the basic text to obtain a feature vector corresponding to the basic text; an encoding unit, used to perform encoding processing on the feature vector at least twice through a pre-trained text generation model to obtain encoded data corresponding to the basic text; and a decoding unit, used to perform decoding processing on the encoded data corresponding to the basic text at least twice through the text generation model to generate the target text.
[0009] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. The computer program includes executable instructions. When the executable instructions are executed by a processor, the text generation method described in the above embodiment is implemented.
[0010] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; and a memory for storing executable instructions of the processors, wherein when the executable instructions are executed by the one or more processors, the one or more processors implement the text generation method as described in the above embodiments.
[0011] In the technical solutions of some embodiments of the present application, the feature vector is encoded at least twice by a pre-trained text generation model, and the encoded data corresponding to the basic text is decoded at least twice by the text generation model, which can ensure that deeper semantics are captured and learned, thereby generating a fluent target text. Enhancing the semantic coherence in the target text is a technical problem that needs to be solved urgently.
[0012] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The above and other features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
[0014] Figure 1 A flowchart of a text generation method according to an embodiment of the present application is shown;
[0015] Figure 2 A detailed flow chart of performing vector conversion on the basic text according to an embodiment of the present application is shown;
[0016] Figure 3 A schematic diagram of a framework of a text generation model according to an embodiment of the present application is shown;
[0017] Figure 4 A detailed flowchart of encoding the feature vector at least twice by using a pre-trained text generation model according to an embodiment of the present application is shown;
[0018] Figure 5 A detailed flow chart of encoding feature vectors corresponding to each word set in a pre-trained text generation model according to an embodiment of the present application is shown;
[0019] Figure 6A detailed flow chart of encoding the encoding data corresponding to each word set in a pre-trained text generation model according to one embodiment of the present application is shown;
[0020] Figure 7 A detailed flow chart showing at least two decoding processes of the encoded data corresponding to the base text by the text generation model to generate a target text according to an embodiment of the present application is shown;
[0021] Figure 8 A detailed flow chart of a training text generation model according to an embodiment of the present application is shown;
[0022] Fig. 9 A block diagram of a text generation device according to an embodiment of the present application is shown;
[0023] Fig.10 A computer-readable storage medium for implementing a text generation method according to an embodiment of the present application is shown;
[0024] Fig.11 An example block diagram of an electronic device for implementing a text generation method according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0025] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.
[0026] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.
[0027] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0028] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0029] It should also be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those shown or described.
[0030] First, this application briefly describes the application scenarios of the text generation method.
[0031] In this application, the text generation method can be applied in the fields of conversational robots, intelligent customer service, smart medical care (including but not limited to smart consultation, smart registration, etc.), automatic news writing, and novel generation.
[0032] In one embodiment of the present application, the text generation method may be executed in a server.
[0033] Specifically, the text generation method can be executed by a cloud server with cloud computing function. Specifically, cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, and expanded at any time. By establishing a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform, various types of virtual resources are deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.
[0034] In other embodiments of the present application, the server may also be replaced by an electronic device with a touch screen, such as a tablet computer or a laptop computer.
[0035] The following will be combined Figure 2 The implementation details of the technical solution of the text generation method of the embodiment of the present application are elaborated in detail:
[0036] See also Figure 2, shows a flowchart of text generation according to an embodiment of the present application. The text generation method can be executed by a physical device or a virtual device with a computing and processing function. Figure 2 As shown, the text generation method at least includes steps 110 to 170:
[0037] In step 110 , a basic text for generating a target text is obtained.
[0038] In the present application, the basic text includes at least one word set, and the word set includes at least one word. For example, it may refer to a long text, such as an article, wherein each word set corresponds to a sentence in the article, and each word corresponds to a word in a sentence.
[0039] In the present application, the basic text may also refer to keywords arranged in a continuous sequence, such as "I", "yesterday", "go", "Lotus Mountain", and "Spring Outing".
[0040] In step 130, vector conversion is performed on the basic text to obtain a feature vector corresponding to the basic text.
[0041] In the present application, the basic text is vectorized to obtain a feature vector corresponding to the basic text. Figure 2 Follow the steps shown.
[0042] See also Figure 2 , shows a detailed flow chart of vector conversion of the basic text according to an embodiment of the present application. Specifically, it includes steps 131 to 132:
[0043] Step 131, mapping each word in the basic text into a word vector according to a preset mapping relationship.
[0044] In this application, mapping each word in the basic text into a word vector can be completed through the word2vec algorithm. Word2vec is a natural language processing algorithm, which is characterized by vectorizing words so that the relationship between words can be quantitatively measured and the connection between words can be explored.
[0045] In the process of mapping each word in the basic text into a vector, for example, the basic text is an article D, D = {S1, S2...S m}, S is one of the paragraphs, s is one of the sentences, that is, D = {(s1, s2…s k1 ), (s k1+1 ,s k1+2 ,…s k2 ),…,(s k(m-1)+1 ,..skm )}, convert each word into a vector of fixed length is the vector of the first word in the ith sentence, and the vectors of each word in the ith sentence are
[0046] For another example, the basic text is a series of keywords T, where T = {t1, t2 ... t i}, t i Represents the vector corresponding to the i-th word, for example, t i =[0 1 0 1 1 1 0 0 1 0].
[0047] Continue to see Figure 2 In step 132, the word vectors corresponding to the words in each word set are respectively reduced in dimension to obtain the feature vectors corresponding to each word set, wherein the feature vectors corresponding to each word set constitute the feature vector of the basic text.
[0048] In the present application, the word vectors corresponding to the words in each word set are reduced in dimensionality. The word vectors can be reduced in dimensionality through the Embedding function, and the high-dimensional feature vectors are projected into relatively low-dimensional feature vectors to increase space utilization and reduce resource occupancy.
[0049] Continue to refer to Figure 1 In step 150, the feature vector is encoded at least twice using a pre-trained text generation model to obtain encoded data corresponding to the basic text.
[0050] In order to enable those skilled in the art to better understand the text generation model proposed in one embodiment of the present application, Figure 3 A brief introduction to the text generation model proposed in this application:
[0051] See also Figure 3 , showing a framework schematic diagram of a text generation model according to an embodiment of the present application.
[0052] In the text application model proposed in one embodiment of the present application, an input dimension reduction layer, a first encoding layer, a second encoding layer, a first decoding layer, a second decoding layer and a text output layer are included. Among them, the first encoding layer and the second encoding layer can be convolutional neural network layers (CNN), and the first decoding layer and the second decoding layer can be long short-term memory network layers (LSTM, Long Short-Term Memory).
[0053] In one embodiment of the present application, the feature vector is encoded at least twice by a pre-trained text generation model, which can be as follows: Figure 4 The steps shown are performed.
[0054] See also Figure 4 , shows a detailed flowchart of encoding the feature vector at least twice through a pre-trained text generation model according to an embodiment of the present application. Specifically comprising steps 151 to 152:
[0055] In step 151, the feature vectors corresponding to each word set are encoded in the pre-trained text generation model to obtain the encoding data corresponding to each word set.
[0056] It should be noted that, in the present application, the pre-trained text generation model includes a convolutional neural network layer.
[0057] In a specific implementation of this embodiment, the feature vectors corresponding to each word set are encoded in the pre-trained text generation model, which can be as follows: Figure 5 The steps shown are performed.
[0058] See also Figure 5 , shows a detailed flow chart of encoding feature vectors corresponding to each word set in a pre-trained text generation model according to an embodiment of the present application. Specifically, steps 1511 to 1512 are included:
[0059] Step 1511, convolution processing is performed on the feature vectors corresponding to each word set through a convolutional neural network layer to obtain first convolution feature data corresponding to each word set.
[0060] Step 1512, determining the first mathematical distribution of each first convolution feature data according to the first convolution feature data corresponding to each word set, so as to obtain the encoding data corresponding to each word set according to each first mathematical distribution.
[0061] In a specific implementation, encoding the feature vectors corresponding to each word set can be performed as follows: Figure 3 The first encoding layer in the text generation model shown in the figure is completed. Specifically, the sentence-level features in the basic text are extracted as the underlying features through the convolutional neural network layer, and the first mathematical distribution z1 of the underlying feature latent variable is described by the sampling mean and variance. z1~(μ1,δ1) indicates that z1 obeys this first mathematical distribution. Represents encoding and sampling, namely:
[0062]
[0063]
[0064] z1~(μ1,δ1)
[0065] After obtaining the first mathematical distribution of each first convolution feature data, sampling can be performed in each first mathematical distribution respectively to obtain the encoding data corresponding to each word set.
[0066] Continue to refer to Figure 4 In step 152, the encoding data corresponding to each word set is encoded in the pre-trained text generation model to obtain the encoding data corresponding to the basic text.
[0067] In a specific implementation of this embodiment, the encoding data corresponding to each word set is encoded in the pre-trained text generation model, which can be as follows: Figure 6 The steps shown are performed.
[0068] See also Figure 6 , shows a detailed flow chart of encoding the encoding data corresponding to each word set in a pre-trained text generation model according to an embodiment of the present application. Specifically including steps 1521 to 1522:
[0069] Step 1521, convolution processing is performed on the encoded data corresponding to each word set through a convolutional neural network layer to obtain second convolution feature data corresponding to the basic text.
[0070] Step 1522: Determine a second mathematical distribution of the second convolution feature data according to the second convolution feature data corresponding to the basic text, so as to obtain the encoding data corresponding to the basic text according to the second mathematical distribution.
[0071] In a specific implementation, encoding the encoding data corresponding to each word set can be performed as follows: Figure 3 Specifically, the convolutional neural network layer extracts high-level features as global features based on the sentence-level features encoded and extracted by the first encoding layer. The second mathematical distribution z2 of the underlying feature latent variable is also described by the sampling mean and variance. is the coded data of the first coding layer, z2~(μ2,δ2) means that z2 obeys the second mathematical distribution, Represents encoding and sampling, namely:
[0072]
[0073]
[0074] z2~(μ2,δ2)
[0075] After obtaining the second mathematical distribution of the second convolution feature data, sampling can be performed in the second mathematical distribution to obtain the encoded data corresponding to the basic text.
[0076] In the present application, encoding the feature vector at least twice can ensure that deeper semantics are captured and learned, thereby generating a fluent target text and enhancing the semantic coherence in the target text. Furthermore, the encoded data obtained by sampling the first mathematical distribution determined by the first convolution feature data and sampling the second mathematical distribution determined by the second convolution feature data has the characteristics of diversity, which can improve the generated target text content to be richer.
[0077] Continue to refer to Figure 1 In step 170, the encoded data corresponding to the basic text is decoded at least twice by the text generation model to generate a target text.
[0078] In one embodiment of the present application, the encoded data corresponding to the basic text is decoded at least twice by the text generation model to generate the target text, which can be performed as follows: Figure 7 The steps shown are performed.
[0079] See also Figure 7 , shows a detailed flow chart of performing at least two decoding processes on the encoded data corresponding to the base text by the text generation model according to an embodiment of the present application to generate the target text. Specifically, steps 171 to 173 are included:
[0080] Step 171, decoding the encoded data corresponding to the basic text in the text generation model to obtain first decoded data.
[0081] Step 172: decoding the first decoded data in the text generation model to obtain second decoded data.
[0082] Step 173, generating the current word with the maximum probability through the second decoded data to obtain the target text.
[0083] In the present application, decoding the encoded data corresponding to the basic text may be performed as follows: Figure 3 The first decoding layer in the text generation model shown in the figure is used to complete the decoding. That is, the first decoding layer decodes the current state from the hidden variable z and the previous decoding state, and captures the decoded sentence information. Represents the state of the sentence-level (s) layer at time step t (the tth sentence), LSTM sent Indicates the decoded sentence, i.e., information:
[0084]
[0085] In the present application, decoding the first decoded data may be performed as follows: Figure 3 The second decoding layer in the text generation model shown is completed, that is, the current word is generated at the second decoding layer, and the second decoding layer decrypts on the basis of the decoding of the first decoding layer and the previous decoding state and decoding result to generate the current word with the maximum probability. Represents the word level (word), the i-1th word of the tth sentence, LSTM word It is word-level decoding, and finally the word with the highest probability is normalized by softmax probability.
[0086]
[0087]
[0088] In this application, it is possible to repeat Figure 8 The steps shown in the figure train the text generation model to be trained until the loss function in the text generation model to be trained converges, and the result is as follows Figure 3 The text generation model shown.
[0089] For details, see Figure 8 , shows a detailed flow chart of a training text generation model according to an embodiment of the present application. Specifically, it includes steps 181 to 183:
[0090] Step 181, obtaining a training data set, wherein the training data set includes input data and supervision data.
[0091] Step 181, inputting the input data into the text generation model to be trained to output output data, wherein the text generation model to be trained includes at least two convolutional neural network layers and at least two recursive neural network layers.
[0092] Step 181, based on the comparison between the supervision data and the output data, correct the hidden layer parameters in the text generation model to be trained through gradient back transfer.
[0093] In the training phase of the text generation model in this application, for the distribution of the underlying feature latent variables z1~(μ1,φ1) and the distribution of the underlying feature latent variables z2~(μ2,φ2), the loss function loss can be The text generation model of the present application includes two hidden variables, which enhances the second KL divergence of the loss function loss, thereby enhancing the original text generation model's ability to learn the training data set. In summary, in the technical solutions of some embodiments of the present application, the feature vector is encoded at least twice by a pre-trained text generation model, and the encoded data corresponding to the basic text is decoded at least twice by the text generation model, which can ensure that deeper semantics are captured and learned, thereby generating a fluent target text. Enhancing the semantic coherence in the target text is a technical problem that needs to be solved urgently.
[0094] The following describes an apparatus embodiment of the present application, which can be used to execute the text generation method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the text generation method of the present application.
[0095] Fig. 9 A block diagram of a text generation device according to an embodiment of the present application is shown.
[0096] Reference Fig. 9 As shown, according to an embodiment of the present application, a text generation device 900 includes: an acquisition unit 901, a conversion unit 902, an encoding unit 903 and a decoding unit 904.
[0097] Among them, the acquisition unit 901 is used to acquire the basic text for generating the target text; the conversion unit 902 is used to perform vector conversion on the basic text to obtain the feature vector corresponding to the basic text; the encoding unit 903 is used to perform at least two encoding processes on the feature vector through a pre-trained text generation model to obtain the encoded data corresponding to the basic text; the decoding unit 904 is used to perform at least two decoding processes on the encoded data corresponding to the basic text through the text generation model to generate the target text.
[0098] In some embodiments of the present application, based on the aforementioned scheme, the basic text includes at least one word set, and the word set includes at least one word. The conversion unit 902 includes a mapping unit, which is used to map each word in the basic text into a word vector according to a preset mapping relationship; a dimensionality reduction unit, which is used to reduce the dimensionality of the word vectors corresponding to the words in each word set to obtain feature vectors corresponding to each word set, wherein the feature vectors corresponding to each word set constitute the feature vector of the basic text.
[0099] In some embodiments of the present application, based on the aforementioned scheme, the encoding unit 903 includes a first encoding unit, which is used to encode the feature vectors corresponding to each word set in a pre-trained text generation model, and obtain the encoding data corresponding to each word set; the second encoding unit is used to encode the encoding data corresponding to each word set in the pre-trained text generation model, and obtain the encoding data corresponding to the basic text.
[0100] In some embodiments of the present application, based on the aforementioned scheme, the pre-trained text generation model includes a convolutional neural network layer, and the first encoding unit is configured to: perform convolution processing on the feature vectors corresponding to each word set through the convolutional neural network layer, and obtain first convolution feature data corresponding to each word set; according to the first convolution feature data corresponding to each word set, determine the first mathematical distribution of each first convolution feature data, so as to obtain the encoding data corresponding to each word set according to each first mathematical distribution.
[0101] In some embodiments of the present application, based on the aforementioned scheme, the second encoding unit is configured to: perform convolution processing on the encoding data corresponding to the each word set through a convolutional neural network layer to obtain second convolution feature data corresponding to the basic text; determine the second mathematical distribution of the second convolution feature data according to the second convolution feature data corresponding to the basic text, so as to obtain the encoding data corresponding to the basic text according to the second mathematical distribution.
[0102] In some embodiments of the present application, based on the aforementioned scheme, the decoding unit 904 is configured to: decode the encoded data corresponding to the basic text in the text generation model to obtain first decoded data; decode the first decoded data in the text generation model to obtain second decoded data; generate the current word with the maximum probability through the second decoded data to obtain the target text.
[0103] In some embodiments of the present application, based on the above scheme, the device further includes: a training unit, which is used to repeat the following steps until the loss function in the text generation model to be trained converges to obtain the text generation model: obtain a training data set, the training data set includes input data and supervision data; input the input data into the text generation model to be trained to output output data, the text generation model to be trained includes at least two convolutional neural network layers and at least two recursive neural network layers. Based on the comparison between the supervision data and the output data, the hidden layer parameters in the text generation model to be trained are corrected by gradient back transfer.
[0104] It should be noted that although several units of the text generation method and the text generation device are mentioned in the above detailed description, this division is not mandatory. In fact, according to the implementation mode of the present application, two or more units and functions described above can be concretized in one unit. Conversely, the features and functions of a unit described above can be further divided into multiple units for concretization. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present application scheme. Those of ordinary skill in the art can understand and implement it without paying creative work.
[0105] As another aspect, the present application further provides a computer-readable storage medium on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present application may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present application described in the above "Exemplary Method" section of the present specification.
[0106] refer to Fig.10 As shown, a program product 1000 for implementing the above method according to an embodiment of the present application is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.
[0107] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0108] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0109] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0110] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0111] As another aspect, the present application also provides an electronic device capable of implementing the above method.
[0112] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.
[0113] Refer to the following Fig.11 1100 according to this embodiment of the present application is described. Fig.11 The electronic device 1100 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0114] like Fig.11As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include but are not limited to: at least one processing unit 1110, at least one storage unit 1120, and a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110).
[0115] The storage unit stores program codes, which can be executed by the processing unit 1110, so that the processing unit 1110 executes the steps described in the above “Example Method” section of this specification according to various exemplary embodiments of the present application.
[0116] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1121 and / or a cache memory unit 1122 , and may further include a read-only memory unit (ROM) 1123 .
[0117] The storage unit 1120 may also include a program / utility 1124 having a set (at least one) of program modules 1125, such program modules 1125 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0118] Bus 1130 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0119] The electronic device 1100 may also communicate with one or more external devices 1100 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1100, and / or communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1160. As shown, the network adapter 1160 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0120] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.
[0121] The text generation method, device, medium and electronic device in this application can be applied to smart medical systems.
[0122] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0123] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be performed without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A text generation method, characterized in that: The method comprises: Acquire a basic text for generating a target text; the basic text includes at least one word set, and the word set includes at least one word; Performing vector conversion on the basic text to obtain a feature vector corresponding to the basic text; specifically comprising: mapping each word in the basic text to a word vector according to a preset mapping relationship; performing dimensionality reduction on the word vectors corresponding to the words in each word set to obtain a feature vector corresponding to each word set, wherein the feature vectors corresponding to each word set constitute the feature vector of the basic text; The feature vector corresponding to the basic text is encoded at least twice by a pre-trained text generation model to obtain encoded data corresponding to the basic text; the text generation model includes an input dimension reduction layer, a first encoding layer, a second encoding layer, a first decoding layer, a second decoding layer and a text output layer, and the first encoding layer and the second encoding layer include convolutional neural network layers; The invention comprises: performing convolution processing on the feature vectors corresponding to each word set through a convolutional neural network layer to obtain first convolution feature data corresponding to each word set; determining first mathematical distributions of each first convolution feature data according to the first convolution feature data corresponding to each word set, so as to obtain coding data corresponding to each word set according to each first mathematical distribution; wherein, encoding the feature vectors corresponding to each word set is completed in the first coding layer, and each sentence-level feature in the basic text is extracted as the underlying feature through the convolutional neural network layer, and the sampling mean is used. With variance The first mathematical distribution describing the underlying characteristic latent variable , ~ ( )express Following the first mathematical distribution, after obtaining the first mathematical distribution of each first convolution feature data, sampling is performed in each first mathematical distribution to obtain the encoding data corresponding to each word set; The coded data corresponding to each word set is convoluted by a convolutional neural network layer to obtain second convolutional feature data corresponding to the basic text; a second mathematical distribution of the second convolutional feature data is determined according to the second convolutional feature data corresponding to the basic text, so as to obtain the coded data corresponding to the basic text according to the second mathematical distribution; the method comprises: encoding the coded data corresponding to each word set in the second coding layer, extracting high-level features as global features based on the sentence-level features extracted by encoding in the first coding layer through the convolutional neural network layer, and using the sampled mean. With variance The second mathematical distribution describing the underlying characteristic latent variable , ~ ( )express Following a second mathematical distribution, after obtaining the second mathematical distribution of the second convolution feature data, sampling is performed in the second mathematical distribution to obtain the encoded data corresponding to the basic text; The encoded data corresponding to the basic text is decoded at least twice by the text generation model to generate a target text.
2. The method according to claim 1, characterized in that The decoding process is performed at least twice on the encoded data corresponding to the basic text by the text generation model to generate the target text, including: Decoding the encoded data corresponding to the basic text in the text generation model to obtain first decoded data; Decoding the first decoded data in the text generation model to obtain second decoded data; The current word with the maximum probability is generated by the second decoded data to obtain the target text.
3. The method according to claim 1, characterized in that Repeat the following steps until the loss function in the text generation model to be trained converges to obtain the text generation model: Obtaining a training data set, wherein the training data set includes input data and supervision data; Inputting the input data into the to-be-trained text generation model to output output data, wherein the to-be-trained text generation model includes at least two convolutional neural network layers and at least two recursive neural network layers; Based on the comparison between the supervision data and the output data, the hidden layer parameters in the text generation model to be trained are corrected through gradient back transfer.
4. A text generation device, characterized in that: To implement the text generation method described in claim 1, the device comprises: An acquisition unit is used to acquire a basic text for generating a target text; the basic text includes at least one word set, and the word set includes at least one word; The conversion unit is used to perform vector conversion on the basic text to obtain a feature vector corresponding to the basic text; specifically, it is used to map each word in the basic text to a word vector according to a preset mapping relationship; and to perform dimensionality reduction on the word vectors corresponding to the words in each word set to obtain a feature vector corresponding to each word set, wherein the feature vectors corresponding to each word set constitute the feature vector of the basic text; The encoding unit is used to encode the feature vector corresponding to the basic text at least twice through the pre-trained text generation model to obtain the encoded data corresponding to the basic text; specifically, it is used to encode the feature vector corresponding to each word set in the pre-trained text generation model to obtain the encoded data corresponding to each word set; encode the encoded data corresponding to each word set in the pre-trained text generation model to obtain the encoded data corresponding to the basic text; The decoding unit is used to perform at least two decoding processes on the encoded data corresponding to the basic text through the text generation model to generate a target text.
5. A computer-readable program medium, characterized in that: The computer program instructions are stored therein, and when the computer program instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1 to 3.
6. A text generating electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 3 is implemented.
