Random sampling dialogue text generation training method and device and computer medium

Abstract

The present application relates to artificial intelligence technology, disclose a kind of random sampling dialogue text generation training method, device, equipment and medium for dialogue generation model training in medical diagnosis field, the method comprises: dialogue text is randomly cut, the text before cutting point is composed of first training dialogue text, the predicted text of cutting point is generated using dialogue generation model, the first error between the predicted text of cutting point and real text is calculated, the predicted text of cutting point is replaced by the real text of cutting point, and all texts except the last sentence text in the replaced text are used as second training dialogue text, the predicted text of the last sentence text of dialogue text is generated using the model, the second error between the predicted text of the last sentence text and real text is calculated, and the pre-constructed dialogue generation model is trained according to the first error and the second error.The present application can improve the accuracy of dialogue generation model in medical diagnosis field.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a dialogue text generation training method and device based on random sampling of knowledge content and user behavior coordination, an equipment and a computer readable storage medium. BACKGROUND

[0002] In the medical field, operations such as generation and prediction of diagnosis dialogue and patient consultation are often involved. For example, patients consult their own medical conditions online, artificial intelligence performs preliminary medical diagnosis on patients, and generates and predicts dialogue.

[0003] The method commonly used in the medical field at present in the field of multi-round dialogue generation is to use a hierarchical sequence generation model, that is, to generate the current sentence using the context information. During the training of the model, all dialogue sentences except the last sentence of the training dialogue text are usually used as training samples to predict and generate the last sentence of the dialogue text.

[0004] In the above method, on the one hand, since the training sample is a whole, the information amount is usually large, which can easily lead to the model failing to converge quickly. On the other hand, since the content of the training sample is relatively stable during the training of the model, the model is prone to overfitting. The above two situations will reduce the accuracy of the model training. SUMMARY

[0005] The present application provides a dialogue text generation training method and device based on random sampling and a computer readable storage medium, which mainly aims to improve the accuracy of the dialogue text generation model in the medical diagnosis field.

[0006] To achieve the above purpose, the present application provides a dialogue text generation training method based on random sampling, comprising:

[0007] randomly selecting a sentence of text in the preset dialogue text as a cutting point, composing a first training dialogue text from the text before the cutting point, and generating a predicted text of the cutting point by using a pre-constructed dialogue generation model according to the first training dialogue text;

[0008] calculating a first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point;

[0009] replacing the real text of the cutting point with the predicted text of the cutting point, using all dialogue texts except the last sentence of the text in the replaced preset dialogue text as a second training dialogue text, and generating a vector of the predicted text of the last sentence of the preset dialogue text, calculating a second error between the vector of the real text of the last sentence and the vector of the predicted text of the last sentence.

[0010] calculating a total error according to the first error and the second error, and obtaining the dialogue generation model when the total error meets a preset condition, and returning to the step of randomly selecting a text in the preset dialogue text as a cutting point when the preset condition is not met.

[0011] Optionally, the generating, by the pre-constructed dialogue generation model, of the predicted text of the cutting point according to the first training dialogue text comprises:

[0012] extracting, by an encoder of the dialogue generation model, features of the first training dialogue text to obtain a feature vector corresponding to the first training dialogue text;

[0013] generating, by a decoder of the dialogue generation model, the predicted text of the cutting point according to the feature vector.

[0014] Optionally, the calculating of the first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point comprises:

[0015] calculating the first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point by a loss function .

[0016] wherein, represents a total number of cutting points in the preset dialogue text, represents a vector of a real text of an nth cutting point, represents a vector of a predicted text of the nth cutting point, t represents an iteration number of the dialogue generation model in a training process, represents an error between the vector of the predicted text of the nth cutting point and the vector of the real text of the nth cutting point in the iteration number of the training. t

[0017] Optionally, the generating of the vector of the predicted text of the last text of the preset dialogue text comprises:

[0018] sequentially performing vector conversion on each text in the second training dialogue text to obtain a real vector of each text;

[0019] calculating a similarity weight of each real text and each predicted text by using the real vector of each text and the predicted vector of each text;

[0020] generating a similarity vector of the second training dialogue text by using the similarity weight of the vector of each real text and the vector of each predicted text.​​​

[0021] adding the similarity vector and the predicted vector of each of the texts to obtain a predicted vector of the last sentence of the preset dialogue text.

[0022] Optionally, the calculating of the similarity weight of each of the real texts and each of the predicted texts by using the real vector of each of the texts and the predicted vector of each of the texts comprises:

[0023] sequentially calculating the similarity between the real vector and the corresponding predicted vector of each of the texts;

[0024] normalizing each of the similarities to obtain the similarity weight of each of the real texts and each of the predicted texts.

[0025] Optionally, the normalizing of each of the similarities to obtain the similarity weight of each of the real texts and each of the predicted texts comprises:

[0026] normalizing each of the similarities by using a preset activation function;

[0027]

[0028] wherein, similarity between the real vector and the predicted vector of the i th text in the second training dialogue text ， similarity between the real vector and the predicted vector of the j th text in the second training dialogue text, similarity weight of the i th real text and the i th predicted text in the second training dialogue text.

[0029] In addition, to achieve the above-mentioned purpose, the present application also provides a dialogue text generation training device for random sampling, which comprises:

[0030] a first training text generation module, configured to randomly select a sentence in a preset dialogue text as a cutting point, to group the texts before the cutting point into a first training dialogue text, and to generate a predicted text of the cutting point according to the first training dialogue text by using a pre-constructed dialogue generation model;

[0031] a first error calculation module, configured to calculate a first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point;

[0032] The second training text generation module is configured to replace the predicted text of the cutting point with the real text of the cutting point, and use all the dialogue texts in the preset dialogue text except the last dialogue text as a second training dialogue text.

[0033] The second error calculation module is configured to generate a vector of the predicted text of the last dialogue text, and calculate a second error between the vector of the predicted text of the last dialogue text and a vector of the real text of the last dialogue text.

[0034] The model training module is configured to calculate a total error according to the first error and the second error, adjust model parameters of the dialogue generation model by using the total error, and return to the step of randomly cutting the preset dialogue text to obtain the first training dialogue text until the total error meets a preset condition, thereby obtaining the trained dialogue generation model.

[0035] To solve the above problems, the present application further provides an electronic device, which comprises:

[0036] at least one processor; and

[0037] a memory connected with the at least one processor; wherein

[0038] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the above-mentioned randomly sampled dialogue text generation training method.

[0039] In addition, to achieve the above-mentioned purpose, the present application further provides a computer readable storage medium, which stores a randomly sampled dialogue text generation training program executable by one or more processors to implement the steps of the above-mentioned randomly sampled dialogue text generation training method.

[0040] The present application can guarantee that the first training dialogue text generated after cutting has a small amount of information relative to the second training dialogue text, which is beneficial to accelerate the convergence speed of the dialogue generation model in the training process. On the other hand, after replacing the predicted text of the cutting point with the real text of the cutting point, the phenomenon of overfitting of the dialogue generation model in the training process caused by the solidification of the second training dialogue text can be prevented. Therefore, the randomly sampled dialogue text generation training method, device and computer readable storage medium provided by the present application can improve the accuracy of the dialogue generation model in the medical diagnosis field. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A flowchart of a random sampling dialogue text generation training method provided by an embodiment of the present application is shown in FIG. 1.

[0042] Figure 2 A detailed flowchart of a step in the random sampling dialogue text generation training method provided by an embodiment of the present application is shown in FIG. 2.

[0043] Figure 3 An internal structure diagram of a random sampling dialogue text generation training device provided by an embodiment of the present application is shown in FIG. 3.

[0044] Figure 4 A module diagram of a random sampling dialogue text generation training program in the random sampling dialogue text generation training device provided by an embodiment of the present application is shown in FIG. 4.

[0045] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are merely intended to explain the present application and not to limit the present application.

[0047] An embodiment of the present application provides a random sampling dialogue text generation training method. The execution subject of the random sampling dialogue text generation training method includes but is not limited to at least one of electronic devices such as a server and a terminal which can be configured to execute the method provided by the embodiment of the present application. In other words, the random sampling dialogue text generation training method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and big data and artificial intelligence platforms, etc. basic cloud computing services.

[0048] Referring to Figure 1 A flowchart of a random sampling dialogue text generation training method provided by an embodiment of the present application is shown in FIG. 1. The method can be executed by a device which can be implemented by software and / or hardware. In this embodiment, the random sampling dialogue text generation training method includes:

[0049] S1, randomly selecting a sentence of text in a preset dialogue text as a cutting point, composing a first training dialogue text before the cutting point, and generating a predicted text of the cutting point according to the first training dialogue text by using a pre-constructed dialogue generation model;

[0050] In the embodiment of the present application, the preset dialogue text is composed of no less than two sentences of natural language, which can be a dialogue between a doctor and a patient in the medical field, a dialogue of the doctor diagnosing the patient's condition, a dialogue of the patient making an online appointment for consultation, or a training dialogue text obtained from a specified open source natural language learning model corpus, or a dialogue constructed by using text information scraped from a specified medical platform website or the like using a Python script with data scraping capability.

[0051] Further, before the preset dialogue text is randomly cut, the dialogue text can be preprocessed by removing stop words and useless symbols to obtain the training dialogue text. For example, the dialogue text is "You have the first operation tomorrow morning, and you must fast and abstain from water after dinner tonight!", and the stop words such as "ah, oh" and the symbols such as the colon and exclamation mark are removed. Exemplarily, the preset dialogue text includes N sentences, which can be represented as {U1, U2…UN}, and the position of the cutting point is generated by using a random method. Assuming that the position of the cutting point is the fifth sentence, the first training dialogue text can be represented as {U1, U2, U3, U4}.

[0052] In the present application, the pre-constructed dialogue generation model includes an encoder and a decoder constructed based on a recurrent neural network.

[0053] In detail, the pre-constructed dialogue generation model is used to generate the predicted text of the cutting point according to the first training dialogue text, which includes:

[0054] The encoder of the dialogue generation model is used to extract the features of the first training dialogue text to obtain a feature vector corresponding to the first training dialogue text;

[0055] The decoder of the dialogue generation model is used to generate the predicted text of the cutting point according to the feature vector.

[0056] In the embodiment of the present application, the encoder of the pre-constructed dialogue generation model can convert the dialogue text expressed in natural language into a digital vector recognizable by a computer, and the decoder converts the feature vector into a text output.

[0057] S2, calculate the first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point, replace the real text of the cutting point with the predicted text of the cutting point, and use all the dialogue texts except the last sentence of the replaced preset dialogue text as the second training dialogue text;

[0058] Exemplarily, the predicted text of the cutting point is U4', the real text of the cutting point is U4, and the error value L1 between U4' and U4 is calculated by using the following cross-entropy loss function:

[0059]

[0060] wherein, represents the total number of cutting points in the preset dialogue text, represents the vector of the real text of the nth cutting point, represents the vector of the predicted text of the nth cutting point, t represents the number of iterations of the dialogue generation model in the training process, represents the error between the real vector of the text and the predicted vector of the text in the first iteration training. t

[0061] Exemplarily, the second training dialogue text can be represented as {U1, U2..U4'..U9}.

[0062] In the example of the present application, replacing the real text of the cutting point with the predicted text of the cutting point can further predict the second training text composed after the replacement on the basis of the predicted cutting point text, increase the training amount of the medical related diagnosis dialogue generation model, and effectively improve the accuracy of the medical related diagnosis dialogue generation model.

[0063] S3, generating a vector of the predicted text of the last sentence of the preset dialogue text, and calculating a second error between the vector of the real text of the last sentence and the vector of the predicted text of the last sentence.

[0064] In the example of the present application, since the medical related diagnosis dialogue text is composed of natural language, directly analyzing the medical related diagnosis dialogue text will occupy a large amount of computing resources, resulting in low analysis efficiency. Therefore, the medical related diagnosis dialogue text can be converted into a vector, and then the medical related diagnosis dialogue text can be converted into a vector form.

[0065] In the example of the present application, the predicted vector is obtained by inputting the second training dialogue text into a decoder in the dialogue generation model.

[0066] In detail, referring to FIG. 6, the method for generating a medical related diagnosis dialogue generation model comprises the following steps: Figure 2 As shown in FIG. 6, the step of generating a vector of the predicted text of the last sentence of the preset dialogue text comprises:

[0067] S21, sequentially converting each text in the second training dialogue text into a vector to obtain a real vector of each text. ​​​

[0068] S22. Calculate the similarity weight between each real text and each predicted text using the real vector and the predicted vector of each text.

[0069] S23. Generate a similarity vector for the second training dialogue text using the similarity weights of the vector of each real text and the similarity weights of each predicted text;

[0070] S24. Add the similarity vector to the prediction vector of each text to obtain the vector of the predicted text of the last sentence of the preset dialogue text.

[0071] Further, the step of calculating the similarity weight between each real text and each predicted text using the real vector and the predicted vector of each text includes:

[0072] The similarity between the true vector and the corresponding predicted vector of each text is calculated sequentially.

[0073] Normalization is performed on each of the similarities to obtain the similarity weight between each of the real texts and each of the predicted texts.

[0074] In an optional embodiment of the present invention, the following preset activation function can be used to normalize each similarity:

[0075]

[0076] in, This represents the similarity between the true vector and the predicted vector of the i-th text in the second training dialogue text. ， This represents the similarity between the true vector and the predicted vector of the j-th text in the second training dialogue text. This represents the similarity weight between the i-th real text and the i-th predicted text in the second training dialogue text.

[0077] It should be noted that the calculation method for the second error is the same as that for the first error, and will not be repeated here.

[0078] In this invention, the method of calculating the second error by predicting the vector can prevent a large deviation in the semantic meaning of the medical-related diagnosis between the second training dialogue text containing the cut-off point and the preset dialogue text, thus ensuring the accuracy of model training.

[0079] S4, calculating a total error according to the first error and the second error, when the total error meets a preset condition, obtaining a dialogue generation model, when the preset condition is not met, returning to the step of randomly selecting a sentence of text as a cutting point in the preset dialogue text.

[0080] In the embodiment of the present application, the preset training condition can be that when the comprehensive loss value is less than or equal to a preset loss threshold, the training of the text classification model is stopped.

[0081] In another optional embodiment of the present application, the training of the dialogue generation model can also be controlled by limiting the number of iterations of the training of the text classification model, for example, when the number of iterations of the training of the dialogue generation model reaches a preset maximum number of iterations, the training of the dialogue generation model is stopped.

[0082] In the example of the present application, the dialogue generation model parameter can be a configuration variable inside the model, the value of which can be estimated according to data, and the model parameter can be obtained from a specified algorithm using pytorch.

[0083] Further, the generated total error is gradient reversed, and the model parameter is continuously adjusted, and the gradient reversal can be multiplying the gradient by a negative number and then performing back propagation.

[0084] In the example of the present application, when the total error is used for gradient reversal adjustment of the model training, the gradient is calculated according to the total error, and the model parameter is updated, so that the training effect is generated.

[0085] In the embodiment of the present application, on the one hand, the preset dialogue text is randomly cut, and the prediction text is obtained by predicting the text of the cutting point, and on the other hand, the prediction text of the cutting point is used to replace the real text of the cutting point to combine a new text for prediction, and different errors are obtained by calculating the two predictions, the parameters of the dialogue generation model are adjusted using the different errors, and it is judged whether the parameters meet a preset condition, if not, the text is randomly cut again, if yes, the adjusted dialogue generation model is obtained, through the random cutting of the dialogue text and the replacement of the real text with the prediction text, the convergence speed of the dialogue generation model is improved and the overfitting phenomenon is prevented, thereby improving the accuracy of the dialogue generation model in the medical diagnosis field.

[0086] As Figure 3 shown, a functional module diagram of a random sampling dialogue text generation training device provided by an embodiment of the present application.

[0087] The random sampling dialogue text generation training device 100 can be installed in an electronic device. According to the functions implemented, the random sampling dialogue text generation training device 100 comprises a first training text generation module 101, a first error calculation module 102, a second training text generation module 103, a second error calculation module 104, and a model training module 105. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, which are stored in the memory of the electronic device.

[0088] In the present embodiment, the functions of each module / unit are as follows:

[0089] The first training text generation module 101 is configured to randomly select a sentence of text as a cutting point in a preset dialogue text, compose a first training dialogue text from the text before the cutting point, and generate a predicted text of the cutting point according to a pre-built dialogue generation model;

[0090] The first error calculation module 102 is configured to calculate a first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point;

[0091] The second training text generation module 103 is configured to replace the real text of the cutting point with the predicted text of the cutting point, and use all dialogue texts in the preset dialogue text except the last sentence of text as a second training dialogue text;

[0092] The second error calculation module 104 is configured to generate a vector of a predicted text of the last sentence of text of the preset dialogue text, and calculate a second error between the vector of the predicted text of the last sentence of text and the vector of the real text of the last sentence of text;

[0093] The model training module 105 is configured to calculate a total error according to the first error and the second error, adjust the model parameters of the dialogue generation model using the total error, and obtain a trained dialogue generation model when the total error meets a preset condition.

[0094] In detail, the specific implementation of each module of the random sampling dialogue text generation training device 100 is as follows:

[0095] Step one, cutting at a random position of a preset dialogue text, composing a first training dialogue text from the text before the cutting point, and generating a predicted text of the cutting point according to a pre-built dialogue generation model;

[0096] In the embodiment of the present application, the preset dialogue text is composed of no less than two sentences of natural language, which can be a dialogue between a doctor and a patient in the medical field, a dialogue of the doctor diagnosing the patient's condition, a dialogue of the patient making an online appointment for consultation, or a training dialogue text obtained from a specified open source natural language learning model corpus, or a dialogue constructed by using Python scripts with data grabbing capability to grab text information from a specified medical platform website.

[0097] Further, before the preset dialogue text is randomly cut, the dialogue text can be preprocessed by removing stop words and useless symbols to obtain the training dialogue text. For example, the dialogue text is "You have the first operation in the morning tomorrow, and you must fast and abstain from water after dinner tonight!", and the stop words such as "ah, oh" and the symbols such as the colon and exclamation mark are removed.

[0098] Exemplarily, the preset dialogue text includes N sentences, which can be represented as {U1, U2…UN}, and the position of the cutting point is generated by using a random method. Assuming that the position of the cutting point is the fifth sentence, the first training dialogue text can be represented as {U1, U2, U3, U4}.

[0099] In the example of the present application, the pre-constructed dialogue generation model includes an encoder and a decoder constructed based on a recurrent neural network.

[0100] In detail, the pre-constructed dialogue generation model is used to generate the predicted text of the cutting point according to the first training dialogue text, which includes:

[0101] The first training dialogue text is sequentially input into the encoder of the model to obtain the context feature vector of the first training dialogue text.

[0102] The context feature vector is input into the decoder of the model to obtain the predicted text of the cutting point.

[0103] In the embodiment of the present application, the encoder of the pre-constructed dialogue generation model can convert the dialogue text expressed by natural language into a digital vector recognizable by a computer, and the decoder converts the feature vector into a text output.

[0104] Step two, calculating a first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point, replacing the real text of the cutting point with the predicted text of the cutting point, and taking all the dialogue texts of the preset dialogue text except the last sentence as a second training dialogue text.

[0105] Exemplarily, the medical prediction text of the cutting point is U4', the medical true text of the cutting point is U4, and the error value L1 between U4' and U4 is calculated by using the following cross-entropy loss function:

[0106]

[0107] wherein, represents the total number of cutting points in the preset dialogue text, represents the vector of the true text of the nth cutting point, represents the vector of the prediction text of the nth cutting point, t represents the number of iterations of the dialogue generation model in the training process, represents the error between the prediction text and the true text of the nth cutting point in the first iteration training. t

[0108] Exemplarily, the second training dialogue text can be represented as {U1, U2..U4'..U9}.

[0109] In the example of the present application, replacing the true text of the cutting point with the prediction text of the cutting point can further predict the second training text composed after the replacement on the basis of the prediction text of the cutting point, increase the training amount of the medical related diagnosis dialogue generation model, and effectively improve the accuracy of the medical related diagnosis dialogue generation model.

[0110] Step three, generating the vector of the prediction text of the last sentence of the preset dialogue text, and calculating the second error between the vector of the true text of the last sentence and the vector of the prediction text of the last sentence.

[0111] In the example of the present application, since the medical related diagnosis dialogue text is composed of natural language, directly analyzing the medical related diagnosis dialogue text will occupy a large amount of computing resources, resulting in low analysis efficiency. Therefore, the medical related diagnosis dialogue text can be converted into a vector, and then the medical related diagnosis dialogue text can be converted into a vector form.

[0112] In the example of the present application, the prediction vector is obtained by inputting the second training dialogue text into a decoder in the dialogue generation model.

[0113] In detail, the generation of the vector of the prediction text of the last sentence of the preset dialogue text includes: sequentially converting each text in the second training dialogue text into a vector to obtain the true vector of each text.

[0114] ​​​The similarity weight of each real text and each predicted text is calculated by using the real vector of each text and the predicted vector of each text.

[0115] The similarity vector of the second training dialogue text is generated by using the similarity weight of each real text and each predicted text.

[0116] Further, the similarity weight of each real text and each predicted text is calculated by using the real vector of each text and the predicted vector of each text, comprising:

[0117] The similarity between the real vector and the corresponding predicted vector of each text is calculated in sequence.

[0118] Each similarity is normalized to obtain the similarity weight of each real text and each predicted text.

[0119] In an optional embodiment of the present application, the following preset activation function can be used for normalizing each similarity:

[0120]

[0121] Wherein, represents the similarity between the real vector and the predicted vector of the i th text in the second training dialogue text ， represents the similarity between the real vector and the predicted vector of the j th text in the second training dialogue text, represents the similarity weight of the i th real text and the i th predicted text in the second training dialogue text.

[0122] It should be noted that the calculation method of the second error and the first error is similar, which will not be repeated here.

[0123] In the example of the present application, the way of calculating the second error by the predicted vector can prevent the second training dialogue text containing the predicted text of the cutting point from having a large deviation in the semantic meaning of the relevant medical diagnosis between the predicted dialogue text and the preset dialogue text, and ensure the accuracy of model training.

[0124] Step four, the total error is calculated according to the first error and the second error, when the total error meets the preset condition, the dialogue generation model is obtained, when the preset condition is not met, the step of randomly selecting a text in the preset dialogue text as a cutting point is returned.

[0125] In the embodiment of the present application, the preset training condition can be that when the comprehensive loss value is less than or equal to a preset loss threshold, the training of the text classification model is stopped.

[0126] In another optional embodiment of the present application, the training of the text classification model can also be controlled by limiting the number of iterations of the training of the text classification model, for example, when the number of iterations of the training of the text classification model reaches a preset maximum number of iterations, the training of the text classification model is stopped.

[0127] In the example of the present application, the model parameter can be a configuration variable inside the model, and the value thereof can be estimated according to data, and the model parameter can be obtained from a specified algorithm by using pytorch.

[0128] Further, the generated total error is gradient reversed, and the model parameter is continuously adjusted, and the gradient reversal can be that the gradient is multiplied by a negative number and then back propagated.

[0129] In the example of the present application, when the model training is adjusted by using the total error gradient reversal, the gradient is calculated according to the total error, and the model parameter is updated, so that the training effect is generated.

[0130] The random sampling dialogue text generation training device provided in the present application, on the one hand, randomly cuts the preset dialogue text, and predicts the text at the cutting point to obtain a predicted text, and on the other hand, the predicted text at the cutting point is used to replace the real text at the cutting point to combine a new text for prediction, different errors are obtained by calculating two times of prediction, the parameters of the dialogue generation model are adjusted by using the different errors, whether the parameters meet a preset condition is judged, if not, the random cutting of the text is restarted, and if yes, the adjusted dialogue generation model is obtained, through the random cutting of the dialogue text and the replacement of the real text with the predicted text, the convergence speed of the dialogue generation model is improved and the overfitting phenomenon is prevented, so that the accuracy of the dialogue generation model in the medical diagnosis field is improved.

[0131] As Figure 4 shown is a structural schematic diagram of an electronic device for implementing a random sampling-based dialogue text generation training method according to an embodiment of the present application.

[0132] The electronic device 1 can include a processor 10, a memory 11 and a bus, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a random sampling-based dialogue text generation training.

[0133] The memory 11 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used to store application software and various data installed in the electronic device 1, such as code for training of a dialogue text generation based on random sampling, and can also be used to temporarily store data that has been output or will be output.

[0134] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 10 is a control unit of the electronic device, which connects various components of the electronic device through various interfaces and lines, executes programs or modules stored in the memory 11 (such as training of a dialogue text generation based on random sampling), and calls data stored in the memory 11, to perform various functions and process data of the electronic device 1.

[0135] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0136] Figure 4 Only an electronic device with components is shown, and those skilled in the art can understand that, Figure 4The illustrated structure does not constitute a limitation on the electronic device 1, and can include fewer or more components than illustrated, or combine certain components, or different component arrangements.

[0137] For example, although not shown, the electronic device 1 can also include a power source (such as a battery) to power the various components, and preferably the power source can be logically connected to the at least one processor 10 through a power management device, so that the power management device can implement functions such as charge management, discharge management, and power consumption management. The power source can also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0138] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device 1 and other electronic devices.

[0139] Optionally, the electronic device 1 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 1 and to display a visualized user interface.

[0140] It should be understood that the embodiments are for illustration only and are not limited in scope by the structure.

[0141] The random sampling-based dialogue text generation training stored in the memory 11 in the electronic device 1 is a combination of a plurality of instructions, which, when executed in the processor 10, can implement:

[0142] randomly selecting a sentence of text in the preset dialogue text as a cutting point, composing a first training dialogue text from the text before the cutting point, and using a pre-constructed dialogue generation model to generate a predicted text of the cutting point according to the first training dialogue text;

[0143] calculating a first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point;

[0144] replace the real text of the cutting point with the predicted text of the cutting point, and use all the dialogue texts in the preset dialogue text except the last sentence text as second training dialogue texts, and generate a vector of the predicted text of the last sentence text of the preset dialogue text, and calculate a second error between the vector of the real text of the last sentence text and the vector of the predicted text of the last sentence text;

[0145] calculate a total error according to the first error and the second error, obtain a dialogue generation model when the total error meets a preset condition, and return the step of randomly selecting a text in the preset dialogue text as a cutting point when the preset condition is not met.

[0146] Further, the modules / units of the electronic device 1 are implemented in the form of software function units and sold or used as independent products, which can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory).

[0147] The application also provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device:

[0148] randomly select a text in a preset dialogue text as a cutting point, compose a first training dialogue text from the text before the cutting point, and generate a predicted text of the cutting point according to the first training dialogue text by using a pre-constructed dialogue generation model;

[0149] calculate a first error between a vector of the predicted text of the cutting point and a vector of the real text of the cutting point;

[0150] replace the real text of the cutting point with the predicted text of the cutting point, and use all the dialogue texts in the preset dialogue text except the last sentence text as second training dialogue texts, and generate a vector of the predicted text of the last sentence text of the preset dialogue text, and calculate a second error between the vector of the real text of the last sentence text and the vector of the predicted text of the last sentence text;

[0151] calculate a total error according to the first error and the second error, obtain a dialogue generation model when the total error meets a preset condition, and return the step of randomly selecting a text in the preset dialogue text as a cutting point when the preset condition is not met.

[0152] In addition, each functional module in various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0153] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be realized in other specific forms without departing from the spirit or essential characteristics of the present application.

[0154] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims involved.

[0155] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. Blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptographic methods, each data block containing information of a batch of network transactions, for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0156] The embodiments of the present application can acquire and process related data based on holographic projection technology. Among them, artificial intelligence (Artificial Intelligence, AI) is to use digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0157] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the system claims can also be realized by one unit or device through software or hardware. The second word is used to indicate the name, and does not mean any specific order.

[0158] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for training a random sampling dialogue generation, characterized in that, The method comprises: randomly selecting a sentence of text in a preset dialogue text as a cutting point, composing a first training dialogue text by the text before the cutting point, generating a predicted text of the cutting point according to the first training dialogue text by using a pre-constructed dialogue generation model; calculating a first error between the vector of the predicted text of the cutting point and the vector of the real text of the cutting point; replacing the real text of the cutting point with the predicted text of the cutting point, taking all dialogue texts except the last sentence of text in the preset dialogue text after the replacement as a second training dialogue text, and generating a vector of a predicted text of the last sentence of text of the preset dialogue text, calculating a second error between the vector of the real text of the last sentence of text and the vector of the predicted text of the last sentence of text; calculating a total error according to the first error and the second error, obtaining a dialogue generation model when the total error meets a preset condition, and returning to the step of randomly selecting a sentence of text in a preset dialogue text as a cutting point when the preset condition is not met.

2. The training method of claim 1, wherein, The method comprises: extracting features of the first training dialogue text by using an encoder of the dialogue generation model to obtain a feature vector corresponding to the first training dialogue text; generating the predicted text of the cutting point according to the feature vector by using a decoder of the dialogue generation model.

3. The training method of claim 1, wherein, The method comprises: a first error between a vector of the predicted text of the cut point and a vector of the true text of the cut point is calculated by a loss function : wherein, represents the total number of cutting points in the preset dialogue text, represents the vector of the real text of the nth cutting point, represents the vector of the predicted text of the nth cutting point, t represents the number of iterations of the dialogue generation model in the training process, represents the error between the real text and the predicted text of the nth cutting point in the first iteration training, t represents the error between the real text and the predicted text of the nth cutting point in the mth iteration training, represents the error between the real text and the predicted text of the nth cutting point in the first iteration training, represents the error between the real text and the predicted text of the nth cutting point in the mth iteration training.

4. The training method of claim 1, wherein, The method comprises: converting vectors of each text in the second training dialogue text in sequence to obtain a real vector of each text; calculating a similarity weight of each real text and each predicted text by using the real vector of each text and the predicted vector of each text; generating a similarity vector of the second training dialogue text by using the vector of each real text and the similarity weight of each predicted text; adding the similarity vector and the predicted vector of each text to obtain a vector of a predicted text of the last sentence of text of the preset dialogue text.

5. The training method of claim 4, wherein, The method comprises: calculating the similarity between the real vector of each text and the corresponding predicted vector in sequence; normalizing each similarity to obtain a similarity weight of each real text and each predicted text.

6. The training method of claim 5, wherein, The method comprises: normalizing each similarity by using a preset activation function: wherein, represents a similarity between a real vector and a predicted vector of the i-th text in the second training dialogue text ， represents a similarity between a real vector and a predicted vector of the j-th text in the second training dialogue text, represents a similarity weight of the i-th real text and the i-th predicted text in the second training dialogue text. 7.A device for training of a random sampling dialogue text generation, characterized in that, The device comprises: The first training text generation module is configured to select a sentence of text as a cutting point in the preset dialogue text, and generate a first training dialogue text by combining the text before the cutting point, and generate a predicted text of the cutting point by using a pre-built dialogue generation model according to the first training dialogue text. The first error calculation module is configured to calculate a first error between a vector of the predicted text of the cutting point and a vector of the real text of the cutting point. The second training text generation module is configured to replace the real text of the cutting point with the predicted text of the cutting point, and use all dialogue texts in the preset dialogue text except the last sentence of text as a second training dialogue text. The second error calculation module is configured to generate a vector of a predicted text of the last sentence of text of the preset dialogue text, and calculate a second error between the vector of the predicted text of the last sentence of text and a vector of a real text of the last sentence of text. The model training module is configured to calculate a total error according to the first error and the second error, adjust model parameters of the dialogue generation model by using the total error, and obtain a trained dialogue generation model when the total error meets a preset condition.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the randomly sampled dialogue text generation training method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a randomly sampled dialogue text generation training program, and the randomly sampled dialogue text generation training program can be executed by one or more processors to implement the steps of the randomly sampled dialogue text generation training method according to any one of claims 1 to 6.

Images