Dialogue text processing method and device, computer device and storage medium

Abstract

The application relates to the fields of artificial intelligence and medical health, and discloses a dialogue text processing method and device, computer equipment and a storage medium, which comprise the following steps: performing entity extraction on a text vector of dialogue text to obtain a plurality of text entities; performing normalization processing on each text entity to obtain a normalized entity corresponding to each text entity; performing screening on each normalized entity based on an affirmative sentence according to the text vector and each text entity to obtain a target normalized entity in the affirmative sentence; and determining a target entity corresponding to the dialogue text according to the target normalized entity. The technical scheme of the embodiment of the application realizes obtaining of a normalized entity from dialogue text, standardizes the entity, and then determines the normalized entity in the affirmative sentence of the dialogue text as the target entity of the dialogue text, thereby improving the accuracy of target entity determination.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and medical health, and in particular to a dialogue text processing method and device, a computer device and a storage medium. BACKGROUND

[0002] The natural language processing method for dialogue scene understanding is generally applied to online stores, online drugstores, online medical consultation systems and the like, and can realize automatic reply to user questions. For example, in an online medical consultation system, in order to improve disease diagnosis efficiency and reduce the workload of doctors, large medical institutions have launched convenient intelligent consultation systems, which inquire multiple times to users with diagnosis needs according to individual symptoms, and compare the symptom information collected in each dialogue with the symptom information that may appear in various diseases, and gradually exclude a certain disease with the closest symptoms in the continuous matching process. This type of scheme mainly terminates the inquiry and outputs the diagnosis result through the preset value of the dialogue round or the matching degree and the like. Online medical consultation systems such as online health consultation, users can perform online health consultation or consultation and the like.

[0003] However, at present, these online health consultation and consultation and the like, due to the fact that the symptom entities described by patients in actual situations are non-professional terms, the accuracy of matching the symptom entities with standard entities is not high when matching, thereby affecting the diagnosis accuracy and reducing the user experience. SUMMARY

[0004] The present application provides a dialogue text processing method, device, computer device and storage medium to solve the technical problem of low accuracy of entity recognition in the dialogue text at present.

[0005] In a first aspect, an embodiment of the present application provides a dialogue text processing method, the method comprising:

[0006] performing entity extraction on a text vector of dialogue text to obtain a plurality of text entities;

[0007] performing normalization processing on each of the text entities to obtain a normalized entity corresponding to each of the text entities;

[0008] performing filtering on each of the normalized entities based on positive sentences according to the text vector and each of the text entities to obtain a target normalized entity in the positive sentences;

[0009] determining a target entity corresponding to the dialogue text according to the target normalized entity.

[0010] In a second aspect, an embodiment of the present application further provides a dialogue text processing device, the device comprising:

[0011] The text entity acquisition module is configured to perform entity extraction on the text vector of the dialogue text to obtain a plurality of text entities.

[0012] The normalization entity acquisition module is configured to perform normalization processing on each of the text entities to obtain a normalization entity corresponding to each of the text entities.

[0013] The target normalization entity acquisition module is configured to perform affirmative sentence-based screening on each of the normalization entities according to the text vector and each of the text entities to obtain a target normalization entity in an affirmative sentence.

[0014] The target entity determination module is configured to determine a target entity corresponding to the dialogue text according to the target normalization entity.

[0015] In a third aspect, an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the dialogue text processing method when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the steps of the dialogue text processing method.

[0017] In the scheme implemented by the dialogue text processing method, the dialogue text processing device, the computer device, and the storage medium, a plurality of text entities can be obtained by performing entity extraction on a text vector of dialogue text. Each text entity is normalized to obtain a normalization entity corresponding to each text entity. Each normalization entity is screened based on an affirmative sentence according to the text vector and each text entity to obtain a target normalization entity in an affirmative sentence. A target entity corresponding to the dialogue text is determined according to the target normalization entity. The embodiment of the present application realizes obtaining text entities according to a text vector, then normalizing the text entities to standardize the text entities, and screening the normalization entities obtained by normalization processing in an affirmative sentence to obtain a target entity of the dialogue text. Since the normalization entities in a negative sentence are removed, the target entity is more consistent with the text semantic information in the dialogue text, and thus the target entity is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is an application environment schematic diagram of a dialogue text processing method in an embodiment of the present application;

[0020] Figure 2 is a flow schematic diagram of a dialogue text processing method in an embodiment of the present application;

[0021] Figure 3 is a structure schematic diagram of a dialogue text processing device in an embodiment of the present application;

[0022] Figure 4 is a structure schematic diagram of a computer device in an embodiment of the present application;

[0023] Figure 5 is a structure schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0025] The dialogue text processing method provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 , wherein the client communicates with the server through the network. The server can receive dialogue text through the client, and obtain a plurality of text entities by performing entity extraction on the text vector of the dialogue text. The normalization processing is performed on each text entity to obtain a normalized entity corresponding to each text entity. The screening of each normalized entity based on the affirmative sentence is performed according to the text vector and each text entity to obtain a target normalized entity in the affirmative sentence. The target entity corresponding to the dialogue text is determined according to the target normalized entity. The target entity is fed back to the client.

[0026] At present, the consultation and inquiry platform is very popular with users. The user can communicate with the platform at any time, and even if there is no manual reply, the user can reply through the robot to solve the user's urgent problems, such as the inquiry APP, which can realize online health consultation and inquiry. However, since the symptoms described by the user are not professional terms, the accuracy of matching the symptom entity in the description text of the user with the standard entity is not high, which further leads to the fact that the reply obtained according to the matching error standard entity does not match the description text of the user well, that is, the reply fed back to the user is not accurate enough, which affects the user experience.

[0027] Therefore, the embodiment of the present application obtains the text entity through the text vector, then normalizes the text entity to standardize the text entity, judges the positive sentence of the normalized entity obtained through the normalization, and screens the normalized entity in the positive sentence as the target entity of the dialogue text. The scheme of the embodiment of the present application improves the accuracy of the target entity in the dialogue text. The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. The present application will be described in detail through specific embodiments.

[0028] In an embodiment of the present application, a dialogue text processing method is provided. The dialogue text processing method can be used to identify entities in dialogue text. The dialogue text processing method can be executed by a dialogue text processing device, which can be implemented in the form of software and / or hardware. Figure 2 As shown in the figure, the dialogue text processing method provided by the embodiment of the present application specifically includes the following steps:

[0029] S110, entity extraction is performed on the text vector of the dialogue text to obtain a plurality of text entities.

[0030] The dialogue text includes dialogue text for inquiry and health consultation, including dialogue text between a user and a doctor, and dialogue text between a user and a system. For example, in a health consultation APP, the user and the doctor have an online dialogue on a platform, or the system can recognize and match the input text of the user to a suitable reply text to have a dialogue with the user. The dialogue text can also be dialogue text of a user on an online shopping platform, such as dialogue text between the user and a merchant. The dialogue text can also be some information platform.

[0031] For example, in the case of inquiry, it can be: A: I have a headache, a sore throat, and a high fever of 39 degrees. B: Do you have a cough? A: No. It should be understood that A and B are only used to distinguish the two parties in the dialogue and have no actual meaning.

[0032] Specifically, entity extraction is performed on the text vector of the dialogue text to obtain a plurality of text entities. This step realizes extraction of text entities in the dialogue text and prepares for subsequent entity normalization.

[0033] Optionally, the text vector of the dialogue text can be subjected to text entity extraction by an entity recognition method to obtain a plurality of text entities. The entity recognition method includes a Chinese named entity recognition (NER) algorithm, and the named entity recognition algorithm includes but is not limited to a rule and dictionary-based named entity recognition method, a statistical machine learning-based named entity recognition method, a deep learning-based named entity recognition method, and an attention mechanism-based named entity recognition method, etc.

[0034] It should be noted that in the embodiments of the present application, after obtaining the dialogue text, the dialogue text is subjected to vectorization representation, which can be realized by an embedding layer of a bidirectional long short-term memory model (Bi-directional Long Short-Term Memory, BiLSTM) to obtain a text vector.

[0035] S120, normalizing each text entity to obtain a normalized entity corresponding to each text entity.

[0036] Specifically, each text entity is subjected to normalization processing to obtain a normalized entity corresponding to each text entity, so as to realize standardization of the text entity and prepare for subsequent screening of the normalized entity in the affirmative sentence.

[0037] S130, screening each normalized entity based on the affirmative sentence according to the text vector and each text entity to obtain a target normalized entity in the affirmative sentence.

[0038] Specifically, the position of each text entity in the text vector can be determined according to the text vector, the text entity in the affirmative sentence is determined according to the position and the text vector, and then the normalized entity corresponding to the text entity in the affirmative sentence is taken as the target normalized entity. The scheme of the embodiments of the present application realizes judging whether each sentence in the dialogue text is an affirmative sentence or a negative sentence according to the text vector. According to the position of the text entity in the text vector, it is judged whether the text entity is located in the affirmative sentence, and the normalized entity corresponding to the text entity located in the affirmative sentence is taken as the target normalized entity, thereby improving the accuracy of determining the target normalized entity in the dialogue text. The affirmative sentence can be, for example, “B: Do you cough? A: Yes”. Alternatively, the affirmative sentence can also be “A: I have a headache. B: I understand”.

[0039] S140, determining a target entity corresponding to the dialogue text according to the target normalized entity.

[0040] The target entity refers to the final text entity obtained in the dialogue text.

[0041] Specifically, in the embodiment of the present application, the target entity corresponding to the dialogue text is determined according to the target normalized entity, so that the target entity in the dialogue text is more accurate. Optionally, the target normalized entity is taken as the target entity of the dialogue text.

[0042] The dialogue text processing method of the embodiment of the present application extracts entities from the text vector of the dialogue text to obtain a plurality of text entities. Each text entity is normalized to obtain a normalized entity corresponding to each text entity. According to the text vector and each text entity, each normalized entity is screened based on affirmative sentences to obtain a target normalized entity in the affirmative sentence. The target entity corresponding to the dialogue text is determined according to the target normalized entity. The embodiment of the present application realizes obtaining text entities from a text vector, then normalizing the text entities to standardize the text entities, and judging the normalized entities obtained by the normalization based on affirmative sentences to screen the normalized entities in the affirmative sentences as the target entity of the dialogue text. Since the normalized entities in the negative sentences are eliminated, the target entity is more accurate and conforms to the text semantic information in the dialogue text.

[0043] In another embodiment of the present application, the normalization processing of each text entity to obtain a normalized entity corresponding to each text entity comprises: matching each standard entity and each text entity through a pre-trained normalization model to obtain a matching degree of each text entity and each standard entity; and taking the standard entity with the highest matching degree as the normalized entity of the text entity for each text entity.

[0044] The normalization model is pre-trained. The normalization model includes but is not limited to a Wide & Deep model, an AutoRec model, etc. The standard entity refers to various entity terms that are the same as or similar to professional terms, such as “headache”, “mild headache”, “intermittent headache”, etc. Optionally, the standard entity can be stored in a standard entity library, and each standard entity is obtained when the text entity is normalized.

[0045] Specifically, each standard entity and each text entity are matched through a pre-trained normalization model to obtain a matching degree of each text entity and each standard entity, and the standard entity with the highest matching degree is taken as the normalized entity of the text entity for each text entity. The embodiment of the present application realizes the normalization processing of the text entity and the standardization of the text entity.

[0046] In another embodiment of the present application, before the normalization processing of the text entity is performed, a part of the candidate standard entities corresponding to each text entity is extracted from the standard entity library. The extraction can be performed by similarity matching between the text entity and each standard entity in the standard entity library to obtain a part of the candidate standard entities corresponding to each text entity. For each text entity, the normalization model is used to match the text entity and each candidate entity to obtain a matching degree of the text entity and each candidate entity, and the candidate standard entity with the highest matching degree is taken as the normalized entity of the text entity. Through the scheme of the embodiment of the present application, instead of matching each text entity with each standard entity, a part of the candidate standard entities is recalled for each text entity, so that when the normalization model is used to match each text entity and each candidate standard entity, the workload of the normalization model can be reduced, the performance of the normalization model can be improved, and the efficiency of the normalization processing of the text entity can be improved.

[0047] In another embodiment of the present application, before the matching between each standard entity and each text entity is performed by the pre-trained normalization model, the method further includes: performing normalization processing on each training sample pair by an initial model to obtain a normalization processing result, wherein the training sample pair includes a training text entity, a positive sample of the training text entity, and a negative sample of the training text entity; inputting the normalization processing result into a loss function to obtain a first loss value; in a case where the first loss value does not satisfy a preset loss condition, adjusting a model parameter of the initial model and continuing training; in a case where the first loss value satisfies the loss condition or an iteration number reaches a number threshold, taking the initial model as the normalization model.

[0048] Each training sample pair includes a training text entity, a positive sample of the training text entity, and a negative sample of the training text entity. The positive sample of the training text entity can be a term identical to or similar to the training text entity, for example, the training text entity is "headache", and the positive sample can include "headache" and "headache". The negative sample of the training text entity can be a term different from the training text entity, for example, the training text entity is "headache", and the negative sample can include "dry throat" and the like. The loss condition includes but is not limited to convergence of the loss value, the loss value being less than a preset loss threshold, and the loss value being within a loss threshold range.

[0049] Specifically, each training sample pair is processed by the initial model to obtain a normalized processing result, and the normalized processing result is input into a loss function. After obtaining a first loss value, it is determined whether the first loss value satisfies a loss condition. If the loss condition is not satisfied, the model parameters of the initial model are adjusted to obtain an updated initial model. Each training sample pair is processed by the updated initial model to obtain a normalized processing result, and then a new first loss value is obtained. It is determined whether the new first loss value satisfies the loss condition. If the loss condition is not satisfied, the model parameters of the initial model are adjusted, and the training is repeated until the first loss value satisfies the loss condition, and the training is stopped. If the loss condition is satisfied, the initial model at this time is taken as the normalization model. The scheme of the embodiment of the present application realizes the training of the normalization model.

[0050] In another embodiment of the present application, the normalization model is a Wide&Deep model with a parwise structure. The Wide part can perform preliminary processing on the text entity and the standard entity based on a rule library of combed naming bodies related to the text entity, antonyms of the text entity, synonyms of the text entity, degree words of the text entity, etc., to obtain a first matching degree between the text entity and each standard entity. If the text entity is a disease, the related naming body can be a limb or a body part. If the text entity includes a profession, the naming body includes work experience, etc. It should be understood that the words in the rule library can be set as needed, and the naming bodies related to the text entity can also be set as needed. The Deep part can use an Enhanced Sequential Inference Model (ESIM) to match the text entity and the standard entity to obtain a second matching degree. Based on the first matching degree and the second matching degree, the matching degree of the text entity processed by the normalization model and the standard entity is obtained.

[0051] In the training phase, the wide part processes the training text entity, the positive sample of the training text entity, and the negative sample of the training text entity to obtain the first matching degree between the training text entity and the positive sample, and the first matching degree between the training text entity and the negative sample. Similarly, the deep part also obtains the second matching degree between the training text entity and the positive sample, and the second matching degree between the training text entity and the negative sample. According to the first matching degree and the second matching degree between the training text entity and the negative sample, the matching degree between the training text entity and the negative sample is obtained, and the matching degree between the training text entity and the positive sample is obtained in the same way. The matching degree between the training text entity and the negative sample and the matching degree between the training text entity and the positive sample are input into a loss function to calculate a first loss value. In the case of convergence of the first loss value, the initial model can be taken as the normalization model. The loss function is expressed as:

[0052] L(a, p, n) = max(0, -f(a, p) + f(a, n) + margin)

[0053] Wherein, L represents a loss function, a represents a training text entity, p represents a positive sample, n represents a negative sample, f(a, p) is a matching degree obtained by inputting the training text entity and the positive sample into the normalization model. max is a maximum value. f(a, n) is a matching degree obtained by inputting the training text entity and the negative sample into the normalization model. Optionally, margin is a hyperparameter. The matching degree can be a score, for example, the matching degree b ∈ [0, 1]. It should be understood that the matching degree is 1, which means that the training text entity is completely the same as the positive sample, and the matching degree is 0, which means that the training text entity is completely different from the negative sample. It should be understood that in the training phase, the initial model can be trained by the training sample set, that is, multiple training text entities, positive samples of each training text entity and negative samples of each training text entity are input at the same time.

[0054] It should be understood that the text entity, the training text entity, the positive sample, the negative sample and the like in the embodiment of the application are represented in the form of a vector.

[0055] In the use phase of the normalization model, the matching degree between each text entity and each standard entity is obtained through wide and deep processing, and according to the pariwise ranking, the standard entity with the highest matching degree is taken as the normalized entity of the text entity. Moreover, when the standard entities corresponding to the matching degrees are ranked according to pariwise, the first pre-set number of standard entities can also be taken as the intermediate normalized entity of the text entity. The intermediate normalized entity can be taken as the reference target entity of the dialogue text, which can be fed back to the staff, so as to facilitate the staff to quickly understand the content of the dialogue text.

[0056] In another embodiment of the application, the entity extraction on the text vector of the dialogue text is performed to obtain multiple text entities, including: performing entity extraction on the text vector through a pre-trained semantic acquisition model to obtain a target text vector, wherein the target text vector includes context semantic information of the dialogue text; and obtaining multiple text entities by processing the target text vector.

[0057] The semantic acquisition model can be a multi-corpus interaction information model, and can also be other models capable of acquiring context semantic information in the dialogue text. For a dialogue text, a certain text entity can appear in multiple sentences, but the embedding of the same text entity obtained by BiLSTM in different sentences is different because the embedding contains the context information of each sentence. Therefore, the embedding of the same word in each sentence is interacted, the content of the entire dialogue is fully used, and more sufficient information is obtained to determine whether the attitude of a symptom expressed at the current position is positive or negative.

[0058] Specifically, in the embodiment of the present application, the semantic acquisition model pre-trained is used to extract entities from the text vector to obtain a target text vector with context semantic information of the dialogue text. By processing the target text vector, a text entity is obtained. The scheme of the embodiment of the present application extracts the context semantic information in the dialogue text, so that the text entity also has semantic information according to the extraction of the text entity from the target text vector, and the subsequent screening of the target normalized entity in the positive sentence can make the target normalized entity screened more accurate.

[0059] For example, in the training phase of the semantic acquisition model: the jth training text entity m of the ith sentence in the dialogue text is trained ij , and the embedding thereof is h ij , all the same training text entities m ij in the remaining sentences in the training dialogue text are found k1 (assuming that the lth training text entity m ij in the kth sentence is taken as an example), and the embedding thereof (e.g. h kl ) is obtained, the training text entity vectors of these training text entities and h ij are respectively subjected to an attention layer and a normalization exponential function softmax to obtain corresponding weight scores, i.e. a ij , and the training text entity vector H ij based on the entire training dialogue text is obtained.

[0060]

[0061] wherein K is the number of occurrences of m ij in the entire training dialogue text. h ij represents the embedding of the training text entity m ij , and m represents the training text entity.

[0062] In this way, each training text entity vector in the training dialogue text can be obtained. Similarly, if a certain training text entity appears in multiple training dialogue texts and has different meanings in different contexts, the vector representation of each training text entity in the multiple training dialogue texts in the entire corpus can be calculated, and the semantic ambiguity problem can be reduced through the information of the training dialogue texts in the entire corpus. The embedding layer, the intra-dialogue attention vector representation, and the inter-dialogue attention vector representation of each training text entity are spliced, and then a BiLSTM model is used to learn the context features, so as to realize the training of the multi-corpus interaction information model. In this way, the semantic acquisition model trained in this way can improve the semantic representation capability of the text vector when acquiring the semantic text vector of the dialogue text.

[0063] In another embodiment of the present application, the positive sentence-based screening of each normalized entity according to the text vector and each text entity to obtain a target normalized entity in the positive sentence comprises: inputting the target text vector and each text entity into a pre-trained entity screening model to obtain a text entity in the positive sentence; and taking the normalized entity corresponding to the text entity in the positive sentence as the target normalized entity.

[0064] Preferably, the pre-trained entity screening model is a neural network model.

[0065] Specifically, the target text vector and each text entity are input into the pre-trained entity screening model to obtain a text entity in the positive sentence, and the normalized entity corresponding to the text entity in the positive sentence is taken as the target normalized entity. In the embodiment of the present application, the positive sentence and the negative sentence in the dialogue text can be determined according to the text vector, and the text vector in the positive sentence and the text vector in the negative sentence can be obtained through the position of the text entity in the dialogue text. The output of the entity screening model can be a label of the positive sentence or the negative sentence for each text entity. For example, if the text entity is located in the positive sentence, a label 1 is added to the text entity, and if the text entity is located in the negative sentence, a label 0 is added to the text entity. The technical solution of the embodiment of the present application can screen the text entity in the positive sentence from the text entity, and determine the normalized entity corresponding to the text entity in the positive sentence as the target normalized entity, thereby improving the accuracy of the determination of the target normalized entity.

[0066] In another embodiment of the application, before the target text vector and each text entity are input into the pre-trained entity screening model to obtain the text entity in the affirmative sentence, the method further comprises: processing the training text vector and the training text entity through an initial entity screening model to obtain a training entity screening result; obtaining a second loss value according to the training entity screening result, a true label corresponding to the training entity screening result, and an entity correlation loss value, and completing training under the condition that the second loss value meets a training condition, and taking the initial entity screening model after training as the entity screening model, wherein the entity correlation loss value is determined by a co-occurrence frequency of a training normalized entity corresponding to the training text entity and a candidate standard entity, and the candidate standard entity includes each standard entity in a standard entity library except the training normalized entity.

[0067] The co-occurrence frequency refers to a frequency of each two standard entities in the standard entity library pre-acquired appearing in the training dialogue text. For each two standard entities, the co-occurrence frequency of each two standard entities is obtained through each training dialogue text. The pre-acquired co-occurrence frequency can be obtained according to two standard entities. The training text entity is obtained according to the training text vector, and the training normalized entity is obtained by normalizing the training text entity. Each training text entity is provided with a true label, which refers to whether the sentence where the training text entity is located is an affirmative sentence or a negative sentence.

[0068] Specifically, the training text vector and the training text entity are processed through the initial entity screening model to obtain a training text entity screening result, the training text entity screening result, a true label corresponding to the training text entity screening result, and an entity correlation loss value are input into a second loss function to obtain a second loss value, and the initial entity screening model at this time is taken as the entity screening model under the condition that the second loss value meets a training condition. If the second loss value does not meet the training condition, the model parameters of the initial entity screening model are adjusted, and the training is continued. It should be understood that in the training of the initial entity screening model, a plurality of training text vectors and a plurality of training text entities in each training text vector can be used at the same time to reduce the training time of the initial entity model.

[0069] The second loss function expression is a weighted sum of a cross-entropy loss value of a training text entity screening result and a real label and an entity correlation loss value. The training text entity screening result can be a score corresponding to the training text entity, which is used to represent a score of the training text entity being in a positive sentence. The real label can be 1 or 0, 1 indicating that the training text entity is in a positive sentence and 0 indicating that the training text entity is in a negative sentence. In this way, the cross-entropy loss value of the training text entity screening result and the real label is obtained, and the cross-entropy loss value is weighted and summed with the entity correlation loss value to obtain the second loss value.

[0070] The entity correlation loss value can be calculated by the following expression:

[0071]

[0072] wherein w i，j is a co-occurrence frequency between the training text entity y i and the training text entity y j , y represents the training text entity, i and j are used to distinguish different training text entities, i is not equal to j, i = 1, 2, …, n, and j = 1, 2, …, n. The entity correlation loss value can be a possibility of the co-occurrence of the training text entities in the training dialogue text. The greater S is, the lower the possibility is, that is, the lower the correlation between the training text entities is.

[0073] In another embodiment of the present application, after the target entity corresponding to the dialogue text is determined according to the target normalized entity, the method further comprises: obtaining a target reply text corresponding to the target entity according to a mapping relationship between each standard entity and a reply text.

[0074] wherein the reply texts corresponding to each standard entity are set in advance, and the reply texts corresponding to each standard entity include: a reply text corresponding to each standard entity, reply texts of several different standard entities, and the like. A reply text library can be set, and each reply text has at least one corresponding standard entity. There is a corresponding relationship between the reply text and the at least one corresponding standard entity.

[0075] Specifically, the target reply text corresponding to the target entity is obtained according to the mapping relationship between each standard entity and a reply text. The technical scheme of the embodiment of the present application realizes that the reply text can be obtained after the target entity is obtained, the user can obtain a reply when the dialogue ends by feeding back the reply text to the user, and the user experience is improved.

[0076] The embodiment of the present application can feed back the reply text to the user through the display page. For example, in a health consultation APP, the user inputs the symptoms in the dialogue window, the robot or the doctor replies, the dialogue text of the two is obtained in real time, the target entity is obtained according to the dialogue text, the reply text is determined according to the target entity, and the reply text is fed back to the dialogue window, so that the user can know the diagnosis and treatment result according to the symptoms. Since the accuracy of the target entity determined in the embodiment of the present application is high, the accuracy of the reply text obtained according to the target entity is also improved, so that the user can accurately understand the condition of himself, and the user experience is improved.

[0077] For example, the dialogue text obtained from the dialogue window includes 'I have a headache and a fever of 39 degrees. Do you have a cough? No.' The 'headache' and 'high fever' are obtained from the dialogue text. According to the 'headache' and 'high fever' and the corresponding relationship between the standard entity and the reply text, the reply text is obtained as 'flu, suggest taking A medicine, if the high fever does not subside, suggest going to the hospital in time' and the like. Of course, the reply text in the reply text library can be updated at any time as needed. The scheme of the embodiment of the present application can obtain the reply text according to the target entity, reduce the workload of the doctor, and can reply to the user in time, and the user experience is improved.

[0078] In another embodiment of the present application, a dialogue text processing device is also provided. The dialogue text processing device provided by the present application can execute the dialogue text processing method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method. As shown in Figure 3 The device includes a text entity obtaining module 410, a normalized entity obtaining module 420, a target normalized entity obtaining module 430, and a target entity determining module 440, wherein:

[0079] The text entity obtaining module 410 is configured to perform entity extraction on the text vector of the dialogue text to obtain a plurality of text entities. The normalized entity obtaining module 420 is configured to perform normalization processing on each of the text entities to obtain a normalized entity corresponding to each of the text entities. The target normalized entity obtaining module 430 is configured to perform positive sentence-based screening on each of the normalized entities according to the text vector and each of the text entities to obtain a target normalized entity in the positive sentence. The target entity determining module 440 is configured to determine a target entity corresponding to the dialogue text according to the target normalized entity.

[0080] In another embodiment of the present application, the normalized entity obtaining module 420 includes:

[0081] The text entity matching submodule is configured to match each of the text entities with each of the standard entities by using a pre-trained normalization model to obtain a matching degree of each text entity with each standard entity.

[0082] The normalization entity determination submodule is configured to determine, for each of the text entities, the standard entity with the highest matching degree as the normalization entity of the text entity.

[0083] In another embodiment of the present application, the text entity matching submodule includes:

[0084] The normalization processing result acquisition unit is configured to perform normalization processing on each training sample pair by using an initial model to obtain a normalization processing result, wherein the training sample pair includes a training text entity, a positive sample of the training text entity, and a negative sample of the training text entity.

[0085] The first loss value acquisition unit is configured to input the normalization processing result into a loss function to obtain a first loss value.

[0086] The model parameter adjustment unit is configured to adjust model parameters of the initial model and continue training in a case where the first loss value does not satisfy a preset loss condition.

[0087] The normalization model acquisition unit is configured to acquire the initial model as a normalization model in a case where the first loss value satisfies the loss condition.

[0088] In another embodiment of the present application, the text entity acquisition module 410 includes:

[0089] The target text vector acquisition submodule is configured to perform entity extraction on the text vector by using a pre-trained semantic acquisition model to obtain a target text vector, wherein the target text vector includes context semantic information of the dialogue text.

[0090] The first text entity acquisition submodule is configured to obtain a plurality of text entities by processing the target text vector.

[0091] In another embodiment of the present application, the target normalization entity acquisition module 430 includes:

[0092] The second text entity acquisition submodule is configured to input the target text vector and each of the text entities into a pre-trained entity screening model to obtain a text entity in a positive sentence.

[0093] The target normalization entity acquisition submodule is configured to determine, as a target normalization entity, the normalization entity corresponding to the text entity in the positive sentence.

[0094] In another embodiment of the application, the device further comprises:

[0095] The initial entity screening model training module is configured to process the training text vector and the training text entity by using the initial entity screening model to obtain a training entity screening result, obtain a second loss value according to the training entity screening result, a true label of the training entity screening result, and an entity correlation loss value, and complete the training under the condition that the second loss value meets a training condition, and use the initial entity screening model after the training as the entity screening model, wherein the entity correlation loss value is determined by a co-occurrence frequency of a training normalized entity corresponding to the training text entity and a candidate standard entity, and the candidate standard entity includes each standard entity in a standard entity library except the training normalized entity.

[0096] In another embodiment of the application, the device further comprises:

[0097] The target reply text acquisition module is configured to obtain a target reply text corresponding to the target entity according to a mapping relationship between each standard entity and a reply text.

[0098] The dialogue text processing device in the embodiment of the application obtains a plurality of text entities by performing entity extraction on a text vector of a dialogue text. Each text entity is normalized to obtain a normalized entity corresponding to each text entity. Each normalized entity is screened based on an affirmative sentence according to the text vector and each text entity to obtain a target normalized entity in the affirmative sentence. The target normalized entity is used to determine a target entity corresponding to the dialogue text. The embodiment of the application realizes obtaining a text entity according to a text vector, then normalizing the text entity to standardize the text entity, and then judging the normalized entity obtained by the normalization according to an affirmative sentence to screen the normalized entity in the affirmative sentence as the target entity of the dialogue text. Since the normalized entity in the negative sentence is removed, the target entity is more consistent with the text semantic information in the dialogue text, and thus the target entity is more accurate.

[0099] For specific limitations of the dialogue text processing device, refer to the limitations of the dialogue text processing method in the foregoing, which will not be repeated here. Each module, sub-module, and unit in the dialogue text processing device described above can be realized by software, hardware, and a combination thereof in whole or in part. Each module, sub-module, and unit described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form to be called and executed by a processor to perform the operations corresponding to each module.

[0100] It is worth noting that each unit, sub-module and module included in the above device is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific name of each functional unit is only for easy mutual distinction, and does not serve to limit the protection scope of the embodiment of the present application.

[0101] In an embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 4 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external client through a network connection. The computer program is executed by the processor to implement the functions or steps of the server side of the dialogue text processing method.

[0102] In an embodiment, a computer device is provided, which can be a client, and an internal structure diagram thereof can be as shown in Figure 5 The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external server through a network connection. The computer program is executed by the processor to implement the functions or steps of the client side of the dialogue text processing method.

[0103] In an embodiment, a computer device is provided, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0104] performing entity extraction on the text vector of the dialogue text to obtain a plurality of text entities;

[0105] performing normalization processing on each of the text entities to obtain a normalized entity corresponding to each of the text entities;

[0106] performing positive sentence-based screening on each of the normalized entities according to the text vector and each of the text entities to obtain a target normalized entity in a positive sentence.

[0107] According to the target normalized entity, a target entity corresponding to the dialogue text is determined.

[0108] The computer device of the embodiment of the application can realize, when the processor executes the computer program, obtaining multiple text entities by performing entity extraction on the text vector of the dialogue text. Each text entity is normalized to obtain a normalized entity corresponding to each text entity. According to the text vector and each text entity, each normalized entity is screened based on an affirmative sentence to obtain a target normalized entity in the affirmative sentence. According to the target normalized entity, a target entity corresponding to the dialogue text is determined. The embodiment of the application realizes obtaining text entities according to a text vector, then normalizing the text entities to standardize the text entities, and then judging the normalized entities obtained by the normalization according to an affirmative sentence to screen the normalized entities in the affirmative sentence as the target entity of the dialogue text. The scheme of the embodiment of the application improves the accuracy of the target entity in the dialogue text.

[0109] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps are realized:

[0110] performing entity extraction on a text vector of dialogue text to obtain multiple text entities;

[0111] normalizing each text entity to obtain a normalized entity corresponding to each text entity;

[0112] screening each normalized entity based on an affirmative sentence according to the text vector and each text entity to obtain a target normalized entity in the affirmative sentence;

[0113] determining a target entity corresponding to the dialogue text according to the target normalized entity.

[0114] The computer readable storage medium of the embodiment of the application, and the computer program executed by the processor can realize obtaining a plurality of text entities by performing entity extraction on a text vector of a dialogue text. Each text entity is normalized to obtain a normalized entity corresponding to each text entity. Each normalized entity is screened based on an affirmative sentence according to the text vector and each text entity to obtain a target normalized entity in the affirmative sentence. The target entity corresponding to the dialogue text is determined according to the target normalized entity. The embodiment of the application realizes obtaining text entities according to a text vector, and then normalizing the text entities to standardize the text entities. The normalized entities obtained by the normalization are judged according to the affirmative sentence, and the normalized entities in the affirmative sentence are screened as the target entity of the dialogue text. The scheme of the embodiment of the application improves the accuracy of the target entity in the dialogue text.

[0115] It should be noted that the functions or steps that can be realized by the computer readable storage medium or the computer device described above can be referred to the related description of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0116] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0118] The above examples are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of processing dialogue text, characterized by, The method comprises the following steps: entity extraction is performed on a text vector of dialogue text to obtain a plurality of text entities; normalization processing is performed on each of the text entities to obtain a normalized entity corresponding to each of the text entities; based on the text vector and each of the text entities, each of the normalized entities is screened based on a positive sentence to obtain a target normalized entity in the positive sentence; based on the target normalized entity, a target entity corresponding to the dialogue text is determined; the entity extraction on the text vector of dialogue text to obtain a plurality of text entities comprises: performing entity extraction on the text vector by using a pre-trained semantic acquisition model to obtain a target text vector, wherein the target text vector comprises context semantic information of the dialogue text; by processing the target text vector, a plurality of text entities are obtained; the screening of each of the normalized entities based on the positive sentence to obtain the target normalized entity in the positive sentence comprises: inputting the target text vector and each of the text entities into a pre-trained entity screening model to obtain a text entity in the positive sentence; the normalized entity corresponding to the text entity in the positive sentence is taken as the target normalized entity; before the inputting of the target text vector and each of the text entities into the pre-trained entity screening model to obtain the text entity in the positive sentence, the method further comprises: processing a training text vector and a training text entity by using an initial entity screening model to obtain a training entity screening result; based on the training entity screening result, a real label corresponding to the training entity screening result, and an entity correlation loss value, a second loss value is obtained, and in a case where the second loss value meets a training condition, training is completed, the initial entity screening model after the training is taken as the entity screening model, wherein the entity correlation loss value is determined by a co-occurrence frequency of a training normalized entity corresponding to the training text entity and a candidate standard entity, and the candidate standard entity comprises each standard entity in a standard entity library except the training normalized entity.

2. The dialog text processing method according to claim 1, characterized by, the normalization processing of each of the text entities to obtain the normalized entity corresponding to each of the text entities comprises: by using a pre-trained normalization model, each standard entity and each of the text entities are matched to obtain a matching degree of each text entity and each standard entity; for each of the text entities, the standard entity with the highest matching degree is taken as the normalized entity of the text entity.

3. The dialog text processing method according to claim 2, characterized in that, before the matching of each of the standard entities and each of the text entities by using the pre-trained normalization model to obtain the matching degree of each text entity and each standard entity, the method further comprises: by using an initial model, a normalization processing result is obtained by performing normalization processing on each training sample pair, wherein the training sample pair comprises a training text entity, a positive sample of the training text entity, and a negative sample of the training text entity; the normalization processing result is input into a loss function to obtain a first loss value; In a case where the first loss value does not satisfy a preset loss condition, model parameters of the initial model are adjusted and training is continued; In a case where the first loss value satisfies the loss condition, the initial model is taken as a normalization model.

4. The dialog text processing method of claim 1, wherein, After the target normalization entity is determined according to the target normalization entity, the method further comprises: According to the mapping relationship between each standard entity and the reply text, a target reply text corresponding to the target entity is obtained.

5. A dialogue text processing apparatus characterized by comprising: Comprise: A text entity acquisition module is configured to perform entity extraction on a text vector of the dialogue text to obtain a plurality of text entities; A normalization entity acquisition module is configured to perform normalization processing on each of the text entities to obtain a normalization entity corresponding to each of the text entities; A target normalization entity acquisition module is configured to perform positive sentence-based screening on each of the normalization entities according to the text vector and each of the text entities to obtain a target normalization entity in a positive sentence; A target entity determination module is configured to determine a target entity corresponding to the dialogue text according to the target normalization entity; The entity extraction on the text vector of the dialogue text to obtain a plurality of text entities comprises: Performing entity extraction on the text vector by using a pre-trained semantic acquisition model to obtain a target text vector, wherein the target text vector includes context semantic information of the dialogue text; Obtaining a plurality of text entities by processing the target text vector; The positive sentence-based screening on each of the normalization entities according to the text vector and each of the text entities to obtain a target normalization entity in a positive sentence comprises: Inputting the target text vector and each of the text entities into a pre-trained entity screening model to obtain a text entity in a positive sentence; Taking the normalization entity corresponding to the text entity in the positive sentence as the target normalization entity; Before the inputting of the target text vector and each of the text entities into the pre-trained entity screening model to obtain a text entity in a positive sentence, the method further comprises: Processing a training text vector and a training text entity by using an initial entity screening model to obtain a training entity screening result; Obtaining a second loss value according to the training entity screening result, a real label corresponding to the training entity screening result, and an entity correlation loss value, and completing training in a case where the second loss value satisfies a training condition, wherein the entity correlation loss value is determined by a co-occurrence frequency of a training normalization entity corresponding to the training text entity and a candidate standard entity, and the candidate standard entity includes each standard entity in a standard entity library except the training normalization entity.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the dialogue text processing method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor to implement the steps of the dialogue text processing method according to any one of claims 1 to 4.

Images