A communication opportunity judgment method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a communication opportunity judgment method and device, electronic equipment and storage medium, which are applied to a man-machine conversation scene. The method comprises: obtaining a text sequence of a user voice input; the text sequence comprises multiple characters; performing punctuation prediction on the text sequence by using a trained punctuation prediction model to obtain a predicted text labeled with a label; the label comprises a blank label and a punctuation label; according to the label labeled by the last character of the predicted text, if it is judged that the user has completed voice input, a preset conversation content is broadcast; if it is judged that the user has not completed voice input, a waiting state is entered to enable the user to complete voice input. By labeling punctuation on the text sequence to determine whether the current voice input of the user is a complete sentence, it is judged from the perspective of semantic analysis whether the user has completed speaking according to the speaking content, so that the communication opportunity can be more accurately judged.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction, in particular to a communication opportunity judgment method and device, electronic equipment and storage medium. BACKGROUND

[0002] In a human-computer dialogue scenario, a user has a dialogue with a robot, and the robot needs to communicate with the user after the user finishes speaking. Therefore, the robot needs to judge the communication opportunity. If the communication opportunity is judged incorrectly, the robot will interrupt the user's speech. On the one hand, this seriously affects the user's dialogue experience, and on the other hand, it causes the robot to miss the user's speech content, resulting in the robot being unable to correctly understand the user's intention. Therefore, how to accurately judge the communication opportunity of the robot is a technical problem that needs to be solved in the field. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a communication opportunity judgment method and device, electronic equipment and storage medium, to achieve the technical effect of accurately judging the communication opportunity.

[0004] The first aspect of the embodiments of the present application provides a communication opportunity judgment method applied to a human-computer dialogue scenario, the method comprising:

[0005] obtaining a text sequence of a user voice input; the text sequence comprising a plurality of characters;

[0006] using a trained punctuation prediction model to perform punctuation prediction on the text sequence to obtain a predicted text annotated with labels; the labels comprising a blank label and a punctuation label;

[0007] According to the label annotated by the last character of the predicted text, if it is judged that the user has completed voice input, a preset dialogue content is broadcast; if it is judged that the user has not completed voice input, a waiting state is entered to enable the user to complete voice input.

[0008] In the above implementation process, the punctuation prediction model is used to perform punctuation prediction on the text sequence of the user voice input, which can annotate the text sequence with punctuation. According to the label annotated by the last character, it is determined whether the text sequence is a complete sentence, so as to determine whether the user has completed voice input, and thus determine whether the current time is a suitable communication opportunity. In addition, by annotating the text sequence with punctuation to determine whether the current voice input of the user is a complete sentence, it is determined from the semantic analysis point of view whether the user has completed the speech according to the speech content, so as to more accurately determine the communication opportunity.

[0009] Further, the method further comprises:

[0010] performing keyword matching and / or semantic analysis processing on the text sequence;

[0011] if it is determined according to the processing result that the user expresses the conversation pause intention, entering a conversation pause state;

[0012] if it is determined according to the processing result that the user does not express the conversation pause intention, performing the step of punctuating the text sequence.

[0013] In the above implementation process, first, it is determined whether the user expresses the conversation pause intention through keyword matching and / or semantic analysis, if yes, it can be directly determined that the user has not completed the voice input, without the help of punctuation prediction for judgment. In the case where the user does not express the conversation pause intention, punctuation prediction is used to determine whether the user has completed the voice input. By predicting the conversation pause intention and the completion of the voice input in sequence, the accuracy of the communication opportunity prediction can be further improved.

[0014] Further, after entering the waiting state, the method further comprises:

[0015] if the waiting state is maintained for more than a first waiting time, playing a preset conversation content.

[0016] In the above implementation process, by continuing to play the conversation content when the waiting state exceeds the first waiting time, the situation that the model prediction is wrong, leading to misjudgment that the user has completed the voice input, but the user has actually completed the voice input, and the robot is in a waiting state, is avoided.

[0017] Further, if it is determined that the user has completed the voice input, playing the preset conversation content, comprising:

[0018] if it is determined that the user has completed the voice input, playing the preset conversation content after a second waiting time; wherein the first waiting time is greater than the second waiting time.

[0019] In the above implementation process, after determining that the user has completed the voice input, the conversation content will be played after a second waiting time, thereby ensuring the continuity of the conversation with the user.

[0020] Further, the punctuation prediction model comprises a semantic feature extraction layer and a labeling layer; the punctuation prediction of the text sequence by using the trained punctuation prediction model to obtain the predicted text labeled with tags, comprising:

[0021] obtaining a text vector of the text sequence;

[0022] extracting semantic features from the text vector by using the semantic feature extraction layer;

[0023] Input the semantic features into the labeling layer, and obtain predicted text labeled with a label output by the labeling layer.

[0024] In the implementation process, the semantic features of the text sequence are extracted by the semantic feature extraction layer, which can more accurately determine the communication opportunity by judging whether the user has finished speaking according to the content of the speech from the perspective of semantic analysis.

[0025] Further, the semantic feature extraction layer includes a BERT model and a long short-term memory network (LSTM), and the labeling layer is a conditional random field (CRF) model.

[0026] In the implementation process, the BERT model can effectively enrich the feature representation, and the LSTM can consider the information of the previous characters in the semantic calculation of the subsequent characters, thereby better extracting the semantic features.

[0027] Further, the punctuation prediction model is obtained through supervised training, and the sample data includes complete sentence samples and incomplete sentence samples.

[0028] In the implementation process, the punctuation prediction model is supervised trained by using the sample data including the complete sentence samples and the incomplete sentence samples, so that the trained model can accurately predict the punctuation of various text sequences, thereby determining whether the user has completed the voice input.

[0029] The second aspect of the embodiment of the application provides a communication opportunity determination device, which is applied to a man-machine dialogue scene, and the device comprises:

[0030] An acquisition module is configured to acquire a text sequence input by a user voice, wherein the text sequence includes a plurality of characters.

[0031] A prediction module is configured to predict punctuation of the text sequence by using a trained punctuation prediction model, and acquire predicted text labeled with a label, wherein the label includes a blank label and a punctuation label.

[0032] A determination module is configured to determine the label labeled by a last character of the predicted text, and if it is determined that the user has completed the voice input, broadcast preset dialogue content, and if it is determined that the user has not completed the voice input, enter a waiting state to enable the user to complete the voice input.

[0033] The third aspect of the embodiment of the application provides an electronic device, which comprises:

[0034] A processor;

[0035] A memory for storing processor-executable instructions;

[0036] The processor realizes the operation of the method of any one of the first aspect when calling the executable instruction.

[0037] The fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions realize the steps of the method of any one of the first aspect when executed by a processor. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0039] Figure 1 The flowchart of a communication opportunity judgment method provided by the embodiment of the present application is shown in the figure.

[0040] Figure 2 The flowchart of another communication opportunity judgment method provided by the embodiment of the present application is shown in the figure.

[0041] Figure 3 The structure diagram of a punctuation prediction model provided by the embodiment of the present application is shown in the figure.

[0042] Figure 4 The flowchart of another communication opportunity judgment method provided by the embodiment of the present application is shown in the figure.

[0043] Figure 5 The structure diagram of another punctuation prediction model provided by the embodiment of the present application is shown in the figure.

[0044] Figure 6 The structure block diagram of a communication opportunity judgment device provided by the embodiment of the present application is shown in the figure.

[0045] Figure 7 The hardware structure diagram of an electronic device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0046] The technical solutions of the embodiments of the present application will be described below in combination with the drawings in the embodiments of the present application.

[0047] It should be noted that: similar labels and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present application, the terms "first", "second" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0048] In a human-robot conversation scenario, a user has a conversation with a robot, and the robot needs to communicate with the user after the user finishes speaking. Therefore, the robot needs to determine the timing of the communication. If the timing of the communication is determined incorrectly, the robot may interrupt the user, i.e., the robot starts speaking before the user finishes speaking. For example, the user is saying "Yes, now", and the robot interrupts the user or speaks at the same time as the user.

[0049] The occurrence of the interrupting phenomenon seriously affects the user's conversation experience, and on the other hand, the robot cannot correctly understand the user's intention due to missing the content of the user's speech. Therefore, how to accurately determine the communication timing of the robot is a technical problem to be solved in the field.

[0050] To this end, the present application provides a communication timing determination method applied to a human-robot conversation scenario.

[0051] As shown in Figure 1 the method comprises the steps of:

[0052] Step S1: obtaining a text sequence input by a user's voice;

[0053] The text sequence comprises a plurality of characters;

[0054] Step S3: using a trained punctuation prediction model to predict the punctuation of the text sequence, and obtaining a predicted text labeled with a label;

[0055] The label comprises a blank label and a punctuation label;

[0056] According to the label labeled by the last character of the predicted text, step S41 or step S42 is performed.

[0057] Step S41: if it is determined that the user has finished voice input, a preset conversation content is played back;

[0058] Step S42: if it is determined that the user has not finished voice input, a waiting state is entered, so that the user finishes voice input.

[0059] In a human-robot conversation, a user and a robot have a conversation in the form of one question and one answer. Each speech of the user can be a question to the robot or an answer to the robot. One human-robot conversation includes one or more than one speech of the user. That is, in one human-robot conversation, the user will make at least one speech, usually multiple speeches.

[0060] Regarding obtaining a text sequence of a user's speech input. Illustratively, speech data of the user can be obtained, and then converted into a text sequence through an ASR (Automatic Speech Recognition) technique. The text sequence does not carry punctuation.

[0061] Regarding the timing of obtaining the text sequence, in some embodiments, the text sequence of the speech input is obtained whenever a user speech pause is detected. For each text sequence obtained at a user speech pause, it is determined whether the pause is a suitable communication timing, i.e., whether the robot should speak after the pause, by performing steps as shown in Figure 1

[0062] After obtaining the text sequence, punctuation prediction can be performed on the text sequence using a trained punctuation prediction model. After inputting the text sequence into the punctuation prediction model, the punctuation prediction model can output a predicted text with labels. The labels include a blank label and a punctuation label.

[0063] The text sequence includes multiple characters, and after being labeled by the punctuation prediction model, each character in the text sequence is labeled with a label, obtaining a predicted text. If a character is labeled with a blank label, it means that no punctuation is generated after the character. If a character is labeled with a punctuation label, it means that a punctuation corresponding to the punctuation label is generated after the character.

[0064] Illustratively, the punctuation label includes one or more of a comma label, a period label, a question mark label, and an exclamation mark label. Each punctuation label corresponds to a different punctuation.

[0065] Subsequently, it can be determined whether the user has completed the speech input according to the label of the last character of the predicted text.

[0066] If the last character of the predicted text is labeled with a punctuation label, it is determined that the user has completed the speech input. Alternatively, if the last character of the predicted text is labeled with one or more of a period label, a question mark label, and an exclamation mark label, it is determined that the user has completed the speech input. In this way, a preset dialogue content is played. Illustratively, semantic analysis can be performed on the text sequence to analyze the content of the user's speech. For the speech content, a dialogue content corresponding to the speech content is played in advance to respond to the user's speech.

[0067] If the last character of the predicted text is labeled with a blank label, it is determined that the user has not completed the speech input. Alternatively, if the last character of the predicted text is labeled with a comma label, it is determined that the user has not completed the speech input. In this way, a waiting state is entered to allow the user to complete the speech input.

[0068] ​For example, if the text sequence input by the user voice is "Yes, it is like this now", the predicted text "Yes / O this / O like / O ah / O DOU now / O in / O" can be obtained after punctuation prediction by the punctuation prediction model. Among them, " / O" is an empty label, and " / DOU" is a comma label. According to the label of the last character "in" in the predicted text, which is a blank label, it can be determined that the user has not completed voice input. Then enter the waiting state to make the user complete voice input.

[0069] For another example, if the text sequence input by the user voice is "What time to handle", the predicted text "What / O time / O when / O handle / O Wen / WEN" can be obtained after punctuation prediction by the punctuation prediction model. Among them, " / WEN" is a question mark label. According to the label of the last character "Wen" in the predicted text, which is a question mark label in the punctuation label, it can be determined that the user has completed voice input, and the preset dialogue content is played.

[0070] The communication opportunity judgment method provided by the present application can be understood as that, in order to accurately judge the communication opportunity, it is necessary to accurately judge whether the user has completed voice input, or whether the user has completed speech. Therefore, the punctuation prediction model is used to predict the punctuation of the text sequence input by the user voice, so as to label the punctuation of the text sequence. And according to the label of the last character, it is judged whether the text sequence is a complete sentence, so as to judge whether the user has completed voice input, and thus judge whether the current time is a suitable communication opportunity. In addition, by labeling the punctuation of the text sequence to determine whether the current voice input of the user is a complete sentence, it is judged from the semantic analysis point of view whether the user has completed speech according to the speech content, so as to more accurately judge the communication opportunity.

[0071] In some embodiments, the communication opportunity judgment method provided by the present application further includes the steps as shown in Figure 2

[0072] Step S1: obtaining a text sequence input by a user voice;

[0073] Step S2: processing the text sequence by keyword matching and / or semantic analysis;

[0074] Step S31: if it is judged according to the processing result that the user shows a dialogue pause intention, entering a dialogue pause state;

[0075] Step S32: if it is judged according to the processing result that the user does not show a dialogue pause intention, using a trained punctuation prediction model to predict the punctuation of the text sequence, and obtaining a predicted text labeled with a label;

[0076] ​Step S41 or step S42 is performed according to the label marked by the last character of the predicted text.

[0077] Step S41: If it is judged that the user has completed voice input, a preset dialogue content is played back.

[0078] Step S42: If it is judged that the user has not completed voice input, a waiting state is entered to enable the user to complete voice input.

[0079] In the embodiment, the implementation processes of steps S1, S32, S41 and S42 can refer to the implementation processes of the corresponding steps in the above embodiments, which will not be described herein again.

[0080] After obtaining the text sequence of the user's voice input, the text sequence is first processed by keyword matching and / or semantic analysis. And according to the processing result, it is judged whether the user shows the dialogue pause intention.

[0081] In the keyword matching processing, the keyword library can be established in advance according to the historical human-computer dialogue data to collect the keywords for expressing the dialogue pause intention. For example, the keywords can include but are not limited to "I want to think" and "wait a moment". After obtaining the text sequence, the text sequence can be matched with the keyword library. If the matching is successful, it is determined that the user shows the dialogue pause intention; if the matching is not successful, it is determined that the user does not show the dialogue pause intention.

[0082] In the semantic analysis processing, the text sequence can be analyzed by using the trained intention recognition model to predict the intention type shown by the user. If the output result of the intention recognition model is the dialogue pause intention, it is determined that the user shows the dialogue pause intention; otherwise, it is determined that the user does not show the dialogue pause intention.

[0083] According to the above processing result, in the case where it is judged that the user shows the dialogue pause intention, the dialogue pause state is entered.

[0084] For example, in the dialogue pause state, the robot waits for the user to continue voice input. In response to the current text sequence of the user voice input in the pause state, semantic analysis is performed on the current text sequence. If it is determined according to the semantic analysis result that the user shows a dialogue continuation intention, the text sequence of the last voice input of the user, or in other words, the text sequence of the last voice input of the user before entering the pause state, is taken as a complete text sequence, and the preset dialogue content is played according to the complete text sequence. If it is determined according to the semantic analysis result that the user does not show a dialogue continuation intention, the current text sequence and the text sequence of the last voice input are combined as a complete text sequence. And according to the complete text sequence, the preset dialogue content is played. In addition, if no user speech is received after a third waiting time, a prompt content is played to prompt the user to continue to complete the speech or to ask the user whether the speech is completed.

[0085] For example, in a telephone sales scenario, the salesperson is an intelligent customer service, and if a transaction intention is reached, the intelligent customer service asks the user for the address of the goods delivery. If the user voice inputs "A city, I want to think", according to the above steps, it can be known that the user shows a dialogue pause intention at this time and needs time to think, and the intelligent customer service (robot) enters a pause state. In the pause state, if the user continues to voice input "B district C road XX number", semantic analysis is performed on the text sequence "B district C road XX number" to determine that the user does not show a dialogue continuation intention, and the text sequence of the last voice input "A city, I want to think" and the current text sequence "B district C road XX number" are combined as a complete text sequence. The delivery address information is obtained therefrom, and the preset dialogue content is played.

[0086] For another example, if the user has completed voice input, but may misjudge that the user shows a dialogue pause intention in the above keyword matching and / or voice analysis processing, resulting in entering a dialogue pause state, and the robot waits for the user to speak. At this time, if the user continues to speak "hello" or "are you there?", semantic analysis is performed on the current speech to determine that the speech shows a dialogue continuation intention, and the text sequence of the last voice input of the user is taken as a complete text sequence, and the preset dialogue content is played according to the complete text sequence. Alternatively, if the user does not continue to speak, the robot can automatically play prompt information such as "please ask whether the speech is completed" and the like.

[0087] According to the processing result, in the case where it is determined that the user does not show a dialogue pause intention, punctuation prediction is further performed on the text sequence by a punctuation prediction model to determine whether the user completes the speech.

[0088] It can be understood that in the process of human-computer conversation, the user sometimes needs to think for a while, so as to pause the conversation. At this time, the robot needs to recognize the conversation pause intention expressed by the user. In this embodiment, the user's conversation pause intention is predicted through keyword prediction and / or semantic analysis of the text sequence. In addition, when judging whether the user has completed the voice input, on the one hand, it can be judged by punctuation prediction; on the other hand, if the user directly expresses the intention to pause the conversation, it can be directly determined that the user has not completed the voice input, so as to enter the conversation pause state. Therefore, in this embodiment, first, whether the user expresses the conversation pause intention is determined through keyword matching and / or semantic analysis, if yes, it can be directly determined that the user has not completed the voice input, without the help of punctuation prediction for judgment. In the case where the user does not express the conversation pause intention, punctuation prediction is used to judge whether the user has completed the voice input. By predicting the user's conversation pause intention and voice input completion in sequence, the accuracy of the communication opportunity prediction can be further improved.

[0089] In some embodiments, after entering the waiting state in step S42, the above method further comprises step S5: if the waiting state maintenance time is greater than the first waiting time, the preset conversation content is broadcasted.

[0090] When the text sequence predicted by the punctuation prediction model is not a complete sentence, and the user has not completed the voice input, the waiting state is entered. When the maintenance time of the waiting state exceeds the first waiting time, for example, 1 second, it is considered that the user has completed the voice input, the text sequence of the last voice input of the user, or in other words, the text sequence of the last voice input of the user before entering the waiting state, is taken as a complete text sequence, and the preset conversation content is broadcasted according to the complete text sequence.

[0091] It can be seen that in this embodiment, by continuing to play the conversation content when the waiting state exceeds the first waiting time, the situation that the model prediction is wrong, the user actually completes the voice input, but the robot is always in the waiting state is avoided.

[0092] Further, in some embodiments, if it is judged that the user has completed the voice input, the preset conversation content is immediately broadcasted.

[0093] In other embodiments, if it is judged that the user has completed the voice input, the preset conversation content is broadcasted after a second waiting time. The first waiting time is greater than the second waiting time. As an example, the first waiting time is 1 second, and the second waiting time is 0.5 second.

[0094] Optionally, if the user's voice input is detected during the second waiting time, the waiting is interrupted, and the execution of step S4 is returned. Figure 1 or Figure 2The steps shown in the embodiments.

[0095] In this embodiment, after judging that the user has completed the voice input, the dialogue content is played back after a second waiting time, thereby ensuring the continuity of the dialogue with the user.

[0096] Regarding the punctuation prediction model, in some embodiments, as shown in Figure 3 The punctuation prediction model 300 includes a semantic feature extraction layer 310 and a labeling layer 320. The above-mentioned step S3 and S32 utilize the acquisition process of the predicted text, including the steps as shown in Figure 4

[0097] Step 410: acquiring a text vector of the text sequence;

[0098] Step 420: extracting semantic features from the text vector by using the semantic feature extraction layer;

[0099] Step 430: inputting the semantic features into the labeling layer to obtain the predicted text labeled with tags output by the labeling layer.

[0100] After acquiring the text vector of the text sequence, the text vector can be used as the input feature of the semantic extraction layer. The semantic extraction layer can process the text vector, establish the relationship between each character, and extract the semantic features of the text sequence from the text vector. The semantic features represent the semantic correlation of the context in the text sequence.

[0101] Subsequently, the extracted semantic features are input into the labeling layer, and the labeling layer can output the predicted text labeled with tags according to the semantic features.

[0102] In this embodiment, the semantic features of the text sequence are extracted by the semantic feature extraction layer. From the perspective of semantic analysis, it is judged whether the user has completed the speech according to the speech content, so as to more accurately judge the communication opportunity.

[0103] In some embodiments, as shown in Figure 5 The semantic feature layer 310 includes a BERT (Bidirectional Encoder Representations from Transformer) model 311 and a long short-term memory LSTM (Long short-term memory) network 312, and the labeling layer is a conditional random field CRF model 320.

[0104] ​Wherein, the text vector obtained from the text sequence is in the form of an input vector of the BERT model 311, including word embedding vectors, segmentation embedding vectors, and position encoding tensors. The dimensions of the above three vectors are preset values, such as 128 dimensions or 256 dimensions. Then, the above three vectors are added to obtain the text vector, which is input into the BERT model 520.

[0105] The BERT model 311 is used as a sub-layer in the semantic feature layer 310, which can effectively enrich the feature representation. The BERT model 311 includes multiple layers, each of which performs self-attention calculation, linear projection calculation, and feed-forward calculation.

[0106] After the text vector passes through the BERT model 311, the output feature can be input into the LSTM network 312 for updating. The LSTM network is a sequential calculation process. The calculation at each time considers the result of the previous time. Alternatively, the calculation result of the previous time is transmitted to the calculation of the next time. Therefore, the information of the previous character is transmitted to the calculation of the subsequent character. The semantic information of the previous character affects the semantic information of the subsequent character. After the LSTM network 312 processes, the semantic feature is obtained. The semantic feature is input into the CRF model 320, and the CRF model 320 can output the predicted text with labels according to the semantic feature. Thus, the entire punctuation prediction process is completed.

[0107] In the training process, the above punctuation prediction model is obtained through supervised training. The sample data used for training includes complete sentence samples and incomplete sentence samples. Before training, the sample data is first labeled with symbols, and the sample data is preprocessed into sample data carrying real labels according to the labeled symbols.

[0108] For example, for the complete sentence sample "wait a moment and I am eating", and the incomplete sentence sample "where are you from and how", first, the symbols are marked to obtain "wait a moment, and I am eating.", and "where are you from? And how". Then, according to the marked symbols, the sample data is marked with real labels to obtain "wait / O a / O moment / O and / O I / O am / O eating / JU", and "where / O are / O you / O from / WEN and / O how / O". Then, the sample data carrying the real labels is used to train the punctuation prediction model. Optionally, when the punctuation prediction model is trained for a preset number of times, the training can be stopped to obtain the trained punctuation prediction model. In this embodiment, when the punctuation prediction model is trained for 10 rounds, the accuracy on the test set reaches 98%.

[0109] The application provides a communication opportunity judgment method. The punctuation prediction model is used to predict the punctuation of the text sequence input by the user voice, so that the text sequence is marked with punctuation. The last character of the text sequence is used to determine whether the text sequence is a complete sentence, so as to determine whether the user has completed the voice input, and thus determine whether the current time is a suitable communication opportunity. In addition, the text sequence is marked with punctuation to determine whether the current voice input of the user is a complete sentence.

[0110] The punctuation prediction model is used to predict the punctuation of the text sequence input by the user voice, so that the text sequence is marked with punctuation. The last character of the text sequence is used to determine whether the text sequence is a complete sentence, so as to determine whether the user has completed the voice input, and thus determine whether the current time is a suitable communication opportunity. In addition, the text sequence is marked with punctuation to determine whether the current voice input of the user is a complete sentence.

[0111] The punctuation prediction model is used to predict the punctuation of the text sequence input by the user voice, so that the text sequence is marked with punctuation. The last character of the text sequence is used to determine whether the text sequence is a complete sentence, so as to determine whether the user has completed the voice input, and thus determine whether the current time is a suitable communication opportunity. In addition, the text sequence is marked with punctuation to determine whether the current voice input of the user is a complete sentence.

[0112] Based on the communication opportunity judgment method provided in any of the above embodiments, the application further provides a communication opportunity judgment device applied to a man-machine dialogue scene. As shown in Figure 6 The communication opportunity judgment device 600 comprises:

[0113] The acquisition module 610 is configured to acquire a text sequence input by a user voice; the text sequence comprises a plurality of characters;

[0114] The prediction module 620 is configured to use a trained punctuation prediction model to predict the punctuation of the text sequence, and acquire a predicted text marked with labels; the labels comprise a blank label and a punctuation label;

[0115] The judgment module 630 is configured to determine, according to the label marked by the last character of the predicted text, whether the user has completed the voice input. If the user has completed the voice input, a preset dialogue content is broadcasted. If the user has not completed the voice input, a waiting state is entered, so that the user completes the voice input.

[0116] In some embodiments, the communication opportunity judgment device 600 further comprises:

[0117] a processing module configured to perform keyword matching and / or semantic analysis on the text sequence;

[0118] a pausing module configured to, if it is determined, according to the processing result, that the user exhibits a conversation pausing intention, enter a conversation pausing state; and if it is determined, according to the processing result, that the user does not exhibit the conversation pausing intention, perform the step of predicting punctuation for the text sequence.

[0119] a pausing module configured to, if it is determined, according to the processing result, that the user exhibits a conversation pausing intention, enter a conversation pausing state; and if it is determined, according to the processing result, that the user does not exhibit the conversation pausing intention, perform the step of predicting punctuation for the text sequence.

[0120] In some embodiments, the determining module 630 is specifically configured to:

[0121] if the waiting state maintaining time is greater than a first waiting time, play a preset conversation content.

[0122] In some embodiments, the determining module 630 is specifically configured to:

[0123] if it is determined that the user has completed voice input, after a second waiting time, play a preset conversation content; wherein the first waiting time is greater than the second waiting time.

[0124] In some embodiments, the punctuation prediction model includes a semantic feature extraction layer and a labeling layer; the prediction module 620 is specifically configured to:

[0125] obtain a text vector of the text sequence;

[0126] extract semantic features from the text vector by using the semantic feature extraction layer;

[0127] input the semantic features into the labeling layer, and obtain a predicted text with labels output by the labeling layer.

[0128] In some embodiments, the semantic feature extraction layer includes a BERT model and a long short-term memory network (LSTM); and the labeling layer is a conditional random field (CRF) model.

[0129] In some embodiments, the punctuation prediction model is obtained through supervised training; and the sample data includes complete sentence samples and incomplete sentence samples.

[0130] The implementation processes of the functions and roles of the various modules in the above device are specifically described in the implementation processes of the corresponding steps in the above method, which will not be repeated here.

[0131] Based on the communication opportunity determination method described in any of the above embodiments, the present application further provides an electronic device as shown in Figure 7 the structural schematic diagram of an electronic device. As shown in Figure 7At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the communication opportunity determination method described in any of the embodiments.

[0132] The application further provides a computer storage medium, the storage medium stores a computer program, and the computer program is executed by a processor to implement the communication opportunity determination method described in any of the embodiments.

[0133] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only schematic, for example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the devices, methods and computer program products according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0134] In addition, the functional modules in each of the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0135] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts of the prior art that make contributions or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0136] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0137] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0138] It should be noted that, in this document, the terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

Claims

1. A communication timing determination method, characterized by, Applied to a human-computer dialogue scene, the method comprises: obtaining a text sequence of user voice input, and performing semantic analysis processing on the text sequence; the text sequence comprises a plurality of characters; if it is judged according to the processing result that the user shows a dialogue pause intention, entering a dialogue pause state, responding to a current text sequence of user voice input in the pause state, performing semantic analysis on the current text sequence; if it is determined according to the semantic analysis result that the user shows a dialogue continuation intention, taking the text sequence of the last voice input of the user before entering the pause state as a complete text sequence; if it is determined according to the semantic analysis result that the user does not show a dialogue continuation intention, combining the current text sequence and the text sequence of the last voice input of the user before entering the pause state as a complete text sequence; and playing a preset dialogue content according to the complete text sequence; if it is judged according to the processing result that the user does not show the dialogue pause intention, using a trained punctuation prediction model to perform punctuation prediction on the text sequence, and obtaining a predicted text annotated with labels; the labels comprise a blank label and a punctuation label; the punctuation prediction model comprises a semantic feature extraction layer for extracting semantic features of the text sequence; according to the label annotated by the last character of the predicted text, if it is judged that the user has completed voice input, playing a preset dialogue content; if it is judged that the user has not completed voice input, entering a waiting state to enable the user to complete voice input.

2. The method of claim 1, wherein, after entering the waiting state, the method further comprises: if the waiting state is maintained for a time greater than a first waiting time, playing a preset dialogue content.

3. The method of claim 2, wherein, if it is judged that the user has completed voice input, playing a preset dialogue content, comprising: if it is judged that the user has completed voice input, playing a preset dialogue content after a second waiting time; wherein the first waiting time is greater than the second waiting time.

4. The method of claim 1, wherein, the punctuation prediction model comprises a semantic feature extraction layer and a labeling layer; using a trained punctuation prediction model to perform punctuation prediction on the text sequence and obtaining a predicted text annotated with labels, comprising: obtaining a text vector of the text sequence; using the semantic feature extraction layer to extract semantic features from the text vector; inputting the semantic features into the labeling layer to obtain a predicted text annotated with labels output by the labeling layer.

5. The method of claim 4, wherein, the semantic feature extraction layer comprises a BERT model and a long short-term memory network LSTM; and the labeling layer is a conditional random field CRF model.

6. The method according to any of claims 4-5, characterized by, the punctuation prediction model is obtained through supervised training; sample data comprises complete sentence samples and incomplete sentence samples.

7. A communication timing determination apparatus characterized by comprising: applied to a human-computer dialogue scene, the device comprises: an acquisition module for acquiring a text sequence of user voice input; the text sequence comprises a plurality of characters; a processing module for performing semantic analysis processing on the text sequence; The suspension module is configured to: if it is determined according to the processing result that the user expresses the conversation suspension intention, enter a conversation suspension state, and perform semantic analysis on a current text sequence input by the user in the conversation suspension state; if it is determined according to the semantic analysis result that the user expresses the conversation continuation intention, take a text sequence input by the user last time before entering the conversation suspension state as a complete text sequence; if it is determined according to the semantic analysis result that the user does not express the conversation continuation intention, combine the current text sequence and the text sequence input by the user last time before entering the conversation suspension state as the complete text sequence; and play preset conversation content according to the complete text sequence; The prediction module is configured to: if it is determined according to the processing result that the user does not express the conversation suspension intention, perform punctuation prediction on the text sequence by using a trained punctuation prediction model to obtain a predicted text with labels; the labels include a blank label and a punctuation label; and the punctuation prediction model includes a semantic feature extraction layer configured to extract semantic features of the text sequence. The judgment module is configured to: if it is determined according to a label of a last character of the predicted text that the user has completed voice input, play preset conversation content; and if it is determined that the user has not completed voice input, enter a waiting state to enable the user to complete voice input.

8. An electronic device, comprising: The electronic device comprises: a processor; a memory for storing processor-executable instructions; wherein the processor, when invoking the executable instructions, implements the operations of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program product, comprising a computer readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of any one of claims 1-6.

Images