Interaction method and device, electronic equipment, storage medium and program product

Abstract

The invention provides an interaction method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: carrying out the semantic analysis of a to-be-processed problem, and generating a reply text and a target action of the to-be-processed problem; determining a first text matched with the target action from the reply text according to the reply text and the target action; performing time alignment on the execution starting time of the target action and the playing starting time of the first text; the reply text is played, and in the playing process of the reply text, according to the execution starting time of the target action and the playing starting time of the first text after time alignment, the target action is synchronously executed when the first text is played, so that synchronization of the action and the reply text can be realized; and the interaction performance is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an interaction method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of artificial intelligence, a better human-computer interaction experience can be achieved based on artificial intelligence. Users can interact with robots or virtual digital people through voice and actions. However, in related technologies, it is difficult to ensure the synchronization of action execution and dialogue content during human-computer interaction, which reduces the interactive experience. Summary of the invention

[0003] The present disclosure provides an interaction method, device, electronic device, storage medium and program product.

[0004] In a first aspect, the present disclosure provides an interaction method, the interaction method comprising:

[0005] Perform semantic analysis on the problem to be processed, and generate a reply text and a target action for the problem to be processed;

[0006] According to the reply text and the target action, determining a first text matching the target action from the reply text;

[0007] Aligning the execution start time of the target action with the playback start time of the first text;

[0008] The reply text is played, and during the playing of the reply text, according to the execution start time of the target action after time alignment and the playing start time of the first text, when the first text is played, the target action is synchronously executed.

[0009] In a second aspect, the present disclosure provides an interactive device, the interactive device comprising:

[0010] A generation module, used for performing semantic analysis on the problem to be processed, and generating a reply text and a target action for the problem to be processed;

[0011] A determination module, configured to determine, based on the reply text and the target action, a first text matching the target action from the reply text;

[0012] A time processing module, used for aligning the execution start time of the target action with the playback start time of the first text;

[0013] The output module is used to play the reply text, and during the playback of the reply text, according to the execution start time of the target action after time alignment and the playback start time of the first text, when playing to the first text, the target action is synchronously executed.

[0014] In a third aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can perform the above-mentioned interaction method.

[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned interaction method when executed by a processor.

[0016] In a fifth aspect, the present disclosure provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned interaction method.

[0017] The interactive method provided by the embodiment of the present disclosure performs semantic analysis on the problem to be processed, generates a reply text and a target action for the problem to be processed, and determines a first text matching the target action from the reply text. It can determine the semantic point in the reply text that matches the target action, and align the execution start time of the target action with the playback start time of the first text. Therefore, during the playback of the reply text, the target action is synchronously executed when the first text is played according to the execution start time of the target action after time alignment and the playback start time of the first text. In this way, through semantic point matching and time alignment, real-time synchronization of the action and the reply text is achieved, the interactive performance is improved, and the user can perform human-computer interaction more naturally and smoothly, thereby enhancing the interactive experience.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 An application scenario diagram of the interactive method and device provided in the embodiments of the present disclosure;

[0021] Figure 2 A flowchart of an interactive method provided by an embodiment of the present disclosure;

[0022] Figure 3 A flowchart of another interactive method provided by an embodiment of the present disclosure;

[0023] Figure 4 A block diagram of an interactive device provided in an embodiment of the present disclosure;

[0024] Figure 5 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0026] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0027] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0028] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0030] In the technical solution disclosed in this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution complies with relevant national laws and regulations (for example, the "Information Security Technology Personal Information Security Specification", etc.). For example: corresponding prescribed measures are taken for personal information access control; prescribed restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; when using personal information, clear identity orientation is eliminated to avoid precise positioning of specific individuals.

[0031] Human-computer interaction is achieved based on artificial intelligence. Users can interact with robots or virtual digital people through voice and actions. However, in related technologies, during human-computer interaction, there may be a problem of mismatch between conversation content and actions, and it is difficult to synchronize action execution and conversation content, which reduces the interactive experience.

[0032] The interactive method provided in the embodiment of the present disclosure generates a reply text and a target action for a question to be processed, and determines a first text matching the target action from the reply text, and aligns the execution start time of the target action with the playback start time of the first text. The relevant parameters of the action execution can be dynamically adjusted to be able to adapt to the reply text, so that in the process of playing the reply text, when the first text is played, the target action can be executed synchronously, thereby realizing real-time synchronization of the action and the dialogue content, improving the human-computer interaction performance, and enabling users to interact with the robot more naturally and smoothly, thereby enhancing the interactive experience.

[0033] Figure 1 The application scenario diagram of the interaction method and device provided by the embodiments of the present disclosure is schematically shown.

[0034] like Figure 1 As shown, the application scenario of the embodiment of the present disclosure may include a terminal device 101, a network 103 and a server 102. The network 103 is used to provide a medium for a communication link between the terminal device 101 and the server 102. The network 103 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0035] The user can use the terminal device 101 to interact with the server 102 through the network 103 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0036] The terminal device 101 includes but is not limited to user equipment (UE), mobile devices, terminals, cordless phones, personal digital assistants (PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, intelligent robots, etc.

[0037] The server 102 may be a server that provides various services, such as a background management server that provides support for websites browsed by users using the terminal device 101 (for example only). The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device 101. For another example, the server 102 may understand and answer questions received from users, and return the answer results to the terminal device 101.

[0038] It should be noted that the interactive method and apparatus provided in the embodiments of the present disclosure can be executed by the server 102. Accordingly, the interactive method and apparatus provided in the embodiments of the present disclosure can be arranged in the server 102. The interactive method and apparatus provided in the embodiments of the present disclosure can also be executed by a server or a server cluster that is different from the server 102 and can communicate with the terminal device 101 and / or the server 102. Accordingly, the interactive method and apparatus provided in the embodiments of the present disclosure can also be arranged in a server or a server cluster that is different from the server 102 and can communicate with the terminal device 101 and / or the server 102. Of course, the interactive method and apparatus provided in the embodiments of the present disclosure can also be executed by the terminal device 101, which is not limited in the embodiments of the present disclosure.

[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0040] Figure 2 A flowchart of an interactive method provided by an embodiment of the present disclosure. Figure 2 , the method comprising:

[0041] S201: Perform semantic analysis on the problem to be processed, and generate a reply text and a target action for the problem to be processed.

[0042] The disclosed embodiments may be applicable to any human-machine interaction scenario, for example, user-robot interaction, user-virtual robot interaction in a terminal device, etc.

[0043] Taking the robot as an example, the user can trigger the human-computer interaction, and the user starts a conversation with the robot. In each round of conversation, the user can input the pending questions through text or voice, and then the robot can process the pending questions after receiving them to generate reply text and target actions for the pending questions, and then output them to complete the conversation with the user. Alternatively, the robot can send the pending questions to the server, and the server will generate reply text and target actions and return them to the robot, which will then output them to complete the conversation with the user. This is not limited in the embodiments of the present disclosure.

[0044] For step S201, the present disclosure provides a specific implementation method, which performs semantic analysis on the problem to be processed, generates a reply text and a target action for the problem to be processed, including:

[0045] Based on the model, the problem to be processed and the actions in the preset action library are used as input, a semantic analysis is performed on the problem to be processed, and a reply text for the problem to be processed is generated; according to the actions and reply texts in the preset action library, a target action that matches the reply text is determined from each action.

[0046] Among them, in the embodiment of the present disclosure, an action library can be pre-set, wherein the action library can include the names of multiple actions and the preset third durations for executing the actions. For example, the action library includes the action of nodding in affirmation, and the third duration for executing the nodding in affirmation is 2s; the action is shaking the head in negation, and the third duration for executing the shaking the head in negation is 1s; the action includes jumping with excitement, and the third duration for executing the jumping with excitement is 3s, etc. The actions and execution durations in the action library can be set according to needs and experience, and are not limited in the embodiment of the present disclosure.

[0047] In the disclosed embodiment, it can be implemented based on a model, and the result generation prompt is input into the model. The result generation prompt includes at least the name of the problem to be processed and each action in the preset action library, and the generated prompt is used to indicate the answer to the problem to be processed according to each action and determine the matching action, and then the intention recognition and semantic analysis and understanding of the problem to be processed are performed based on the model, and the reply text and target action of the problem to be processed are output in combination with the name of each action.

[0048] In addition, in a possible embodiment, in the embodiment of the present disclosure, when generating a target action, the target action can also be determined from various actions based on the model, according to the reply text, the type of question to be processed, etc. For example, different question types can match corresponding actions, and then combined with the reply text, the target action can be determined from the matched corresponding actions. The specific implementation method is not limited in the embodiment of the present disclosure.

[0049] For example, the question to be processed is: "Can you sing?" The model outputs the data result result_1 in the JSON structure. Result_1 includes the target action and the reply text. If the output result is:

[0050]

[0051] That is, the reply text generated by the model is "I am a conversational robot, and I can't sing yet.", and the target action is "shake head in denial".

[0052] Furthermore, in order to improve the accuracy and effectiveness of the results, in the embodiments of the present disclosure, after the model outputs the results, data verification can be performed. After the verification is passed, the next step can be continued. If the verification fails, the model can be required to generate again. The data verification can, for example, include determining whether the structure or format of the output results is correct, whether the values ​​of the fields corresponding to the reply text and the target action are empty, etc.

[0053] In this way, in the embodiment of the present disclosure, a deep understanding and analysis of the problem to be processed is achieved based on the model, which improves the accuracy of understanding the user's intention, thereby improving the accuracy of reply and action generation.

[0054] S202: According to the reply text and the target action, determine a first text matching the target action from the reply text.

[0055] With respect to step S202, a possible embodiment of the present disclosure includes: based on the model, performing grammatical analysis on the reply text, and determining candidate texts that conform to the target grammatical structure from the reply text; and determining a first text that matches the target action based on the candidate texts and the target action.

[0056] For example, the alignment semantic point prompt can be input to the model, wherein the alignment semantic point prompt includes the problem to be processed, the reply text and the target action, and is used to instruct the model to give the first text corresponding to the target action in the reply result, and then the model can perform grammatical analysis and matching according to the requirements of the alignment semantic point prompt, and output the first text. Of course, the alignment semantic point prompt can also include other requirements, such as output format requirements, output fields, etc., which are not limited in the specific embodiments of the present disclosure.

[0057] Among them, the target grammatical structure can be, for example, a subject-verb structure, etc. Since the target action represents a certain action, the subject-verb structure includes a verb and an object, which usually describes what action the executor of the action performs on a certain object. Therefore, the text of the subject-verb structure can usually correspond to a certain action. Of course, it can also be other grammatical structures, which is not limited in the embodiments of the present disclosure.

[0058] For example, input the question to be processed and result_1 into the model, and require the model to give the first text corresponding to action_type in the answer of result_1, and output the json structure data result_2. The output result is:

[0059] result_2 = {

[0060] "action_word":"Can't sing"

[0061] }.

[0062] That is, the first text determined from the reply text to match the target action is "can't sing".

[0063] In this way, in the embodiment of the present disclosure, the reply text and the target action can be semantically aligned to determine the first text corresponding to the target action, so as to achieve synchronization between action execution and text playback.

[0064] S203: Align the execution start time of the target action with the playback start time of the first text.

[0065] S204: Play the reply text, and during the playback of the reply text, according to the execution start time of the time-aligned target action and the playback start time of the first text, when the first text is played, synchronously execute the target action.

[0066] In one possible embodiment, in the disclosed embodiment, the reply text can be played by voice according to preset playback speed parameters, timbre parameters, etc. Since the target action and the first text are time-aligned, when the first text is played, taking a robot as an example, the robot can be synchronously controlled to perform the target action. For another example, taking a virtual digital human as an example, the animation of the target action of the virtual digital human can be synchronously rendered. In this way, when the user hears the reply text played by voice, he can see the target action of the robot at the same time when the first text is played.

[0067] In one possible embodiment, the reply text may be played in the form of text output. For example, it may be output word by word, and when the first word of the first text is output, the target action is simultaneously started. In another possible embodiment, the reply text may be played in the form of combined voice and text output, with the voice and text output synchronized, and the output of the first text is synchronized with the execution of the target action.

[0068] In the disclosed embodiment, a reply text and a target action for the problem to be processed are generated, a first text matching the target action is determined from the reply text, and the execution start time of the target action is aligned with the playback start time of the first text. Then, during the playback of the reply text, when the first text is played, the target action can be synchronously executed. In this way, by determining the first text matching the target action and dynamically adjusting the time alignment between the first text and the target action, real-time synchronization of actions and dialogue content is achieved, thereby improving interaction performance and enhancing interaction experience.

[0069] The interactive method according to the embodiment of the present disclosure is described below.

[0070] As mentioned above, for the above step S203, the present disclosure further provides a specific implementation method for aligning the execution start time of the target action with the playback start time of the first text, including:

[0071] 1) Determine from the reply text a second text that is located before the first text.

[0072] For example, the reply text is: "I am a conversational robot, and I can't sing yet.", and the first text is: "can't sing", then the second text before the first text in the reply text is: "I am a conversational robot, and I can't sing yet".

[0073] 2) Determine a first duration for playing the second text according to the playback speech speed parameter.

[0074] 3) Based on the first duration and the target start time, the execution start time of the target action is time-aligned with the playback start time of the first text, wherein the target start time indicates the start time of playing the reply text.

[0075] In the disclosed embodiment, based on the target start time, the reply text starts to be played, the second text is played first, and after reaching the first duration, the first text is played. Therefore, according to the first duration and the target start time, the playback start time of the first text can be determined. In order to align the execution start time of the target action with the playback start time of the first text, the execution start time of the target action can also be determined in the same way according to the first duration and the target start time.

[0076] Furthermore, in an embodiment of the present disclosure, in order to further improve the interactive performance, the duration of executing the target action can also be dynamically adjusted. Specifically, the present disclosure provides a possible implementation method, including: determining the second duration of playing the first text according to the playback speech rate parameter; obtaining a preset third duration corresponding to the execution of the target action; and determining the target duration of executing the target action according to the second duration and the third duration.

[0077] Among them, in the embodiment of the present disclosure, the third duration of the target action can be set in advance and saved in the preset action library. Since the third duration is pre-fixed, it may not be compatible with the real-time synchronization of the matching first text. For example, the first text may be finished playing, but the target action has not been executed, which reduces the interaction effect and also affects the text playback after the first text in the reply text, causing the target action to be incompatible with the text after the first text. Therefore, in the embodiment of the present disclosure, the target duration of the target action can also be dynamically adjusted according to the second duration of the first text playback.

[0078] In a possible embodiment, a target duration for executing the target action is determined based on the second duration and the third duration, including: judging whether the third duration needs to be adjusted based on the second duration and the third duration; if the third duration needs to be adjusted, adjusting the target duration for executing the target action from the third duration to a value less than or equal to the second duration; if the third duration does not need to be adjusted, determining the target duration for executing the target action to be the third duration.

[0079] For example, when the third duration is greater than the second duration, it indicates that the voice playback of the first text has ended but the target action has not yet been completed. Therefore, in this case, it is determined that the third duration needs to be adjusted, and the target duration is determined to be the second duration, or the target duration is determined to be a value less than the second duration. To ensure the effect of the action execution, the value less than the second duration should not be much different from the second duration. In this way, the execution speed of the target action can be improved, such as by adjusting the rotation speed of the robot's mechanical action, or by adjusting the rendering speed of the animation action of the virtual digital human.

[0080] Of course, other conditions for determining whether the third duration needs to be adjusted may be set as required, which is not limited in the embodiments of the present disclosure.

[0081] Furthermore, in the embodiment of the present disclosure, based on the aligned execution start time and playback start time, and the adjusted target duration corresponding to the target action, the synchronous output of the reply text and the target action can be achieved. Specifically, with respect to the above step S204, the reply text is played, and during the playback of the reply text, according to the time-aligned execution start time of the target action and the playback start time of the first text, when the first text is played, the target action is synchronously executed, including:

[0082] Based on the target start time, the reply text is played, and during the playback of the reply text, when the first duration is reached, the control is based on the target duration to execute the target action, so that when the first duration is reached, the first text is played and the target action is executed synchronously.

[0083] For example, a reply text is played based on a voice module, a target action is executed based on an action execution module, the action execution module corresponding to the target action is determined, and the target duration and the first duration are passed as parameters to the action execution module. After waiting for the first duration, the target action is executed. This can be achieved based on a timer of the first duration, and the specific method is not limited. In synchronization with the action execution module, the reply text is passed to the voice module for playback. The voice module can play the reply text based on the playback speed parameter, and when the first duration is reached, it can play to the first text. At this time, the target action can be executed synchronously. In this way, the user can see the execution of the target action while hearing the first text, and the target action can be completed before the first text is finished playing, without affecting the text playback after the first text, thereby improving the interactive performance and making the interaction more natural and smooth.

[0084] In addition, in the embodiments of the present disclosure, the number of target actions generated for the problem to be processed is not limited, and can be one or more. When there are multiple target actions, based on the embodiments of the above-mentioned interaction method, similarly, the first text matching each target action can be determined from the reply text, and the execution start time of each target action can be aligned with the playback start time of the corresponding first text. Therefore, during the playback of the reply text, when each first text is played, the corresponding matching target action can be executed synchronously.

[0085] In this way, in the embodiment of the present disclosure, the target action and the matching first text can be dynamically aligned in time to achieve real-time synchronization of the action and the dialogue text content, thereby improving the human-computer interaction performance.

[0086] The following uses a specific application scenario to illustrate the interaction method in the embodiment of the present disclosure. Figure 3 As shown, it is a flow chart of an interactive method in an embodiment of the present disclosure. Figure 3 As shown, including:

[0087] S301: Generate a reply text and a target action for the question to be processed.

[0088] In the disclosed embodiment, based on the model, the question to be processed and each action in the preset action library can be used as input to generate a reply text and a target action for the question to be processed.

[0089] S302: Determine a first text matching the target action from the reply text.

[0090] In the disclosed embodiment, the reply text can be grammatically analyzed based on the model to determine the first text that matches the target action. It can be understood that the first text is the semantic point aligned with the target action in the reply text, and the semantics expressed by the target action is consistent with that of the first text.

[0091] S303: Determine the playing time of the reply text.

[0092] Specifically, the playing time of the reply text is determined according to the playing speech speed parameter, that is, the total playing time of the reply can be known.

[0093] S304: Determine a second duration for playing the first text.

[0094] Specifically, the second duration of playing the first text is determined according to the playing speech speed parameter.

[0095] S305: Determine a first duration of playing a second text that is located before the first text in the reply text.

[0096] Specifically, the second duration of playing the first text is determined according to the playing speech speed parameter.

[0097] S306: Determine the target duration for executing the target action.

[0098] In a possible embodiment, based on the second duration of the first text and the preset third duration corresponding to the target action, it is determined whether the third duration needs to be adjusted. If the third duration needs to be adjusted, the target duration for executing the target action is adjusted from the third duration to a value less than or equal to the second duration; if the third duration does not need to be adjusted, the target duration for executing the target action is determined to be the third duration.

[0099] S307: Play the reply text.

[0100] Specifically, based on the target start time, the reply text is played according to the playback speech speed parameter.

[0101] S308: Execute the target action.

[0102] Specifically, in the embodiment of the present disclosure, based on the target start time, starting from the target start time, when the first duration is reached, the robot or virtual digital human is controlled to perform the target action based on the target duration, so that when the first duration is reached, the first text can be played synchronously and the target action can be performed.

[0103] Furthermore, in the embodiments of the present disclosure, in multiple rounds of dialogues of human-computer interaction, based on the interaction method in the above embodiments, a round of dialogue is achieved in which the user inputs questions to be processed and the robot outputs reply text and target actions, and the next round of questions and answers can be continued.

[0104] In the disclosed embodiments, a reply text and a target action for a question to be processed can be generated based on a model, and a deep understanding of the conversation content can be achieved through the model, thereby improving the accuracy of understanding the intention of the user's question. The target action and the first text with semantic matching in the reply text can be time-aligned, so that when the reply text is played, the target action can be executed synchronously when the first text is played, thereby achieving real-time synchronization of the action and the conversation content and improving the human-computer interaction performance.

[0105] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0106] In addition, the present disclosure also provides an interactive device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any interactive method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

[0107] Figure 4 A block diagram of an interactive device provided in an embodiment of the present disclosure.

[0108] Reference Figure 4 , an embodiment of the present disclosure provides an interactive device, the interactive device comprising:

[0109] A generation module 41 is used to perform semantic analysis on the problem to be processed and generate a reply text and a target action for the problem to be processed;

[0110] A determination module 42, configured to determine, from the reply text, a first text matching the target action according to the reply text and the target action;

[0111] A time processing module 43, used for aligning the execution start time of the target action with the playback start time of the first text;

[0112] The output module 44 is used to play the reply text, and during the playback of the reply text, according to the execution start time of the target action after time alignment and the playback start time of the first text, when playing to the first text, the target action is synchronously executed.

[0113] In a possible embodiment, when semantic analysis is performed on the problem to be processed and a reply text and a target action for the problem to be processed are generated, the generating module 41 is used to:

[0114] Based on the model, taking the problem to be processed and each action in the preset action library as input, semantic analysis is performed on the problem to be processed, and a reply text of the problem to be processed is generated;

[0115] According to the actions in the preset action library and the reply text, a target action matching the reply text is determined from the actions.

[0116] In a possible embodiment, when determining the first text matching the target action from the reply text according to the reply text and the target action, the determination module 42 is used to:

[0117] Based on the model, the reply text is parsed, and candidate texts that conform to the target grammatical structure are determined from the reply text;

[0118] A first text matching the target action is determined according to the candidate texts and the target action.

[0119] In a possible embodiment, when the execution start time of the target action is aligned with the playback start time of the first text, the time processing module 43 is used to:

[0120] Determine a second text located before the first text from the reply text;

[0121] Determining a first duration for playing the second text according to the playback speech speed parameter;

[0122] According to the first duration and the target start time, the execution start time of the target action is aligned with the playback start time of the first text, wherein the target start time indicates the start time of playing the reply text.

[0123] In a possible embodiment, the time processing module 43 is further used for:

[0124] Determining a second duration for playing the first text according to the playback speech speed parameter;

[0125] Obtaining a preset third duration corresponding to executing the target action;

[0126] A target duration for performing the target action is determined according to the second duration and the third duration.

[0127] In a possible embodiment, when determining the target duration for executing the target action according to the second duration and the third duration, the time processing module 43 is used to:

[0128] Determining whether the third duration needs to be adjusted according to the second duration and the third duration;

[0129] If the third duration needs to be adjusted, the target duration for executing the target action is adjusted from the third duration to a value less than or equal to the second duration;

[0130] When there is no need to adjust the third duration, the target duration for performing the target action is determined as the third duration.

[0131] In a possible embodiment, the reply text is played, and during the playback of the reply text, according to the time-aligned execution start time of the target action and the playback start time of the first text, when the first text is played, the target action is synchronously executed, and the output module 44 is used to:

[0132] Based on the target start time, the reply text is played, and during the playback of the reply text, when the first duration is reached, control is performed based on the target duration to execute the target action, so that when the first duration is reached, the first text is played and the target action is executed synchronously.

[0133] Each module in the above-mentioned interactive device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0134] Figure 5 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0135] Reference Figure 5 An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 501; at least one memory 502, and one or more I / O interfaces 503, connected between the processor 501 and the memory 502; wherein the memory 502 stores one or more computer programs that can be executed by the at least one processor 501, and the one or more computer programs are executed by the at least one processor 501 so that the at least one processor 501 can execute the above-mentioned interaction method.

[0136] Each module in the above electronic device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0137] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned interaction method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0138] The embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned interaction method.

[0139] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0140] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0141] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0142] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0143] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0144] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0145] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0146] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0147] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0148] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. An interactive method, characterized in that: include: Perform semantic analysis on the problem to be processed, and generate a reply text and a target action for the problem to be processed; According to the reply text and the target action, determining a first text matching the target action from the reply text; Aligning the execution start time of the target action with the playback start time of the first text; The reply text is played, and during the playing of the reply text, according to the execution start time of the target action after time alignment and the playing start time of the first text, when the first text is played, the target action is synchronously executed.

2. The interactive method according to claim 1, characterized in that: The semantic analysis of the problem to be processed is performed to generate a reply text and a target action for the problem to be processed, including: Based on the model, taking the problem to be processed and each action in the preset action library as input, semantic analysis is performed on the problem to be processed, and a reply text of the problem to be processed is generated; According to the actions in the preset action library and the reply text, a target action matching the reply text is determined from the actions.

3. The interactive method according to claim 1, characterized in that: The step of determining, according to the reply text and the target action, a first text matching the target action from the reply text comprises: Based on the model, the reply text is parsed, and candidate texts that conform to the target grammatical structure are determined from the reply text; A first text matching the target action is determined according to the candidate texts and the target action.

4. The interactive method according to any one of claims 1 to 3, characterized in that: The step of aligning the execution start time of the target action with the playback start time of the first text includes: Determine a second text located before the first text from the reply text; Determining a first duration for playing the second text according to the playback speech speed parameter; According to the first duration and the target start time, the execution start time of the target action is aligned with the playback start time of the first text, wherein the target start time indicates the start time of playing the reply text.

5. The interactive method according to claim 4, characterized in that: The method further comprises: Determining a second duration for playing the first text according to the playback speech speed parameter; Obtaining a preset third duration corresponding to executing the target action; A target duration for performing the target action is determined according to the second duration and the third duration.

6. The interactive method according to claim 5, characterized in that: The step of determining a target duration for executing the target action according to the second duration and the third duration includes: Determining whether the third duration needs to be adjusted according to the second duration and the third duration; If the third duration needs to be adjusted, the target duration for executing the target action is adjusted from the third duration to a value less than or equal to the second duration; When there is no need to adjust the third duration, the target duration for performing the target action is determined as the third duration.

7. The interactive method according to claim 5 or 6, characterized in that: The playing of the reply text and, during the playing of the reply text, synchronously executing the target action when playing the first text according to the time-aligned execution start time of the target action and the playing start time of the first text, include: Based on the target start time, the reply text is played, and during the playback of the reply text, when the first duration is reached, control is performed based on the target duration to execute the target action, so that when the first duration is reached, the first text is played and the target action is executed synchronously.

8. An interactive device, characterized in that: include: A generation module, used for performing semantic analysis on the problem to be processed, and generating a reply text and a target action for the problem to be processed; A determination module, configured to determine, based on the reply text and the target action, a first text matching the target action from the reply text; A time processing module, used for aligning the execution start time of the target action with the playback start time of the first text; The output module is used to play the reply text, and during the playback of the reply text, according to the execution start time of the target action after time alignment and the playback start time of the first text, when playing to the first text, the target action is synchronously executed.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the interaction method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the interaction method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The invention comprises a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the interaction method as described in any one of claims 1 to 7.

Images