Call method, electronic equipment, storage medium and product
By configuring scene modes and voice features for the agent, the multiple rounds of dialogue requirements in the interaction between the user and the agent are solved, and the efficient and diversified experience of voice interaction is achieved, and the interaction efficiency and experience between the user and the agent are improved.
Patent Information
- Application Number
- CN202480002211.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-14
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art is difficult to meet the needs of multiple rounds of dialogue between users and agents, especially after the functions of the agent are improved, the interaction frequency and information volume between users and agents have increased significantly, and traditional dialogue interfaces are difficult to meet the needs of users' convenient interaction.
By configuring one or more scene modes for the agent, each scene mode is configured with voice features. The user determines the target scene mode in the interactive interface, and controls the agent to perform voice interaction based on the voice feature, providing a richer interactive experience and Improve interaction efficiency.
It realizes more efficient and diversified information acquisition in the voice interaction between users and agents, improves user experience, and enhances the convenience and richness of interaction through voice feature matching.
Smart Images

Figure CN119096252A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a communication method, electronic equipment, storage medium and product. Background Art
[0002] With the development of the Internet and artificial intelligence (AI) technology, users can chat with AI-controlled objects through electronic devices. For example, in application scenarios such as intelligent customer service, intelligent assistants, and intelligent question and answer, users can send questions to the intelligent agent, and the intelligent agent will return the answer. Summary of the invention
[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0004] According to some embodiments of the present disclosure, a calling method is provided, comprising: in an interaction interface between a first object and a second object, based on input from the first object, determining a target scene mode from one or more scene modes configured for the second object, wherein each scene mode is configured with a voice feature; and controlling the second object to perform voice interaction with the first object based on the voice feature of the target scene mode.
[0005] According to some embodiments of the present disclosure, an electronic device is provided, including: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the call method of any embodiment of the present disclosure based on instructions stored in the memory.
[0006] According to some embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the call method of any embodiment of the present disclosure is executed.
[0007] According to some embodiments of the present disclosure, a computer program product is provided. When the computer program product is run on a computer, the computer executes the call method of any embodiment of the present disclosure.
[0008] Other features, aspects and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The preferred embodiments of the present disclosure are described below with reference to the accompanying drawings. The drawings described herein are used to provide a further understanding of the present disclosure. Each of the drawings together with the following detailed description is included in this specification and forms a part of the specification, and is used to explain the present disclosure. It should be understood that the drawings described below only relate to some embodiments of the present disclosure and do not constitute a limitation on the present disclosure. In the drawings:
[0010] Figure 1 A flowchart of a calling method according to some embodiments of the present disclosure is shown.
[0011] Figure 2 A flow chart of a method for controlling voice interaction according to some embodiments of the present disclosure is shown.
[0012] Figure 3 A flow chart of a method for controlling voice interaction according to other embodiments of the present disclosure is shown.
[0013] Figure 4 A flow chart of a method for controlling voice interaction according to some further embodiments of the present disclosure is shown.
[0014] Figure 5 A flow chart of a method for controlling voice interaction according to some further embodiments of the present disclosure is shown.
[0015] Fig. 6A , 6B and 6C show schematic diagrams of a call interface according to some embodiments of the present disclosure.
[0016] Figure 7 A schematic diagram of the structure of a communication device according to some embodiments of the present disclosure is shown.
[0017] Figure 8 A schematic structural diagram of an electronic device according to some embodiments of the present disclosure is shown.
[0018] Fig. 9 A schematic diagram of the structure of a computer system according to some embodiments of the present disclosure is shown.
[0019] It should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not necessarily drawn according to the actual proportional relationship. The same or similar reference numerals are used in the various drawings to represent the same or similar parts. Therefore, once an item is defined in one drawing, it may not be further discussed in subsequent drawings. DETAILED DESCRIPTION
[0020] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure, but it is obvious that the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of the embodiments is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. It should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein.
[0021] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be performed in different orders, and / or performed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown in the execution. The scope of the present disclosure is not limited in this respect. Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values of the parts and steps set forth in these embodiments should be interpreted as being merely exemplary and do not limit the scope of the present disclosure.
[0022] The term "include" and its variations used in the present disclosure mean an open term that includes at least the following elements / features but does not exclude other elements / features, that is, "including but not limited to". In addition, the term "include" and its variations used in the present disclosure mean an open term that includes at least the following elements / features but does not exclude other elements / features, that is, "including but not limited to". Therefore, include is synonymous with include. The term "based on" means "based at least in part on".
[0023] References throughout this specification to "one embodiment," "some embodiments," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. For example, the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; and the term "some embodiments" means "at least some embodiments." Furthermore, the appearances of the phrases "in one embodiment," "in some embodiments," or "in an embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment, but may do so.
[0024] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. Unless otherwise specified, the concepts of "first", "second", etc. are not intended to imply that the objects described in this way must be in a given order in time, space, ranking, or any other manner.
[0025] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0026] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0027] The embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings, but the present disclosure is not limited to these specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In addition, in one or more embodiments, specific features, structures or characteristics can be combined in any suitable manner that will be clear from the present disclosure by a person of ordinary skill in the art.
[0028] As the functions of intelligent agents become more and more complete, the dialogue between users and intelligent agents is no longer limited to solving a specific problem of the user, and the frequency and amount of information in the dialogue have increased significantly. The method of sending messages in sequence through the dialogue interface to conduct multiple rounds of dialogue is difficult to meet the usage needs of some users.
[0029] In order to improve the convenience of interaction with an intelligent agent, the present disclosure provides a call method, an electronic device, a storage medium and a product, so that a user or an object controlled by the user can perform voice interaction with the intelligent agent, such as voice chat, to improve the efficiency of interaction with the intelligent agent. In addition, the intelligent agent is configured with one or more scene modes, and each scene mode is configured with a voice feature corresponding to the scene mode, so that when interacting with the same intelligent agent, a richer interaction experience can be brought according to different scene modes. Figure 1 An embodiment of the communication method disclosed in the present invention is described.
[0030] Figure 1 FIG. 1 is a flow chart showing a method for making a call according to some embodiments of the present disclosure. Figure 1 As shown, the communication method of this embodiment includes steps S102 to S104.
[0031] In step S102, in the interaction interface between the first object and the second object, based on the input of the first object, a target scene mode is determined from one or more scene modes configured for the second object, wherein each scene mode is configured with a voice feature.
[0032] The first object may be a user; or, the first object may be an object specified by the user, such as a first agent. The second object may be an agent, such as a second agent. That is, the embodiments of the present disclosure support scenarios in which a user makes a voice call with an agent, and also support scenarios in which a first agent makes a voice call with a second agent. A user may initiate a call with a second object in an application of a terminal, such as initiating a voice call in a dialogue interface, or initiating a voice call on a homepage of the second object, and so on. A user may also select a first agent and a second agent in order to initiate a call between the first agent and the second agent, and observe the content of the call between the two through the terminal.
[0033] In the case where the first object is the first agent, after confirming that the first agent is interacting with the second agent, the user can completely not control the first agent, but only intervene when the interaction needs to be ended, or partially or completely control the first agent to perform certain behaviors during the interaction between the two. During the voice call between the first agent and the second agent, the user can obtain information from it as a bystander.
[0034] The agent can generate corresponding content based on the information it obtains (such as text, voice, image, video, etc.). It can be implemented in software, hardware, or a combination of software and hardware. The agent can also be called a robot, a digital person, a virtual agent of a machine learning model, etc. The agent can be implemented based on a machine learning model, such as a large language model (LLM) or a foundation model. The machine learning model can be a generative model, which is used to output target content based on the input information. The information input to the generative model includes the processing basis of the generative model during the generation process, such as which information is referenced to perform the generation process, the requirements of the output target content, etc. The generative model includes, for example, a model generated based on text, or a model generated based on an image, and the output of the generative model may include text, images, or a combination of the two. Of course, the input or output of the generative model may also be data of other modalities, such as audio, video, or a combination of multiple types of data. The generative model can be a single-modal model, such as a model that generates text from text (referred to as the "text-to-text model") or a model that generates images from images (referred to as the "image-to-image model"). Alternatively, the generative model can be a cross-modal model, that is, a model whose input and output belong to different modalities, such as a model that generates images from text (referred to as the "text-to-image model"). Alternatively, the input of the generative model can include multiple modalities, and the output can also include multiple modalities.
[0035] The interaction interface between the first object and the second object may be any interaction interface, such as a dialogue interface or a voice call interface. In the case where the interaction interface is a dialogue interface, the input of the first object may include, for example, text messages, voice messages, emoticons, images, videos, instructions generated by triggering controls in the dialogue interface, etc. sent by the first object. In the case where the interaction interface is a voice call interface, the input of the first object may include, for example, the content of the speech of the first object during the voice call, instructions generated by triggering controls in the voice call interface, and may also include text messages, voice messages, emoticons, images, videos, etc. sent through input controls provided by the voice call interface.
[0036] One or more scene modes configured for the second object have different voice features. Each scene mode may, for example, belong to a different theme to enrich the scene of the voice call. The voice feature is used to describe the characteristics of the voice output by the second object during the voice call, which may reflect one or more of the second object's sound characteristics, language style, response characteristics, content characteristics, etc., thereby providing a rich voice call experience for the user.
[0037] The input of the first object can directly indicate the target scene mode. For example, information of one or more scene modes can be displayed in the interactive interface, and the scene mode selected by the first object can be determined as the target scene mode. For another example, by identifying the text, voice (including voice messages in the dialogue interface or voice in a voice call), emoticons, images, videos, files, etc. sent by the first object, it can be determined whether the content sent by the first object contains or is associated with a certain scene mode, and the associated or contained scene mode is determined as the target scene mode. In the latter example, one or more scene modes may be displayed in the interface, or they may not be displayed at all, and the selection of the target scene mode is performed directly in the background based on the user's input.
[0038] In step S104, the second object is controlled to perform voice interaction with the first object based on the voice features of the target scene mode.
[0039] During the voice interaction process, the voice generated by the second object matches the voice features. For example, the voice of the second object is generated using the voice and voice features of the first object during the voice interaction process. When the voice of the second object is generated using a generative model, the text that the second object is about to say can be generated first, and then the voice is generated based on the text, and then the voice is played in a timely manner. In these two generation processes and at least one of the playback processes, the generation or playback process can be completed based on the voice features. In some embodiments, the voice features are expressed in natural language, and the voice features and the speech content of the first object can be input into the generative model to obtain the result output by the generative model. Of course, those skilled in the art can also use other forms to represent voice features, which will not be repeated here.
[0040] The voice interaction between the first object and the second object includes a voice call. A voice call refers to an instant, real-time, and continuous voice communication between the first object and the second object, similar to a phone call. In some embodiments, during the voice call, the first object may also be supported to interact with the second object in other ways, such as sending text, emoticons, videos, images, files, etc. It should be noted that the voice call in this application may also cover a video call.
[0041] The above-mentioned embodiment configures one or more scene modes for the second object that interacts with the first object, and determines the target scene mode according to the input of the first object, so as to control the second object to perform voice interaction with the first object based on the target scene mode. This enables the second object to provide richer interaction during the voice call, improves the efficiency and diversity of the user's acquisition of information, and improves the user experience.
[0042] Several embodiments of controlling a second object to perform voice interaction based on a target scene mode are described below by way of example.
[0043] Figure 2 A flow chart of a method for controlling voice interaction according to some embodiments of the present disclosure is shown. This embodiment describes the process of controlling the second object to perform voice interaction from the perspective of generating chat text and voice. Figure 2 As shown, the control method of this embodiment includes steps S202 to S206.
[0044] In step S202, a chat text is generated for the second object based on the input of the first object and the attribute of the second object.
[0045] The input of the first object includes the voice of the first object during the voice call, and the voice includes the most recent voice spoken by the first object (for example, the voice spoken within a specified time period closest to the current time, or the voice in the most recent voice conversation), so that the second object can respond more accurately to the most recent content spoken by the first object. In addition, the voice may also include the voice spoken by the first object before, so that the content of the second object's response to the first object can take into account the content discussed by the two before.
[0046] The input of the first object may also include other content, such as text, emoticons, videos, images, files, etc. input by the first object through the voice call interface or the dialogue interface during the voice call. The second object may also respond based on these contents.
[0047] The attribute of the second object is, for example, setting information of the second object, which is a description of the characteristics of the second object itself, such as the hobbies, profession, gender, age, etc. of the second object. Thus, the generated text can meet the characteristics of the second object.
[0048] In some embodiments, the chat text may be generated using a generative model. For example, the input of the first object and the attribute of the second object are input into the text generation model to generate the chat text. The information input into the text generation model may also include other content, which will not be described in detail here.
[0049] In step S204, the speech of the second object is generated based on the speech features of the target scene mode and the chat text.
[0050] In some embodiments, the voice feature includes a sound feature, and generating the voice of the second object includes: generating the voice of the second object corresponding to the chat text according to the sound feature of the target scene mode. That is, the voice of the second object can completely correspond to the chat text. In some embodiments, the sound feature includes at least one of the timbre, speech speed, and tone of the second object. That is, the voice of the second object can be a complete repetition of the chat text according to specific sound features such as timbre, speech speed, and tone of voice. Thereby, the sound attributes of the generated voice can be matched with the target scene mode.
[0051] In some embodiments, the voice feature includes a language style feature, and generating the voice of the second object includes: adjusting the chat text based on the language style feature in the target scene mode; generating the voice of the second object based on the adjusted chat text. That is, the semantics of the voice of the second object can correspond to the chat text, that is, the voice of the second object expresses the main content of the chat text, but is adjusted in the way of expression. For example, the target scene mode is the coaxing mode, the language style of the coaxing mode is cute, and the chat text is "Do you want a cat?", then the adjusted chat text can be "Do you want a kitten?" Thus, the language style of the generated voice can be matched with the target scene mode.
[0052] The above-mentioned sound features and language style features can be used alone or in combination. For example, the chat text can be adjusted based on the language style features first, and then the voice of the second object can be generated based on the sound features and the adjusted chat text.
[0053] In some embodiments, speech can be generated using a generative model. For example, the speech features of the target scene mode and the chat text are input into the speech generation model to generate speech. The information input into the speech generation model may also include other content, which will not be described in detail here.
[0054] In step S206, the voice of the second object is played.
[0055] Through the above embodiments, a chat text can be generated first, and then the chat text can be converted into speech based on the speech features. Thus, the generated speech can have speech features that match the target scene mode, thereby improving the interaction diversity in the speech interaction process.
[0056] Figure 3 FIG. 1 is a flow chart of a method for controlling voice interaction according to other embodiments of the present disclosure. In this embodiment, the voice feature includes a response frequency feature. Figure 3 As shown, the control method of this embodiment includes steps S302 to S304.
[0057] In step S302, first reference information is determined based on the voice of the first object, where the first reference information is used to indicate the degree of necessity for the second object to respond to the first object.
[0058] The first reference information may be represented by numbers, or by various types of information indicating levels, or by other means.
[0059] In some embodiments, the speech of the first object may be subjected to intent recognition to determine the sentence type, content type, keywords, and other influencing factors of the content in the speech. For sentence type, the degree of necessity for responding to a question sentence may be higher than that for a declarative sentence. For content type, the degree of necessity for responding to information-intensive content may be higher than that to content with sparse content. For example, when the first object says something like "Well... that is to say... you know... how to say it", although the content is relatively long, the amount of information is relatively sparse, and it is not very necessary to respond at this time. For keywords, the degree of necessity for responding to some keywords may be set to be high, such as "quickly say", etc. In the case of considering multiple influencing factors, the first reference information may be comprehensively determined by methods such as weighting.
[0060] In step S304, it is determined whether the second object responds to the voice of the first object according to the response frequency feature and the first reference information.
[0061] The value of the response frequency feature can represent different response frequency strategies, that is, at what degree of necessity does the voice of the first object need to be responded to. For example, the value of the response frequency feature can be set to "respond to any information", "only respond to the first-level necessary information", "only respond to the second-level necessary information", etc. When determining whether the second object responds to the voice of the first object, it can be determined whether the first reference information satisfies the response strategy corresponding to the response frequency feature. If so, it is determined that the second object needs to respond. For example, when the target scene mode is "tree hole" (the second object listens to the first object speaking), the first user can be allowed to talk as much as possible, so that the second object only responds to information with a high degree of necessity.
[0062] In the case where it is determined that the second object does not respond to the voice of the first object, the second object does not need to speak and continues to wait for the first object to speak.
[0063] This embodiment determines whether the second object responds to the voice of the first object by using the response frequency feature, so that the rhythm of the conversation can be flexibly controlled according to the target scene mode, thereby improving the user experience.
[0064] Figure 4 FIG. 1 is a flow chart of a method for controlling voice interaction according to some other embodiments of the present disclosure. In this embodiment, the voice feature includes a response speed feature. Figure 4 As shown, the control method of this embodiment includes steps S402 to S404.
[0065] In step S402, the waiting time of the second object is determined according to the response speed characteristics.
[0066] The response speed feature is used to directly or indirectly reflect the response time of the second object, that is, how long after the first object speaks, the second object speaks. The response speed feature can be a specific time, that is, it can be the waiting time itself. It can also be a time level used to reflect the length of time. For example, the response time can be determined based on the speech content of the first object and the time level, that is, some speech content can be responded to immediately, and some speech content can be responded to after a while.
[0067] For example, when the target scene mode is "interview" and the first subject is the interviewee, a slower response speed can be set to give the first subject sufficient time to think about and answer questions; for another example, when the target scene mode is "performance comedy", a faster response speed can be set to produce more dense lines and create a joyful atmosphere.
[0068] In step S404, in response to the distance between the time when the first subject last spoke and the current time exceeding the waiting time, the voice of the second subject is played.
[0069] If the distance between the last speech time of the first subject and the current time does not exceed the waiting time, the first subject speaks again, the waiting time is recalculated, and the second subject's voice is not played. In this way, when the first subject speaks intensively, the second subject will not easily interrupt, which can facilitate the first subject to say more content.
[0070] This embodiment uses the response speed feature to determine when the second object responds to the voice of the first object, so that the rhythm of the conversation can be flexibly controlled according to the target scene mode, thereby improving the user experience.
[0071] Figure 5 FIG. 2 is a flow chart of a method for controlling voice interaction according to some other embodiments of the present disclosure. In this embodiment, the voice feature includes a content feature. Figure 5 As shown, the control method of this embodiment includes steps S502 to S506.
[0072] In step S502, based on the voice and content features of the first object and the attributes of the second object, a chat text is generated for the second object. That is, the voice features configured by the target scene mode can also affect the conversation content of the second object.
[0073] In some embodiments, the text generation model can be used to process the voice of the first object (or the text corresponding to the voice), content features and attributes of the second object to generate chat text. For example, in the "coaxing to sleep" mode, the material for generating the chat text of the second object can come from some stories; in the "learning" mode, the material for generating the chat text of the second object can come from some professional materials.
[0074] In step S504, the voice of the second object is generated based on the chat text.
[0075] For example, the sound features of the second object may be directly used to generate speech corresponding to the chat text, or the chat text may be fine-tuned using the solution of the aforementioned embodiment.
[0076] In step S506, the voice of the second object is played.
[0077] Through this embodiment, the second object can generate corresponding dialogue content based on the current target scene mode, so that the topic of the chat of the second object can also match the current target scene mode.
[0078] The above embodiments exemplarily describe specific features in speech features, which may include one or more of sound features, language style features, response frequency features, response speed features, and content features, and the above embodiments may also be executed in a superimposed manner, which will not be repeated here.
[0079] The speech features of each scene mode can be manually configured or automatically generated. In some embodiments, speech features are generated for each scene mode based on the attributes of each scene mode and the attributes of the second object. For example, some intelligent agents are created by users. If the speech features are configured for multiple scene modes of these intelligent agents one by one, the workload of configuration is relatively large and the configuration efficiency is not high. By automatically generating, speech features that match both the characteristics of the second object and the characteristics of the scene mode can be generated based on the attributes of each scene mode and the attributes of the second object. For example, a generative model can be used to process the attributes of the scene mode and the attributes of the second object to generate speech features of the scene mode.
[0080] The scene modes provided by the embodiments of the present disclosure include but are not limited to coaxing to sleep, meditation, tree hole, foreign language learning, simulated interview, waking up, etc. The following describes the interaction methods in several scene modes by way of example.
[0081] In some embodiments, the target scene mode is a first mode, such as a coaxing mode. In the first mode, in response to the distance between the time when the first subject last spoke and the current time exceeding a first threshold, the call between the first subject and the second subject is terminated. That is, when the first subject does not respond to the second subject for a long time, it can be assumed that the first subject has fallen asleep, and the call between the two can be terminated to save resources.
[0082] In some embodiments, the target scene mode is a second mode, such as a foreign language learning mode. The voice features of the second mode include a language recognition indication. In the second mode, based on the language recognition indication, the voice of the first object is recognized to determine the language used by the first object; the voice of the second object is generated using the language used by the first object; and the voice of the second object is played. That is, in the second mode, if the first object speaks in Chinese, the second object also responds in Chinese; if the first object speaks in English, the second object also responds in English. As a result, the first object does not need to additionally indicate the language used by the second object, thereby improving the efficiency of language learning.
[0083] In some embodiments, the target scene mode is a third mode, such as a meditation mode. In the third mode, the second object mainly speaks, and the first object mainly listens to the second object's speech. The voice features of the third mode include pause duration. In the third mode, the first voice generated for the second object is played; in response to the first object not speaking, after the pause duration after playing the first voice, the second voice generated for the second object is played. For example, after the second object says a sentence, the first object has not responded until the pause duration is reached, at which time the second object can continue to speak. Thereby, the second object can be guided into a meditative state.
[0084] In some embodiments, the target scene mode is a fourth mode, such as a wake-up mode. In the fourth mode, the first object can specify a wake-up time. Thus, when the wake-up time arrives, the second object will actively initiate a voice call to the first object and communicate with the first object based on the voice characteristics of the wake-up mode.
[0085] Several embodiments of determining a target scene mode are described below by way of example.
[0086] In some embodiments, the interactive interface is a call interface, and determining the target scene mode includes: displaying one or more scene modes in the call interface; and determining the target scene mode based on a selection operation of the first object on the scene mode.
[0087] Fig. 6A , 6B 6C are schematic diagrams showing a call interface according to some embodiments of the present disclosure. The call interface may be a call interface between a user and an agent, or a call interface between two agents. Fig. 6A As shown in FIG. 6 , in the call interface 6, information of the agent AA as the second object is displayed, such as a nickname 61 and an avatar 62, etc. The call interface 6 also includes a selection control 63 for triggering the process of selecting a target scene mode from one or more scene modes. For example, in response to the triggering operation of the selection control 63, the following may be displayed: Figure 6B The interface shown.
[0088] exist Figure 6B In the example, the call interface 6 also includes a selection panel 64, which includes one or more scene mode selection controls 641 to 645, corresponding to the scenes "meditation", "tree hole", "English learning", "mock interview" and "chatting". In response to any one of the controls 641 to 645 being triggered, the scene mode corresponding to the triggered control is determined as the target scene mode, and the intelligent agent AA is controlled to perform voice interaction based on the voice features of the target scene mode.
[0089] For example, after the "meditation" mode is selected, the call interface 6 can be as follows: Figure 6C As shown. Figure 6C In the example, the name of the target scene mode is displayed on the selection control 63 so that the user can more clearly understand the current scene mode. In response to the selection control 63 being triggered again, the call interface 6 can be switched back to a similar Figure 6B Status shown.
[0090] In the call interface 6, a pause control 64, a hang-up control 65, a share control 66, etc. can also be set. The pause control 64 is used to pause the voice call, and when the voice call is resumed, it continues from the previous paused position. The hang-up control 65 is used to end the voice call. The share control 66 is used to share the agent (i.e., the second object) in the call or part of the content in the call with other users.
[0091] In some embodiments, the interactive interface is a dialogue interface, and determining the target scene mode includes: receiving the input of the first object in the dialogue interface; semantically understanding the input of the first object; and determining the scene mode that matches the result of the semantic understanding in one or more scene modes as the target scene mode. When matching, the understanding result of the input of the first object can be matched with the name, description, attribute, voice feature, etc. of the scene mode to determine the target scene mode. For example, during the dialogue between the first object and the second object, the first object sends a message in the dialogue interface, "I have an interview tomorrow, do you want to help me practice in advance?" By semantically understanding the input, it is found that the input matches the scene mode "simulated interview". Then, a voice call between the first object and the second object can be initiated, and the "simulated interview" mode can be directly entered. Of course, as needed, before initiating the voice call, the first object can be asked whether to confirm the initiation of the voice call process.
[0092] In addition, the voice call and the conversation can also be carried out in parallel. For example, during the voice call, a message can be sent to the second object through the conversation interface without hanging up the voice call. For example, the voice call interface is reduced and suspended on the upper layer of the conversation interface. Thus, the content input by the first object in the conversation interface can also affect the scene mode of the voice call or the content of the voice call.
[0093] In some embodiments, the interactive interface is a voice call interface, and determining the target scene mode includes: receiving the voice of the first object in the voice call interface; performing semantic understanding on the voice of the first object; and determining the scene mode that matches the result of the semantic understanding in one or more scene modes as the target scene mode. For example, during a voice call, the first object can say "I'm going to start meditating." Then, through the semantic understanding of the speech content, it can be determined that it matches the scene mode "meditation", and then the meditation mode can be automatically entered. Of course, before entering the target scene mode determined by semantic understanding, the first object can also be asked whether to confirm the selection as needed.
[0094] The above describes the communication methods of various embodiments of the present disclosure. The following further describes the devices for implementing the methods of the various embodiments described above.
[0095] Figure 7 FIG. 2 shows a schematic diagram of the structure of a communication device according to some embodiments of the present disclosure. Figure 7 As shown, the communication device 70 of this embodiment includes: a determination module 701, which is configured to determine a target scene mode from one or more scene modes configured for the second object based on the input of the first object in the interaction interface between the first object and the second object, wherein each scene mode is configured with a voice feature; and a control module 702, which is configured to control the second object to perform voice interaction with the first object based on the voice feature of the target scene mode.
[0096] In some embodiments, the control module 702 is further configured to: generate chat text for the second object based on the input of the first object and the attributes of the second object; generate voice of the second object based on the voice features of the target scene mode and the chat text; and play the voice of the second object.
[0097] In some embodiments, the voice feature includes a sound feature, and the control module 702 is further configured to generate a voice of the second object corresponding to the chat text according to the sound feature of the target scene mode.
[0098] In some embodiments, the sound feature includes at least one of the timbre, speech speed, and tone of the second object.
[0099] In some embodiments, the voice feature includes a language style feature, and the control module 702 is further configured to: adjust the chat text based on the language style feature in the target scene mode; and generate the voice of the second object based on the adjusted chat text.
[0100] In some embodiments, the voice features include response frequency features, and the control module 702 is further configured to: determine first reference information based on the voice of the first object, the first reference information being used to indicate the degree to which the second object is required to respond to the first object; and determine whether the second object responds to the voice of the first object based on the response frequency features and the first reference information.
[0101] In some embodiments, the voice feature includes a response speed feature, and the control module 702 is further configured to: determine the waiting time of the second object based on the response speed feature; in response to the distance between the time when the first object last spoke and the current time exceeding the waiting time, play the voice of the second object.
[0102] In some embodiments, the voice feature includes a content feature, and the control module 702 is further configured to: generate a chat text for the second object based on the voice, content features and attributes of the first object; generate the voice of the second object based on the chat text; and play the voice of the second object.
[0103] In some embodiments, the target scene mode is the first mode, and the control module 702 is further configured to: in response to the distance between the time when the first subject last spoke and the current time exceeding a first threshold, end the call between the first subject and the second subject.
[0104] In some embodiments, the target scene mode is the second mode, the voice features of the second mode include a language recognition indication, and the control module 702 is further configured to: recognize the voice of the first object based on the language recognition indication to determine the language used by the first object; generate the voice of the second object using the language used by the first object; and play the voice of the second object.
[0105] In some embodiments, the target scene mode is the third mode, the voice features of the third mode include pause duration, and the control module 702 is further configured to: play the first voice generated for the second object; in response to the first object not speaking, play the second voice generated for the second object after a pause duration after playing the first voice.
[0106] In some embodiments, the communication device 70 further includes: a generating module 703 configured to generate a speech feature for each scene mode according to the attributes of each scene mode and the attributes of the second object.
[0107] In some embodiments, the interactive interface is a call interface, and the determination module 701 is further configured to: display one or more scene modes in the call interface; and determine the target scene mode based on the selection operation of the first object on the scene mode.
[0108] In some embodiments, the interactive interface is a dialogue interface, and the determination module 701 is configured to: receive input of a first object in the dialogue interface; perform semantic understanding on the input of the first object; and determine a scene mode that matches the result of the semantic understanding among one or more scene modes as a target scene mode.
[0109] In some embodiments, the first object is a user or a first agent; the second object is a second agent.
[0110] It should be noted that the above-mentioned units are only logical modules divided according to the specific functions implemented by them, and are not used to limit the specific implementation methods. For example, they can be implemented in software, hardware, or a combination of software and hardware. In actual implementation, the above-mentioned units can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.). In addition, the above-mentioned units are shown with dotted lines in the drawings to indicate that these units may not actually exist, and the operations / functions implemented by them can be implemented by the processing circuit itself.
[0111] In addition, although not shown, the device may also include a memory, which may store various information generated by the device, various units contained in the device during operation, programs and data for operation, data to be sent by the communication unit, etc. The memory may be a volatile memory and / or a non-volatile memory. For example, the memory may include, but is not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a read-only memory (ROM), and a flash memory. Of course, the memory may also be located outside the device. Optionally, although not shown, the device may also include a communication unit, which may be used to communicate with other devices. In one example, the communication unit may be implemented in a suitable manner known in the art, such as including communication components such as an antenna array and / or a radio frequency link, various types of interfaces, communication units, etc. It will not be described in detail here. In addition, the device may also include other components not shown, such as a radio frequency link, a baseband processing unit, a network interface, a processor, a controller, etc. It will not be described in detail here.
[0112] Some embodiments of the present disclosure also provide an electronic device. Figure 8A schematic diagram of the structure of an electronic device according to some embodiments of the present disclosure is shown. For example, in some embodiments, the electronic device 8 can be various types of devices, for example, can include but are not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. For example, the electronic device 8 can include a display panel for displaying data and / or execution results used in the scheme according to the present disclosure. For example, the display panel can be in various shapes, such as a rectangular panel, an elliptical panel, or a polygonal panel, etc. In addition, the display panel can be not only a flat panel, but also a curved panel, or even a spherical panel.
[0113] like Figure 8 As shown, the electronic device 8 of this embodiment includes: a memory 81 and a processor 82 coupled to the memory 81. It should be noted that Figure 8 The components of the electronic device 8 shown are only exemplary and non-restrictive. The electronic device 8 may also have other components according to actual application requirements. The processor 82 may control other components in the electronic device 8 to perform desired functions.
[0114] In some embodiments, the memory 81 is used to store one or more computer-readable instructions. When the processor 82 is used to execute the computer-readable instructions, the computer-readable instructions are executed by the processor 82 to implement the method according to any of the above embodiments. For the specific implementation of each step of the method and the related explanation content, please refer to the above embodiments, and the repetition will not be repeated here.
[0115] For example, the processor 82 and the memory 81 may communicate with each other directly or indirectly. For example, the processor 82 and the memory 81 may communicate with each other through a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 82 and the memory 81 may also communicate with each other through a system bus, which is not limited by the present disclosure.
[0116] For example, the processor 82 may be embodied as various appropriate processors, processing devices, etc., such as a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The central processing unit (CPU) may be an X86 or ARM architecture, etc. For example, the memory 81 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The memory 81 may include, for example, a system memory, which may store, for example, an operating system, an application, a boot loader, a database, and other programs. Various applications and various data may also be stored in the storage medium.
[0117] In addition, according to some embodiments of the present disclosure, various operations / processes according to the present disclosure, when implemented by software and / or firmware, can be transmitted from a storage medium or a network to a computer system having a dedicated hardware structure, such as Fig. 9 The computer system 90 shown installs the programs constituting the software, and when the various programs are installed, the computer system can execute various functions, including the functions described above. Fig. 9 A schematic diagram of the structure of a computer system according to some embodiments of the present disclosure is shown.
[0118] exist Fig. 9 In the embodiment, a central processing unit (CPU) 901 performs various processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 to a random access memory (RAM) 903. In the RAM 903, data required when the CPU 901 performs various processes, etc., is also stored as needed. The central processing unit is merely exemplary, and it may also be other types of processors, such as the various processors described above. The ROM 902, the RAM 903, and the storage section 908 may be various forms of computer-readable storage media, as described below. It should be noted that although Fig. 9 ROM 902, RAM 903 and storage device 908 are shown separately in the figure, but one or more of them may be combined or located in the same or different memory or storage modules.
[0119] The CPU 901, the ROM 902, and the RAM 903 are connected to one another via a bus 904. An input / output interface 905 is also connected to the bus 904.
[0120] The following components are connected to the input / output interface 905: an input section 906, such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output section 907, including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage section 908, including a hard disk, a magnetic tape, etc.; and a communication section 909, including a network interface card such as a LAN card, a modem, etc. The communication section 909 allows communication processing to be performed via a network such as the Internet. It is easy to understand that although Fig. 9 The various devices or modules in the computer system 90 are shown to communicate via a bus 904, but they may also communicate via a network or other means, wherein the network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.
[0121] A drive 910 is also connected to the input / output interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like is mounted on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.
[0122] In the case where the above-described series of processing is realized by software, a program constituting the software can be installed from a network such as the Internet or a storage medium such as the removable medium 911 .
[0123] According to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the CPU 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0124] It should be noted that in the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer-readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0125] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0126] In some embodiments, a computer program is further provided, comprising: instructions, which, when executed by a processor, cause the processor to execute the call method of any of the above embodiments. For example, the instructions may be embodied as computer program codes.
[0127] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In situations involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0128] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0129] The modules, components or units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the names of the modules, components or units do not, in some cases, limit the modules, components or units themselves.
[0130] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0131] The above descriptions are only some embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, a technical solution formed by replacing the above features with the technical features with similar functions disclosed in the present disclosure (but not limited to).
[0132] In the description provided herein, many specific details are set forth. However, it is understood that embodiments of the present invention can be implemented without these specific details. In other cases, well-known methods, structures, and techniques are not presented in detail in order not to obscure the understanding of the description.
[0133] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0134] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.
Claims
1. A calling method, comprising: In an interaction interface between a first object and a second object, based on an input of the first object, a target scene mode is determined from one or more scene modes configured for the second object, wherein each scene mode is configured with a voice feature; The second object is controlled to perform voice interaction with the first object based on the voice feature of the target scene mode.
2. The communication method according to claim 1, wherein: The controlling the second object to perform voice interaction with the first object based on the voice feature of the target scene mode includes: generating a chat text for the second object based on the input of the first object and the attribute of the second object; Based on the voice features of the target scene mode and the chat text, generating the voice of the second object; Play the voice of the second object.
3. The communication method according to claim 2, wherein: The voice feature includes a sound feature, and the generating of the voice of the second object based on the voice feature of the target scene mode and the chat text includes: According to the sound features of the target scene mode, a voice of the second object corresponding to the chat text is generated.
4. The communication method according to claim 3, wherein: The sound feature includes at least one of the timbre, speech speed, and tone of the second object.
5. The communication method according to any one of claims 2 to 4, wherein: The voice feature includes a language style feature, and the generating of the voice of the second object based on the voice feature of the target scene mode and the chat text includes: Adjusting the chat text based on the language style features in the target scenario mode; Based on the adjusted chat text, the voice of the second object is generated.
6. The communication method according to any one of claims 1 to 5, wherein: The voice feature includes a response frequency feature, the controlling the second object based on the voice feature of the target scene mode, and performing voice interaction with the first object includes: Determining first reference information based on the speech of the first object, wherein the first reference information is used to indicate the degree to which the second object is required to respond to the first object; According to the response frequency feature and the first reference information, it is determined whether the second object responds to the voice of the first object.
7. The communication method according to any one of claims 1 to 6, wherein: The voice feature includes a response speed feature, and the controlling the second object based on the voice feature of the target scene mode and performing voice interaction with the first object includes: Determining a waiting time of the second object according to the response speed characteristic; In response to the distance between the time when the first object spoke for the last time and the current time exceeding the waiting time, the voice of the second object is played.
8. The communication method according to any one of claims 1 to 7, wherein: The voice feature includes a content feature, and the controlling the second object based on the voice feature of the target scene mode and performing voice interaction with the first object includes: generating a chat text for the second object based on the voice of the first object, the content feature and the attribute of the second object; Based on the chat text, generating a voice of the second object; Play the voice of the second object.
9. The communication method according to any one of claims 1 to 8, wherein: The target scenario mode is the first mode, and the calling method further includes: In response to the distance between the time when the first object spoke for the last time and the current time exceeding a first threshold, the conversation between the first object and the second object is terminated.
10. The communication method according to any one of claims 1 to 8, wherein: The target scene mode is the second mode, the voice features of the second mode include a language recognition indication, and the controlling the second object to perform voice interaction with the first object based on the voice features of the target scene mode includes: Based on the language recognition indication, recognize the speech of the first object to determine the language used by the first object; Generate the speech of the second object using the language used by the first object; Play the voice of the second object.
11. The communication method according to any one of claims 1 to 8, wherein: The target scene mode is the third mode, the voice features of the third mode include pause duration, and the controlling the second object to perform voice interaction with the first object based on the voice features of the selected scene mode includes: Playing the first speech generated for the second object; In response to the first object not speaking, after the pause time after playing the first voice, playing the second voice generated for the second object.
12. The calling method according to any one of claims 1 to 11, further comprising: The speech feature is generated for each scene mode according to the attribute of each scene mode and the attribute of the second object.
13. The communication method according to any one of claims 1 to 12, wherein: The interactive interface is a call interface, and the determining of a target scene mode from one or more scene modes configured by the second object based on the input of the first object includes: In the call interface, displaying the one or more scene modes; The target scene mode is determined based on the selection operation of the first object on the scene mode.
14. The communication method according to any one of claims 1 to 12, wherein: The interactive interface is a dialogue interface, and the determining of the target scene mode from one or more scene modes configured by the second object based on the input of the first object includes: In the dialogue interface, receiving input from the first object; Performing semantic understanding on the input of the first object; The scene mode among the one or more scene modes that matches the result of the semantic understanding is determined as the target scene mode.
15. The communication method according to any one of claims 1 to 14, wherein: The first object is a user or a first agent; The second object is a second agent.
16. An electronic device comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the communication method according to any one of claims 1 to 15 based on instructions stored in the memory.
17. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the communication method according to any one of claims 1 to 15 is implemented.
18. A computer program product, when the computer program product is run on a computer, enables the computer to implement the communication method according to any one of claims 1 to 15.
Citation Information
Cited By
Chat method and device based on artificial intelligence, and storage medium
CN120296136A
