Multi-speaker voice interaction method, storage medium and electronic equipment
By identifying and analyzing the user skills and intentions in the voice of multi-speakers, the problem of insufficient contextual relevance in the existing system in the scenario of multi-speakers is solved, the coherence and accuracy of multiple rounds of dialogue are achieved, and the voice interaction system is improved level of intelligence.
Patent Information
- Application Number
- CN202411319881.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-11-22
AI Technical Summary
The existing voice interaction system cannot effectively distinguish and identify the voice of different user roles in multi-speaker scenarios, resulting in insufficient context relevance and poor coherence of multiple-wheel dialogues, making it difficult to correctly understand and respond to instructions of multi-user roles in scenarios such as on-board vehicles.
By obtaining and analyzing continuous multi-speaker voice, identifying the voice skills of each user role, determining the target speaker intention that meets the conditions for multi-person interaction skills, and using the previous round of intentions to jointly analyze the voice of other speakers in subsequent conversations Semantic content, implementing contextual association and intent reasoning.
It improves the context correlation capability and intelligence level of the voice interaction system, ensures the consistency and accuracy of dialogue during alternating multi-users, and avoids context breakage.
Smart Images

Figure CN119007710A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of human-computer voice interaction, and in particular to a multi-speaker voice interaction method, a storage medium and an electronic device. Background Art
[0002] With the vigorous promotion of intelligent interaction technology, voice interaction technology has been increasingly used in various terminal devices, such as smart speakers, voice assistants, in-car voice systems, etc. These systems realize natural interaction between people and devices through voice recognition and natural language processing technology.
[0003] Currently, most voice interaction systems can only process the voice input of one user role. This limitation makes it impossible for the system to distinguish and recognize the voices of different user roles. For example, in a smart home scenario, when multiple family members interact with a smart speaker at the same time, the system cannot distinguish the voices of different members, resulting in a significant reduction in the accuracy and personalization of the response.
[0004] In addition, the ability to associate context in multi-round conversations is an important factor affecting the intelligence level of voice interaction systems. Existing systems usually have limited context association capabilities when processing multi-round conversations and are unable to maintain the consistency and coherence of conversations in multi-user role interactions. For example, in an in-vehicle voice interaction scenario, the driver and passengers may alternately issue voice commands. Existing systems find it difficult to correctly understand and associate the context information of different user roles, resulting in insufficient continuity and accuracy in conversations.
[0005] The industry has not yet proposed a better solution to the above problems. Summary of the invention
[0006] The present application provides a multi-speaker voice interaction method, a storage medium and an electronic device, which are used to at least solve the problems of insufficient context relevance and poor coherence of multi-round conversations in the current related art voice interaction system in multi-speaker scenarios.
[0007] In a first aspect, an embodiment of the present application provides a multi-speaker voice interaction method, which is applied to a server, and the method includes: obtaining a continuous first round of multi-speaker voice and a second round of multi-speaker voice, and parsing the voice skills hit by each first speaker voice in the first round of multi-speaker voice; when it is determined that the voice skill hit by the first target speaker voice meets the preset multi-person interaction skill condition, parsing the first skill conversation intention corresponding to the first target speaker voice; for each second speaker voice in the second round of multi-speaker voice, parsing the voice semantic content of the second speaker voice, and determining the second skill conversation intention corresponding to the second speaker voice according to the voice semantic content of the second speaker voice and the first skill conversation intention.
[0008] In a second aspect, an embodiment of the present application provides a multi-speaker voice interaction method, which is applied to a client, and the method includes: collecting voice audio in a sound pickup environment; performing multi-zone separation on the voice audio to determine the zone voices corresponding to each zone; assigning corresponding speaker identifiers to each zone voice; sending each zone voice and the corresponding speaker identifier to a server; wherein the server is used to execute any multi-speaker voice interaction method executed by the server in the present application.
[0009] In a third aspect, an embodiment of the present application provides a vehicle-mounted terminal for implementing any multi-speaker voice interaction method executed by a client in the present application.
[0010] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the multi-speaker voice interaction method of any embodiment of the present application.
[0011] In a fifth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the multi-speaker voice interaction method of any embodiment of the present application are implemented.
[0012] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the multi-speaker voice interaction method of any embodiment of the present application.
[0013] The beneficial effects of the embodiments of the present application are: By performing independent skill analysis on the speech of each user role in the previous round of multi-speaker speech, it is identified whether the voice skills hit by each user role meet the multi-person interactive skill conditions. If the multi-person interactive skill conditions are hit, this intention is used in subsequent multi-round conversations to jointly analyze the speech semantic content of other speakers for conversation intention. In this way, when multiple user roles issue instructions alternately, the intentions in the previous round of conversation can be associated to make correct decision responses, avoiding context breaks and improving the context association capability and intelligence level of the voice interaction system. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0015] Figure 1 A flowchart of an example of a multi-speaker voice interaction method according to an embodiment of the present application is shown; Figure 2 An operation flow chart showing an example of determining a second skill conversation intention corresponding to a second speaker's speech; Figure 3 A flowchart of an example of a multi-speaker voice interaction method according to an embodiment of the present application is shown; Figure 4 A schematic diagram showing the effect of an example of microphone layout according to an embodiment of the present application; Figure 5 A flowchart of an example of a context-associated multi-person voice interaction method according to an embodiment of the present application is shown; Figure 6 It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION
[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0017] It should be noted that in recent years, individual technologies such as Automatic Speech Recongnition (ASR), Natural Language Processing (NLP), and Dialogue Management (DM) have entered the mature application stage, providing basic capabilities for voice interaction, and traditional full-link voice dialogue technology (DUI Dialog Service, DDS) based on the dialogue customization platform (Dialogue User Interface, DUI) is also being continuously applied and promoted in various business fields.
[0018] Speech recognition mainly converts the speech content uttered by people into text information that can be read by computers. It has two working modes: recognition mode and command mode. The implementation of speech recognition programs will also use different types of programs according to the differences between the two modes. The working principle of recognition mode is: the engine system directly provides a vocabulary and recognition template library in the background. No system needs to make further changes to the recognition grammar. It only needs to be rewritten according to the main program source code provided by the recognition engine. The command mode is relatively difficult to implement. The dictionary must be written by the programmer himself, and then programming is carried out. Finally, it must be processed and corrected according to the speech dictionary. The biggest difference between recognition mode and command mode is that programmers need to check and modify the code according to the content of the dictionary.
[0019] Natural language processing is an important means to achieve natural language communication between humans and machines. It includes two parts, natural language understanding (NLU) and natural language generation (NLG), which can enable computers to understand the meaning of natural language texts and express given intentions, ideas, etc. in natural language texts. Natural language understanding is to build a computer model. Natural language understanding is based on linguistics and integrates disciplines such as logic, psychology and computer science. It attempts to solve the following problems: How is language organized to transmit information? How do people get information from a series of language symbols? In other words, through the analysis of grammar, semantics and pragmatics, the semantic representation of natural language is obtained and the intention expressed by natural language text is understood. Natural language generation is a branch of artificial intelligence and computational linguistics. The corresponding language generation system is a computer model based on language information processing. Its working process is opposite to natural language analysis. It starts from the abstract concept level and generates text by selecting and executing certain semantic and grammatical rules.
[0020] Traditional full-link voice interaction technology is based on a full-link intelligent dialogue system customization and development platform, which develops full-link dialogue systems from voice wake-up, recognition, synthesis to semantic understanding, dialogue management, speech synthesis, etc., to achieve the docking and implementation of intelligent voice language technology, cloud content resources and terminal application product needs. In addition, the platform has large-scale scenario customization capabilities, which can provide customers with in-depth customized development services, standardized oral interaction software applications differentiated by application scenarios, AI voice chips and AI modules, and customized whole-machine products and self-branded terminal products with full-link technology.
[0021] It should be noted that in the traditional voice interaction system, it only maintains the contextual relationship of single-round dialogue and multi-round dialogue, and does not maintain the contextual relationship of voice interaction from the multi-person dimension. It can realize single-person single-round voice interaction and adopt a one-question-one-answer strategy. For example, the value of customer service robots lies in that they can replace manual solving of most consulting problems for single, repetitive and common problems through knowledge base, intent recognition and other AI technologies, realize self-service, help enterprises upgrade customer service experience, improve service efficiency and reduce labor costs. In addition, with the needs of business scenarios and the development of science and technology, single-person multi-round voice interaction has gradually been introduced, and by maintaining its contextual relationship, it has been applied to products such as smart speakers and story machines.
[0022] However, these voice interaction products or systems only support one-time interaction with a single person, and do not support the response to the demand for simultaneous voice interaction with multiple people. Due to different application scenarios, there are more and more intelligent applications for multi-person voice interaction, such as: in-vehicle voice interaction scenarios, smart ordering systems, etc. In particular, it cannot meet the needs of multi-person voice interaction scenarios with contextual associations. For example, in a car scenario, the main driver, the co-driver, the rear left seat, and the rear right seat passengers all interact with each other at the same time and express different intentions, making it difficult to respond accurately and quickly to such multi-person voice interaction needs.
[0023] Figure 1 The flowchart of an example of a multi-speaker voice interaction method according to an embodiment of the present application is shown. Regarding the implementation subject of the method of the embodiment of the present application, it can be a server.
[0024] like Figure 1 As shown, in step S110, a continuous first round of multi-speaker speech and a second round of multi-speaker speech are obtained, and the voice skills hit by each first speaker speech in the first round of multi-speaker speech are analyzed.
[0025] In some implementations, the server can continuously receive multiple rounds of multi-speaker speech from the client. Here, one round of multi-speaker speech can refer to the speech emitted by multiple speakers simultaneously within the speech sampling time window (for example, 1 minute or 30 seconds, etc.), thereby realizing continuous detection of multi-speaker speech in the sound pickup environment. In the process of voice skill analysis of multi-speaker speech, it is first necessary to separate the speaker speech of each user role (or persona) in the multi-speaker speech. On the one hand, the speech of each speaker in the multi-speaker speech is clear or pre-separated, for example, pre-separated by the client. On the other hand, the server can also obtain multiple speaker speech by implementing a speaker audio separation algorithm on the multi-speaker speech, and all belong to the implementation scope of the embodiment of the present application. Furthermore, for each speaker speech, speech recognition technology and semantic understanding technology are respectively implemented to parse out the voice skills hit by each user speech. The business types of these skills can be diverse, such as playing music, asking about the weather, adjusting the device status, etc.
[0026] In step S120, when it is determined that the voice skill hit by the first target speaker's voice meets the preset multi-person interactive skill condition, the first skill conversation intention corresponding to the first target speaker's voice is analyzed.
[0027] Here, skill conversation intent refers to the target operation that the user wants to complete through voice input, such as switching songs, raising or lowering the air conditioning temperature, etc. It may include the skill name, task name, intent name, and the session identifier (sessionId) of the current conversation round. In addition, the multi-person interaction skill conditions may be diverse, such as specific business voice skills or voice skills that integrate the speaker's identity, and so on. In some examples of the embodiments of the present application, the multi-person interaction skill conditions are defined based on at least one pre-calibrated business voice skill, such as various shared operation skills such as vehicle control skills and music skills, and can be adjusted or optimized according to business needs.
[0028] For example, the voice skills parsed in the first round are compared with the preset multi-user interaction skill conditions, which can be a skill set in a specific scenario, such as tasks that can be controlled by multiple users in a smart home scenario (such as lighting, temperature, media playback, etc.), and window adjustment operations, air conditioning adjustment operations, music adjustment operations for passengers in different seats in a car control scenario, etc. If a voice skill belongs to a skill type in a multi-user interaction scenario, it meets the conditions, and the first speaker voice corresponding to the voice skill is used as the first target speaker voice, and the corresponding first skill conversation intention is parsed. In this way, voice input that meets the multi-user interaction conditions is intelligently screened out to provide context support for subsequent user command linkage processing.
[0029] In step S130, for each second speaker's speech in the second round of multi-speaker speech, the speech semantic content of the second speaker's speech is analyzed, and the second skill conversation intention corresponding to the second speaker's speech is determined according to the speech semantic content of the second speaker's speech and the first skill conversation intention.
[0030] Here, when parsing the second round of speech, the skill conversation intent in the first round will be combined to ensure that the second round of operations maintains consistency with the previous round of conversation content. For example, if the first target speaker issued a command to "turn on the TV" in the first round, and the second target speaker asked to "turn up the volume" in the second round, then based on the intent of the first round, it can be understood that the second target speaker's command is to further adjust the volume of the TV.
[0031] Through the embodiments of the present application, for voice interaction scenarios that meet the conditions of multi-person interactive skills, a context-based semantic reasoning mechanism is used to associate the current voice with the previous conversation content (i.e., the first skill conversation intention). In this way, even if the voice intention issued by the second speaker is vague or omits the context, the voice interaction system can automatically complete it according to the previous round of conversation intention, thereby enhancing the context relevance of multiple rounds of conversations. For example, the second user only says "turn it up a little bit" in the second round, and the voice interaction system can also infer that this is an instruction to adjust the TV volume based on the "turn on the TV" intention of the first round. As a result, multiple users are allowed to issue instructions continuously in the same task scenario, and the voice interaction system can also effectively understand and process these continuous instructions, ensuring the fluency and consistency of the entire conversation process, and improving the naturalness of multi-user alternating interactions.
[0032] Figure 2 An operational flow chart showing an example of determining a second skill conversation intention corresponding to a second speaker's speech.
[0033] like Figure 2 As shown, in step S210, based on the first skill conversation intention, the conversation context information of the second speaker's voice is updated.
[0034] Here, a corresponding conversation context information is maintained for each persona in multiple rounds of conversations. The context information may include comprehensive information about the persona's multi-round conversation intentions, and may indicate the current task or skill status (for example, whether a device has been started, a media is being played, etc.), related device status information (such as TV volume, light brightness, etc.), and skill intentions and their details triggered by users in historical rounds of voice interactions (such as user instructions to play music, adjust the temperature, and other operational intentions).
[0035] In some embodiments, after the first round of conversation ends, the context information of each user (including other users) is synchronously updated based on the parsed first skill conversation intention involving the multi-person interaction scenario. For example, if the first speaker issues a "play music" command, the "playing music" status will be synchronously updated in the context information of each user, and the relevant information of the playback device (such as speakers, TVs, etc.) will be recorded.
[0036] In some additional or alternative implementations, the session context information may also be maintained by the client, for example, the client maintains the session context information of each persona separately. Then, when a new round of conversation occurs, the client may upload the speaker voice and corresponding session context information of each persona to the server. At this time, the server may feed back the first skill session intent and the corresponding update instruction to the client, so that the client will use the first skill session intent to update the session context information of all personas.
[0037] In step S220, based on the speech semantic content of the second speaker's speech and the updated conversation context information of the second speaker's speech, the second skill conversation intention corresponding to the second speaker's speech is determined.
[0038] In some embodiments, when parsing the second speaker's voice, the voice is first converted into text using speech recognition technology, and the semantic content in the voice is analyzed using NLP technology, which is not limited to recognizing specific keywords in the command, but also includes understanding the context, intention, and potential ambiguity or abbreviated expressions of the sentence. For example, if the second speaker says "turn it up a little more", the semantics of "volume" can be inferred from the context. Furthermore, if the context information records the previous round of "playing music" status, then the second speaker's instruction "turn it up a little more" will be inferred as "turn up the music volume" in combination with the context. On the other hand, if the intention of the speaker in the previous round is to "turn on the air conditioner", then when the subsequent speaker issues the instruction "turn it up two degrees", it will be automatically associated with the control intention of the air conditioner and correctly understand the meaning of the instruction.
[0039] Through the embodiments of the present application, the context information of each speaker is updated using the skill conversation intention that meets the conditions of multi-person interactive skills, so that the intention of the current round of speech can be understood more accurately and the implicit information in the instruction can be inferred. In addition, through the real-time update of the context, complex multi-round conversations can be handled, ensuring that in multi-user scenarios, the skill collaborative operations of multiple users are continuously understood and responded to, and the continuity and coherence of the conversation are maintained.
[0040] In some examples of the embodiments of the present application, each speaker's speech in the multi-speaker speech has a unique corresponding sound zone, and the sound zone is a specific spatial area determined by performing sound zone separation on the collected speech audio.
[0041] Exemplarily, the voice zones of multiple speakers are separated by applying a voice zone separation algorithm, thereby obtaining speakers' voices corresponding to different voice zones or sound sources. It should be noted that the voice zone separation operation for multiple speakers' voices can be implemented by the server on the one hand, and by the client on the other hand, and both are within the scope of the embodiments of the present application. More preferably, the voice zone separation algorithm can be completed by each client, which can effectively reduce the resource pressure and response delay on the server, and is conducive to promotion in large-scale business application scenarios. Here, the voice zone separation algorithm can adopt a known or potential algorithm, which should not be limited here for the time being.
[0042] Figure 3 The flowchart of an example of a multi-speaker voice interaction method according to an embodiment of the present application is shown. Regarding the execution subject of the method of the embodiment of the present application, it can be a client. In some more specific scenarios, it can be a vehicle terminal.
[0043] like Figure 3 As shown, in step S310, voice audio in the sound pickup environment is collected.
[0044] In step S320, the speech audio is separated into multiple zones to determine the zone speech corresponding to each zone.
[0045] In step S330, a corresponding speaker identifier is assigned to each voice zone.
[0046] In step S340, each zone voice and the corresponding speaker identifier are sent to the server. Here, the zone voice corresponds to the speaker voice, and the server is used to perform the above combination. Figure 1 Or the operations in the multi-speaker speech interaction method shown in 2.
[0047] Through the embodiments of the present application, the client performs local audio separation on the collected audio according to the region, obtains the simultaneous or alternating voices of multiple speakers and the corresponding user roles or personas, and achieves the goal of separating speakers by region. Furthermore, the server implements recognition, semantics, and dialogue management to complete the response operations of skills in various fields, such as music, vehicle control, navigation, etc. As a result, the server does not need to implement multi-zone separation or multi-speaker separation operations for all clients, which reduces the resource consumption requirements on the server and improves the efficiency of voice interaction.
[0048] Figure 4 The schematic diagram of the effect of an example of microphone layout according to the embodiment of the present application is shown. Specifically, based on multiple distributed microphones, voice audio in the sound pickup environment is collected, and each microphone is respectively set in a different physical space area. Figure 4 As shown in the figure, in the vehicle environment, the interior space is divided into different areas such as the driver's seat, the front passenger seat, and the back seat, and each area is equipped with a microphone. Through intelligent detection technology, the sensitive area of the microphone can be dynamically adjusted according to the intensity of the real-time voice signal and the direction of the sound source, so as to better separate the voice signal of each sound zone.
[0049] It should be noted that a sound zone refers to dividing a fixed physical space (such as the space inside a car) into different areas. A microphone is deployed in each area, indicating that the microphone has the strongest sound reception in this area. The collected audio is then separated into the sound in this area through the core signal processing algorithm. The sound zone separation algorithm separates the audio collected by a microphone into audio in a fixed spatial range. According to different sound zones, the audio of the corresponding area is separated, and the area is referred to as a sound zone.
[0050] Regarding the details of the sound zone separation algorithm, it can adopt a beamforming algorithm or a blind source separation algorithm. On the one hand, beamforming is to enhance the sound signal from a specific direction or area by adjusting the receiving angle and signal weight of the microphone array, while suppressing interference signals from other directions. By calculating the audio signal energy of different sound zones at each microphone, the position of the speaker can be accurately determined, and then the area is defined as a sound zone. On the other hand, the blind source separation algorithm is a technology that separates independent signal sources from mixed signals, which can extract the speech of each sound zone from the mixed audio signals collected by multiple microphones. Specifically, by analyzing the statistical independence of the mixed audio signal sampled by the microphone array, independent speech signals of different sound zones are extracted, thereby separating different speaker signals.
[0051] Figure 5 A flowchart of an example of a context-associated multi-person voice interaction method according to an embodiment of the present application is shown.
[0052] like Figure 5 As shown, distributed microphones are used to collect audio in multiple sound zones, and then multi-person audio is separated. After algorithm processing, the multi-person audio is clearly extracted. Therefore, the sound zones are separated locally, and then recognition, semantics, and dialogue processing are performed, and the contextual relationship of each person's voice interaction is maintained on the server side. Then, when multiple people interact with multiple voice dialogues, the contextual relationship of the person is arbitrated based on the voice and dialogue processing.
[0053] The method provided in the embodiment of the present application mainly includes two types of scenarios, namely, the scenario of multi-person voice interaction without contextual relationship and the scenario of multi-person voice interaction with contextual relationship.
[0054] Combined with the description of the scenario of multi-personnel voice interaction without context, when there are multiple speakers interacting with each other at the same time, for example, person 1 says: open the window, and person 2 says: adjust the air conditioning temperature to 26 degrees. Since there is only a single round of multi-personnel interaction, there is no contextual association analysis. At this time, audio is collected through audio distributed microphones, and the audio of each person is separated for each sound zone, and the person identification and session identification are performed. Then, the multi-personnel audio is sent to the online multi-channel recognition service, the multi-channel recognition results are sent to the online multi-channel semantic service, and the multi-channel semantic results are sent to the multi-channel dialogue service. At this time, the client (for example, the vehicle terminal) receives the dialogue results of the multi-personnel, and executes the command to open the window of person 1 and adjust the air conditioning temperature of person 2 to 26 degrees, and then ends the dialogue and clears the session identification.
[0055] Combined with the description of the scenario of multi-personnel voice interaction with contextual relationship, after the above-mentioned first round of voice interaction, a new round of voice interaction, namely the second round of voice interaction, occurred. For example, persona 1 said: I want to listen to music, and persona 2 said: I want it too. At this time, audio collection is performed through the audio distributed microphone, the audio of each persona is separated for each sound zone, and the persona identifier and the previous round of session identifier are set. Then, the multi-personnel audio is sent to the online multi-channel recognition service, the multi-channel recognition results are sent to the online multi-channel semantic service, the multi-channel semantic results are sent to the multi-channel dialogue service, and the multi-personnel contextual relationship is arbitrated. Finally, the client receives the dialogue results of the multi-personnel, and executes the command to open persona 1 to listen to music, and opens the window of persona 2, then ends the dialogue and clears the session identifier.
[0056] Regarding the explanation of the arbitration of whether there are multiple persona contexts, in some implementations, it is determined whether the skill needs to synchronize the context to other voice zone personas based on business needs. If synchronization is required, the skill name, task name, intent name, and session identifier sessionId of the voice zone persona are synchronously sent to the personas of other voice zones, overwriting the conversation context of the current voice zone persona.
[0057] More specifically, multi-channel voice interaction is when the client sends multiple audio channels to the server at the same time, and the server returns the results of the multi-channel conversation to the client. For example, in a four-way conversation (corresponding to four persona relationships), the client sends the four-way audio channels to the service, and then the client receives the four-way conversation results returned by the server (e.g., zone 1 listens to music, zone 2 asks about the weather, zone 3 opens the car window, and zone 4 turns on the air conditioner). At this time, the four personas will perceive that their respective intentions have been executed. If the business party defines "open the car window" as a multi-persona interaction skill, the contextual relationship of zone 1, zone 2, and zone 4 will be synchronized with zone 3. Then, zone 2 says "I want it too", and the persona of zone 2 will perceive the operation of opening the car window at the position of zone 3.
[0058] Whether a voice skill triggers a multi-person interactive skill condition can be determined based on preset information such as specifying skills, tasks, intents, and configuring corresponding speakers according to business needs. For example, in the first round of voice interaction (said at the same time): Person 1: How is the weather today; Person 2: How is the stock today. At this time, Person 1 receives today's weather information, and Person 2 receives stock information. In addition, assuming that the "weather" skill triggers the multi-person interactive skill condition, the weather skill, task, intent, and session identifier of Person 1 are synchronized to Person 2. Then, in the second round of voice interaction (said at the same time): Person 1: Exit; Person 2: What about tomorrow. At this time, Person 1 will exit the conversation, and Person 2 will receive the conversation result of tomorrow's weather.
[0059] Through the embodiments of the present application, multiple people can interact with each other at the same time, which largely meets the needs of multi-language scenarios, and one round of interaction takes a short time and has high stability. Compared with the traditional single-person voice interaction solution currently on the market, it is more intelligent and flexible.
[0060] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0061] In some embodiments, an embodiment of the present application provides a non-volatile computer-readable storage medium, in which one or more programs including execution instructions are stored, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute any of the above-mentioned multi-speaker voice interaction methods of the present application.
[0062] In some embodiments, the embodiments of the present application further provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any of the above-mentioned multi-speaker speech interaction methods.
[0063] In some embodiments, an embodiment of the present application also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform a multi-speaker voice interaction method.
[0064] Figure 6 is a schematic diagram of the hardware structure of an electronic device for executing a multi-speaker voice interaction method provided by another embodiment of the present application, such as Figure 6 As shown, the device includes: One or more processors 610 and memory 620, Figure 6 A processor 610 is taken as an example.
[0065] The device for executing the multi-speaker voice interaction method may further include: an input device 630 and an output device 640 .
[0066] The processor 610, the memory 620, the input device 630 and the output device 640 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.
[0067] The memory 620 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the multi-speaker voice interaction method in the embodiment of the present application. The processor 610 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 620, that is, the multi-speaker voice interaction method in the above method embodiment is implemented.
[0068] The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 620 may optionally include a memory remotely arranged relative to the processor 610, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0069] The input device 630 can receive input digital or character information and generate signals related to user settings and function control of the electronic device. The output device 640 can include a display device such as a display screen.
[0070] The one or more modules are stored in the memory 620 , and when executed by the one or more processors 610 , the multi-speaker voice interaction method in any of the above method embodiments is executed.
[0071] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.
[0072] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.
[0073] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.
[0074] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.
[0075] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.
[0076] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.
[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A multi-speaker voice interaction method, applied to the server, the method include: Acquire continuous first-round multi-speaker speech and second-round multi-speaker speech, and analyze the speech skills hit by each first speaker speech in the first-round multi-speaker speech; When it is determined that the voice skill targeted by the first target speaker's voice meets the preset multi-person interactive skill condition, parsing the first skill conversation intention corresponding to the first target speaker's voice; For each second speaker's speech in the second round of multi-speaker speech, the speech semantic content of the second speaker's speech is analyzed, and the second skill conversation intention corresponding to the second speaker's speech is determined according to the speech semantic content of the second speaker's speech and the first skill conversation intention.
2. The method according to claim 1, in, The determining, according to the speech semantic content of the second speaker's speech and the first skill conversation intention, the second skill conversation intention corresponding to the second speaker's speech includes: Based on the first skill conversation intention, updating the conversation context information of the second speaker's voice; Based on the speech semantic content of the second speaker's speech and the updated conversation context information of the second speaker's speech, a second skill conversation intention corresponding to the second speaker's speech is determined.
3. The method according to claim 1, in, The multi-person interactive skill condition is defined based on at least one pre-calibrated business voice skill.
4. The method according to claim 1, in, Each speaker's speech in the multi-speaker speech has a unique corresponding sound zone; the sound zone is a specific spatial area determined by performing sound zone separation on the collected speech audio.
5. A multi-speaker voice interaction method, applied to a client, the method include: Collect voice audio in the sound pickup environment; Performing multi-zone separation on the speech audio to determine the zone speech corresponding to each zone; Assigning corresponding speaker identifications to the voices in each of the voice zones; Sending the voice in each of the voice zones and the corresponding speaker identifier to a server; The server is used to execute the multi-speaker voice interaction method according to any one of claims 1 to 4.
6. The method according to claim 5, in, The collecting of voice audio in the sound pickup environment includes: Based on multiple distributed microphones, voice audio in the sound pickup environment is collected; each of the microphones is respectively set in a different physical space area.
7. A vehicle-mounted terminal, used to implement the multi-speaker voice interaction method as claimed in claim 5 or 6.
8. A storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
9. An electronic device, wherein include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method described in any one of claims 1 to 6.
10. A computer program product comprising a computer program / instructions, in, When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
