Voice interaction method and device, electronic equipment and computer storage medium

Abstract

The invention provides a voice interaction method and device, electronic equipment and a computer storage medium, and the method comprises the steps: obtaining voice instruction information sent by a user; performing recognition processing on the voice instruction information to obtain text information corresponding to the voice instruction information; determining a target skill type corresponding to the text information based on the text information; and analyzing the text information through a target processing model corresponding to the target skill type to obtain a processing result corresponding to the text information. According to the embodiment provided by the scheme, the voice interaction efficiency and accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of voice intelligent assistants, and in particular to a voice interaction method, device, electronic device and computer storage medium. Background Art

[0002] As an interactive tool in smart devices, voice assistants have been widely used in many scenarios of daily life. Users can complete various functions through voice assistants. These functions allow users to experience a more convenient and intelligent lifestyle. However, despite the increasing number of application scenarios of voice assistants, voice assistants in related technologies still have some limitations in understanding user questions and accurately identifying user intentions. Summary of the invention

[0003] The embodiments of the present application provide a voice interaction method, device, electronic device and computer storage medium, which can improve the efficiency and accuracy of voice interaction. The above technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a voice interaction method, the method comprising:

[0005] Obtain the voice command information issued by the user;

[0006] Recognize and process the voice command information to obtain text information corresponding to the voice command information;

[0007] Determine the target skill type corresponding to the text information based on the text information;

[0008] The above text information is analyzed by the target processing model corresponding to the above target skill type to obtain the processing result corresponding to the above text information.

[0009] In a possible implementation, determining the target skill type corresponding to the text information based on the text information includes:

[0010] The above text information is identified through a preset intent model to obtain a target skill type corresponding to the above text information, and the above target skill type is any one of the following: cloud skill type, local skill type, and question-and-answer skill type.

[0011] In a possible implementation, when the target skill type is the local skill type, the target processing model corresponding to the target skill type is a local skill processing model;

[0012] The above-mentioned text information is analyzed by the target processing model corresponding to the above-mentioned target skill type to obtain the processing result corresponding to the above-mentioned text information, including:

[0013] Analyze the text information by using the local skill processing model of the first terminal device to obtain a first operation instruction corresponding to the text information;

[0014] Execute the first operation instruction to obtain the processing result corresponding to the text information.

[0015] In a possible implementation, when the target skill type is the cloud skill type, the target processing model corresponding to the target skill type is a cloud skill processing model;

[0016] The above-mentioned text information is analyzed by the target processing model corresponding to the above-mentioned target skill type to obtain the processing result corresponding to the above-mentioned text information, including:

[0017] Generate corresponding request information based on the above text information;

[0018] Sending the request information to a cloud device, so that the cloud device analyzes the text information included in the request information through the cloud skill processing model to obtain a processing result of the text information;

[0019] Receive the processing result corresponding to the above text information fed back by the above cloud device.

[0020] In a possible implementation, the request information is used to instruct the second terminal device to execute the second operation instruction, and the processing result corresponding to the text information is prompt information, and the prompt information is used to indicate whether the second terminal device executes the second operation instruction successfully or fails.

[0021] In a possible implementation, when the target skill type is the question-and-answer skill type, the target processing model corresponding to the target skill type is a question-and-answer skill processing model, and the processing result corresponding to the text information is playing the answer text corresponding to the text information;

[0022] The above-mentioned text information is analyzed by the target processing model corresponding to the above-mentioned target skill type to obtain the processing result corresponding to the above-mentioned text information, including:

[0023] Analyze the text information through the question-answering skill processing model to generate an answer text corresponding to the text information;

[0024] Play the above answer text.

[0025] In a possible implementation, before obtaining the voice command information issued by the user, the method further includes:

[0026] Receiving a voice wake-up signal sent by the user;

[0027] Determine whether the voice wake-up signal matches the wake-up word of the preset voice assistant;

[0028] If yes, the state of the preset voice assistant is determined as the awake state;

[0029] When the state of the preset voice assistant is the wake-up state, detect whether the user issues the voice command information.

[0030] In a second aspect, an embodiment of the present application provides a voice interaction device, the device comprising:

[0031] An acquisition module is used to acquire the voice command information issued by the user;

[0032] A processing module, used to identify and process the voice command information to obtain text information corresponding to the voice command information;

[0033] A determination module, used to determine the target skill type corresponding to the text information based on the text information;

[0034] The analysis module is used to analyze the above text information through the target processing model corresponding to the above target skill type to obtain the processing result corresponding to the above text information.

[0035] In a possible implementation, the determination module includes:

[0036] The recognition unit is used to recognize the above text information through a preset intent model to obtain a target skill type corresponding to the above text information, and the above target skill type is any one of the following: cloud skill type, local skill type, and question-and-answer skill type.

[0037] In a possible implementation, when the target skill type is the local skill type, the target processing model corresponding to the target skill type is a local skill processing model;

[0038] The above analysis module includes:

[0039] A first analyzing unit, configured to analyze the text information by using the local skill processing model of the first terminal device to obtain a first operation instruction corresponding to the text information;

[0040] The execution unit is used to execute the first operation instruction to obtain the processing result corresponding to the text information.

[0041] In a possible implementation, when the target skill type is the cloud skill type, the target processing model corresponding to the target skill type is a cloud skill processing model;

[0042] The above analysis module includes:

[0043] A generating unit, used to generate corresponding request information based on the above text information;

[0044] A sending unit, configured to send the request information to a cloud device, so that the cloud device analyzes the text information included in the request information through the cloud skill processing model to obtain a processing result of the text information;

[0045] The receiving unit is used to receive the processing result corresponding to the above text information fed back by the above cloud device.

[0046] In a possible implementation, the request information is used to instruct the second terminal device to execute the second operation instruction, and the processing result corresponding to the text information is prompt information, and the prompt information is used to indicate whether the second terminal device executes the second operation instruction successfully or fails.

[0047] In a possible implementation, when the target skill type is the question-and-answer skill type, the target processing model corresponding to the target skill type is a question-and-answer skill processing model, and the processing result corresponding to the text information is playing the answer text corresponding to the text information;

[0048] The above analysis module includes:

[0049] A second analysis unit is used to analyze the text information through the question-answering skill processing model to generate an answer text corresponding to the text information;

[0050] The playback unit is used to play the above answer text.

[0051] In a possible implementation, the above device further includes:

[0052] A receiving module, used to receive a voice wake-up signal sent by the user;

[0053] A judgment module is used to judge whether the above-mentioned voice wake-up signal matches the wake-up word of the preset voice assistant;

[0054] The determination module is further configured to determine the state of the preset voice assistant as a wake-up state when the voice wake-up signal matches the wake-up word of the preset voice assistant;

[0055] The detection module is used to detect whether the user issues the voice command information when the preset voice assistant is in the wake-up state.

[0056] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0057] The above-mentioned memory stores a computer program, and the above-mentioned computer program is suitable for being loaded by the above-mentioned processor and executing the steps of the method provided by the first aspect of the embodiment of the present application or any possible implementation method of the first aspect.

[0058] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores multiple instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the method provided by the first aspect of the embodiment of the present application or any possible implementation method of the first aspect.

[0059] The embodiment of the present application obtains the voice command information issued by the user; recognizes and processes the above voice command information to obtain the text information corresponding to the above voice command information; determines the target skill type corresponding to the above text information based on the above text information; analyzes the above text information through the target processing model corresponding to the above target skill type to obtain the processing result corresponding to the above text information. Determining the target skill type requested by the user based on the text information corresponding to the voice command, and then adopting different processing models for different skill types can avoid using a unified model to process requests of different skill types, which is conducive to generating more targeted processing results, thereby improving the efficiency and accuracy of voice interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 A schematic diagram of the structure of a voice interaction system provided by an exemplary embodiment of the present application;

[0062] Figure 2 A flowchart of a voice interaction method provided by an exemplary embodiment of the present application;

[0063] Figure 3 A schematic diagram of the structure of a first terminal device provided for an exemplary embodiment of the present application;

[0064] Figure 4 A schematic diagram of a scenario of a voice interaction system provided by an exemplary embodiment of the present application;

[0065] Figure 5 A specific flow chart of a voice interaction method provided by an exemplary embodiment of the present application;

[0066] Figure 6A schematic diagram of the structure of a voice interaction device provided by an exemplary embodiment of the present application;

[0067] Figure 7 A schematic structural diagram of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0069] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0070] Please refer to the following Figure 1 , which exemplarily shows a structural diagram of a voice interaction system provided in an embodiment of the present application. Figure 1 As shown, the system includes a first terminal device 110 and a cloud device 120. The first terminal device 110 and the cloud device 120 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0071] In some embodiments, the first terminal device 110 can be a smart camera, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart appliance, etc., but is not limited thereto.

[0072] The cloud device 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), big data and artificial intelligence platforms.

[0073] The first terminal device 110 may send a request to the cloud device 120 , and the cloud device 120 may process the request and return a response.

[0074] In some embodiments, the first terminal device 110 is used to perform the following steps: obtain voice command information issued by a user; identify and process the voice command information to obtain text information corresponding to the voice command information; determine the target skill type corresponding to the text information based on the text information; analyze the text information through the target processing model corresponding to the target skill type to obtain the processing result corresponding to the text information.

[0075] Optionally, in some embodiments, a preset voice assistant may be installed in the first terminal device, and the first terminal device runs the preset voice assistant. The user may be a person who uses the first terminal device to perform voice interaction with the preset voice assistant.

[0076] Optionally, the voice command information is information about a request or command sent by the user to the preset voice assistant through voice input, which is used to describe the user's intention, request or operation that the user wants to perform. For example, the voice command information can be "What's the weather like today?", "Tell a fairy tale", etc.

[0077] Optionally, a microphone is installed in the first terminal device, and the microphone can be used to collect voice command information issued by the user.

[0078] Optionally, the voice command information may be recognized and processed using the Automatic Speech Recognition (ASR) technology in the related art to obtain text information corresponding to the voice command information.

[0079] Optionally, the target skill type is any one of the following: cloud skill type, local skill type, and question-and-answer skill type.

[0080] Optionally, a skill type may correspond to one or more target processing models, and by performing targeted analysis on the above text information through the target processing model corresponding to the target skill type, a processing result with high accuracy and targeting may be obtained.

[0081] In some embodiments, the cloud device 120 can also be used to perform the following steps: obtain voice command information issued by a user; identify and process the voice command information to obtain text information corresponding to the voice command information; determine the target skill type corresponding to the text information based on the text information; analyze the text information through the target processing model corresponding to the target skill type to obtain the processing result corresponding to the text information.

[0082] An exemplary embodiment of the present application provides a voice interaction method. The voice interaction method can be applied to the above-mentioned first terminal device. For details, please refer to Figure 2, which exemplarily shows a flow chart of a voice interaction method provided in an embodiment of the present application. Figure 2 As shown, the voice interaction method includes the following S21-S24:

[0083] S21. Acquire voice command information issued by the user.

[0084] In some embodiments, a preset voice assistant may be installed in the first terminal device, and the first terminal device runs the preset voice assistant. The user may be a person who uses the first terminal device to interact with the preset voice assistant by voice. In the first terminal device, the software development kit (SDK) logic provides a software interface and tools for implementing the preset voice assistant function.

[0085] In some embodiments, the above-mentioned voice command information is information about a request or command issued by a user to a preset voice assistant through voice input, which is used to describe the user's intention, request, or operation that the user wishes to perform. Specifically, the voice command information can be continuous speech within a preset duration (such as 30 seconds). For example, the voice command information can be "What's the weather like today?", "Tell a fairy tale", etc.

[0086] In some embodiments, a microphone is installed in the first terminal device, and the microphone can be used to collect voice command information issued by the user.

[0087] S22: Recognize and process the voice command information to obtain text information corresponding to the voice command information.

[0088] In some embodiments, the ASR technology in the related art can be used to recognize and process the voice command information to obtain text information corresponding to the above voice command information.

[0089] In some embodiments, before the voice command information is recognized and processed, in order to improve the recognition effect, the collected voice command information can also be pre-processed, such as noise reduction, normalization, etc.; then the pre-processed voice command information is converted into a series of feature parameters, which can effectively represent the content of the speech; the extracted feature parameters are input into the relevant acoustic model and language model, and the most likely text sequence is generated through the decoding algorithm; and the decoded text is post-processed, such as spelling and grammar checking, punctuation addition, etc., to obtain a text with higher readability and accuracy as the text information corresponding to the above-mentioned voice command information.

[0090] In some embodiments, a voice activity detection (VAD) algorithm may be used to detect whether the user is speaking. VAD is used to distinguish between background noise and voice activity, and further speech recognition processing is initiated only when valid voice activity is detected.

[0091] In some embodiments, voiceprint recognition can also be performed on the voice command information. Voiceprint recognition is a technology for identity verification based on individual voice characteristics. The identity of the speaker can be determined by analyzing and matching the voiceprint characteristics (such as frequency, pitch, sound waveform, etc.) in the audio data. The first terminal device can store the user's voiceprint data, compare the uploaded voice command information with the stored voiceprint characteristics, and perform identity verification or personalized services (such as personalized recommendations or automatic login).

[0092] S23. Determine the target skill type corresponding to the text information based on the text information.

[0093] In some embodiments, the target skill type is any one of the following: a cloud skill type, a local skill type, or a question-and-answer skill type.

[0094] Among them, cloud skill types are functions or services that rely on cloud computing and storage capabilities, and can provide complex processing, data access, or customized interactive experiences through cloud services. These skills expand the functions of preset voice assistants, enabling them to handle more complex tasks or access more data resources, such as information query (including weather services, news updates, encyclopedia content, dictionary queries, etc.), media playback and content recommendation (such as music, audio stories, movies, etc.), smart home control (such as lights, security cameras, smart door locks, etc.), productivity tools (such as setting reminders, calendar management, sending messages, making calls, etc.).

[0095] Local skills are functions or services that are executed locally on the first terminal device (not on the cloud). These skills do not rely on cloud services, but run in the local hardware and software environment of the first terminal device. For example, device control (control of the local hardware functions of the first terminal device), multimedia playback control (local control of media playback functions), local data access (viewing locally stored photos, videos, etc.), calculation and conversion (performing simple calculation and conversion tasks locally on the first terminal device without relying on an external network).

[0096] Question answering skills are specialized for answering user questions and providing relevant information. Such skills usually rely on natural language processing, information retrieval, knowledge base, or machine learning techniques to understand user questions and generate corresponding answers.

[0097] S24. Analyze the above text information through the target processing model corresponding to the above target skill type to obtain the processing result corresponding to the above text information.

[0098] In some embodiments, a skill type may correspond to one or more target processing models. By performing targeted analysis on the above text information through the target processing model corresponding to the target skill type, a processing result with high accuracy and specificity may be obtained.

[0099] In some embodiments, the processing result of the text information may be a series of operations or outputs generated by a preset voice assistant based on the text information after the user's voice command information is converted into text information. The processing result may be in various forms (such as voice or text response, operation execution result, data query and return, data storage or analysis, etc.).

[0100] The embodiment of the present application obtains the voice command information issued by the user; recognizes and processes the above voice command information to obtain the text information corresponding to the above voice command information; determines the target skill type corresponding to the above text information based on the above text information; analyzes the above text information through the target processing model corresponding to the above target skill type to obtain the processing result corresponding to the above text information. Determining the target skill type requested by the user based on the text information corresponding to the voice command, and then adopting different processing models for different skill types can avoid using a unified model to process requests of different skill types, which is conducive to generating more targeted processing results, thereby improving the efficiency and accuracy of voice interaction.

[0101] Figure 3 A schematic diagram of the structure of a first terminal device provided by an exemplary embodiment of the present application is shown in FIG. Figure 3As shown, the first terminal device can be a smart camera 300, which includes a camera 310, a dial button 320 and a night light 330. Among them, the camera 310, as the main image capture unit of the first terminal device, can be composed of two cameras, namely a first camera 311 and a second camera 312. The dual camera system is generally used to improve image quality, depth perception, wide-angle view and other functions. The dial button 320 can be used for emergency calls or to establish a communication connection with other terminal devices (such as mobile phones). When the user needs to call for help or in an emergency, the preset number can be dialed by pressing the dial button 320. The night light 330 can be used to provide soft lighting at night to facilitate capturing clear images or videos under low light conditions. The brightness of the night light 330 may be adjustable to adapt to different scene requirements. In addition, the smart camera 300 may also include a speaker and a microphone, and the speaker is used to play sounds in multiple formats, such as the Moving Picture Experts Group Audio Layer 3 format.

[0102] In some embodiments, before executing S21, obtaining the voice command information issued by the user, the method further includes S201-S204:

[0103] S201: Receive a voice wake-up signal sent by the user.

[0104] In some embodiments, when the preset voice assistant is in an unaware state, it can be detected that the user sends a voice wake-up signal, wherein the voice wake-up signal is a wake-up signal sent by the user to attempt to wake up the preset voice assistant.

[0105] S202: Determine whether the voice wake-up signal matches the wake-up word of the preset voice assistant; if so, execute S203.

[0106] The wake-up word of the preset voice assistant is a specific word or phrase used to wake up the preset voice assistant. The wake-up word can be customized by the user, for example, the wake-up signal can be "Xiaobao Xiaobao", "Xiaobei Xiaobei", etc.

[0107] In some embodiments, the wake-up word of the preset voice assistant may be one or more, and different wake-up words may wake up different digital people, where the digital people refer to virtual assistants or virtual agents created based on artificial intelligence technology in the preset voice assistant. Different wake-up words may wake up different digital characters, indicating that each wake-up word corresponds to a specific virtual assistant or function, and these assistants may perform different tasks or provide different services.

[0108] S203: Determine the state of the preset voice assistant as a wake-up state.

[0109] In some embodiments, when the above-mentioned voice wake-up signal does not match the wake-up word of the preset voice assistant, the preset voice assistant remains in an unaware state.

[0110] S204: When the preset voice assistant is in the wake-up state, detecting whether the user issues the voice command information.

[0111] Further, after detecting that the user issues a voice command message, the above-mentioned step S21 of acquiring the voice command message issued by the user is executed.

[0112] In some embodiments, the wake-up function of the preset voice assistant can be implemented by a speech recognition algorithm run by a central processing unit (CPU) of the first terminal device. The CPU can continuously detect the audio input signal from the microphone and detect a specific wake-up word or command word by running a lightweight speech recognition algorithm.

[0113] In other embodiments, the wake-up function of the preset voice assistant can also be implemented through a related voice chip, which is a hardware specially designed to process voice signals, and has a built-in low-power voice recognition algorithm for detecting specific wake-up words. The voice chip can continuously detect voice input in an extremely low-power state, and only when the voice chip detects the wake-up word will it wake up the main processor (CPU) and enter the full-speed working state to handle the subsequent complex tasks (such as natural language understanding and answer generation).

[0114] In an embodiment of the present application, when the preset smart assistant is in an unaware state, there is no need for continuous high-intensity voice processing and analysis, and only the voice wake-up signal needs to be detected, which reduces unnecessary resource consumption and saves computing and battery energy consumption; in the unaware state, the preset voice assistant does not perform recognition or processing of voice command information, and therefore does not record or analyze the user's daily conversations. Only when the voice wake-up signal is detected to match the preset wake-up word, it enters the awakening state and starts further operations, which helps to protect user privacy and avoid the risk of unintentional recording or data leakage; in addition, by supporting multiple wake-up words, a more flexible configuration can be achieved to adapt to the preferences and language habits of different users. This diversified wake-up word configuration also helps to personalize the user experience.

[0115] In some embodiments, before executing S23 and determining the target skill type corresponding to the text information based on the text information, the text information can also be sent to a cloud device, so that the cloud device can analyze the text information through natural language processing and semantic analysis, anomaly detection, emotion and tone analysis, voiceprint recognition and identity authentication to obtain analysis results. If the analysis result indicates that the text information has any one or more of the following situations, it is determined that the text information needs to be intervened: the text information is ambiguous, the text information is incomplete, the text information involves sensitive words, etc. Intervention processing includes: supplementing or modifying the processing of user requests, security and privacy protection, error correction, personalized adjustment, etc.

[0116] Through intelligent judgment and timely intervention, we can more accurately understand and meet user needs, reduce misunderstandings and errors, and prevent potential security risks and privacy leaks, as well as adjust responses and enhance user experience based on users' historical behavior, preferences and current situations.

[0117] In some embodiments, in S23, the target skill type corresponding to the above text information is determined based on the above text information, including: identifying the above text information through a preset intention model to obtain the target skill type corresponding to the above text information, and the above target skill type is any one of the following: cloud skill type, local skill type, question and answer skill type.

[0118] In some embodiments, the preset intent model may be a pre-trained intent model, which is used to analyze the input text information to understand the user's intent, and then select the corresponding target skill type based on the intent to perform the corresponding operation or provide an answer using the target skill type. Specifically, the preset intent model may use natural language processing techniques, such as word vectors, sentence embedding, named entity recognition, context analysis, etc., to identify the user's intent expressed in the text information.

[0119] In some embodiments, each skill type may correspond to one or more intent categories. For example, the intent categories corresponding to the cloud skill type include: weather services, music story resources, control and manipulation of other devices, etc. The intent categories corresponding to the local skill type include: ordinary control instructions (such as shutting down, turning on, taking pictures, recording videos, calling family members, playing nursery rhymes, adjusting night vision models, etc.) and complex control instructions (such as turning on artificial intelligence skills, setting alarms, notifying relevant users when the baby leaves the care area, etc.). The intent categories corresponding to the question and answer skill type include text questions and answers (such as calling related digital people) and multimodal questions and answers (such as cameras capturing pictures and asking questions, such as "Do I look good today?", voice recording family faces), etc.

[0120] For example, the text information may be "What's the weather like today?". By inputting the text information into the preset intent model, the intent category of "weather service" output by the preset intent model can be obtained. The target skill model corresponding to this intent type is the cloud skill type.

[0121] In the embodiment of the present application, by identifying the voice command information input by the user, it is possible to more accurately match the appropriate skill type, thereby providing a response that better meets the user's needs, improving user satisfaction, and more reasonably allocating and utilizing resources, reducing unnecessary expenses.

[0122] In some embodiments, when the target skill type is the local skill type, the target processing model corresponding to the target skill type is the local skill processing model; in S24, the text information is analyzed by the target processing model corresponding to the target skill type to obtain a processing result corresponding to the text information, including S241-S242:

[0123] S241. Analyze the text information through the local skill processing model of the first terminal device to obtain a first operation instruction corresponding to the text information.

[0124] In some embodiments, the local skill processing model may be a model or algorithm that runs locally on the first terminal device and is used to generate specific operation instructions based on text information.

[0125] S242: Execute the first operation instruction to obtain a processing result corresponding to the text information.

[0126] In some embodiments, the processing result corresponding to the above text information is the result generated by executing the first operation instruction corresponding to the above text information.

[0127] For example, the text message is "Turn on Bluetooth". The local skill processing model first performs keyword recognition to identify the keywords in the text message, such as "turn on" and "Bluetooth"; then performs context analysis to determine that the user's intention is "turn on the device function"; and maps the identified intention and keywords to the specific operation of the first terminal device, and obtains that the first operation instruction corresponding to the "Turn on Bluetooth" text message can be a specific function call inside the first terminal device, and then executes the operation of turning on the Bluetooth module, and feeds back a voice message such as "Bluetooth is turned on." to the user.

[0128] In the embodiment of the present application, the local skill processing model is executed locally on the first terminal device, does not need to rely on a network connection, has a faster processing speed, a more timely response, and better protects user privacy.

[0129] In some embodiments, when the target skill type is the cloud skill type, the target processing model corresponding to the target skill type is the cloud skill processing model. In S24, the text information is analyzed by the target processing model corresponding to the target skill type to obtain the processing result corresponding to the text information, including S243-S245:

[0130] S243. Generate corresponding request information based on the above text information.

[0131] In some embodiments, the text information and the device identification of the first terminal device may be packaged to generate the request information.

[0132] S244. Send the request information to the cloud device, so that the cloud device analyzes the text information included in the request information through the cloud skill processing model to obtain the processing result of the text information.

[0133] Among them, the cloud skill processing model can be various artificial intelligence and machine learning models or algorithms deployed in the cloud (i.e., the background server of the preset voice assistant), which is used to process and analyze data or requests sent from the client (such as the first terminal device).

[0134] In some embodiments, after receiving the request information, the cloud device can use the cloud skill processing model to analyze the text information in the request information. The cloud skill processing model can use natural language processing, text analysis, machine learning or other artificial intelligence related algorithms to analyze the text information, obtain the intent of the text information, and then call the corresponding service to output the processing result of the text information. The processing result of the text information can be the classification, sentiment analysis, keyword extraction, etc. of the text information.

[0135] S245: Receive the processing result corresponding to the above text information fed back by the above cloud device.

[0136] For example, the text message may be "What's the weather like today?" The cloud skill processing model may analyze and process the text message based on the intent category "weather service". For example, it may connect to the weather service and use the keywords "today" and "weather" to query the local (the specific location can be pre-set) weather conditions for the current date, and generate a natural language answer such as "Today's weather is sunny, and the temperature is between 25 and 30 degrees." Then, the cloud device returns the generated answer to the first terminal device and plays it, and the result heard by the user may be the voice content of "Today's weather is sunny, and the temperature is between 25 and 30 degrees."

[0137] In the embodiment of the present application, by sending text information to a cloud device for processing, using advanced artificial intelligence and machine learning models deployed in the cloud, complex text analysis tasks can be processed more efficiently. Compared with computing on local devices, cloud devices have more powerful computing power and updated algorithm libraries, can more accurately understand user intentions, and provide more precise answers or services. In addition, the first terminal device (such as a smart camera or smart speaker, etc.) usually has limited hardware resources. Transferring the text information processing task to the cloud can significantly reduce the computing burden and energy consumption of the local device, so that the first terminal device can still provide high-quality services under low power consumption and small memory conditions. And the cloud skill processing model can be more easily updated and optimized. For example, improvements or new features of artificial intelligence and machine learning models can be quickly deployed in the cloud without the need to update each terminal device separately, which makes it possible to adapt to changes in user needs more quickly and provide better services.

[0138] In some embodiments, the request information is used to instruct the second terminal device to execute the second operation instruction. In this case, the processing result corresponding to the text information is prompt information, and the prompt information is used to indicate whether the second terminal device succeeds or fails in executing the second operation instruction.

[0139] In some embodiments, the second terminal device may be a smart camera, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart home appliance, etc., but is not limited thereto. The second terminal device may be one or more.

[0140] In some embodiments, the text information, the device identification of the first terminal device and the device identification of the second terminal device may be packaged to generate the request information.

[0141] After receiving the request information, the above-mentioned cloud device can analyze the text information included in the above-mentioned request information through the above-mentioned cloud skill processing model to obtain the intention of the text information, and generate a second operation instruction according to the intention of the text information, and send the second operation instruction to the corresponding second terminal device to control the second terminal device to execute the second operation instruction, and generate a processing result corresponding to the text information, and feed back the processing result corresponding to the text information to the cloud device, and the cloud device forwards the processing result corresponding to the text information to the first terminal device.

[0142] Figure 4 A scenario diagram of a voice interaction system provided by an exemplary embodiment of the present application is shown in FIG. Figure 4, in the actual use scenario, it includes a first terminal device 401 and at least one second terminal device 402, both of which are smart cameras. The user can send a voice command message to the first terminal device 401, and the text message corresponding to the voice command message can be "turn off the second terminal device". Then the cloud device 403 generates a second operation instruction "control the second terminal device to turn off" after analyzing and processing the text message, and sends the second operation instruction to the corresponding second terminal device 402 to control the second terminal device 402 to execute the second operation instruction, and generates a processing result corresponding to the text information, and feeds back the processing result corresponding to the text information to the cloud device 403, and the cloud device 403 forwards the processing result corresponding to the text information to the first terminal device 401. The processing result corresponding to the text information is a prompt message, and the prompt message is used to indicate whether the second terminal device 402 succeeds or fails in executing the second operation instruction, such as: "The second terminal device is turned off successfully".

[0143] In the embodiment of the present application, one device (first terminal device) controls the operation of another device (second terminal device) and feeds back the execution result without the user having to manually operate each device. This method significantly simplifies the process of multi-device control, improves the convenience and operation experience of users, and can realize the collaborative work and centralized control between different types of devices, enhancing the overall intelligence level of smart home or office environment.

[0144] In some embodiments, when the target skill type is the question-and-answer skill type, the target processing model corresponding to the target skill type is a question-and-answer skill processing model, and the processing result corresponding to the text information is playing the answer text corresponding to the text information.

[0145] In some embodiments, when the target skill type is the question-and-answer skill type, in S24, the target processing model corresponding to the target skill type is used to analyze the text information to obtain a processing result corresponding to the text information, including S246-S247:

[0146] S246. Analyze the above text information through the above question-answering skill processing model to generate an answer text corresponding to the above text information.

[0147] S247. Play the above answer text.

[0148] Among them, the question-answering skill processing model is an algorithm model based on natural language processing and machine learning, which can be used to understand the user's questions (text information) and generate corresponding answers. The processing model can retrieve the most relevant answers from a predefined document collection (such as a database, knowledge base), and generate a new answer text based on the user's question through natural language generation technology. For example, the question-answering skill processing model can find the answer that best matches the question corresponding to the text information in the background knowledge base, or the question-answering skill processing model will use deep learning technology to generate a natural language answer.

[0149] The question-answering skill processing model can also be a multimodal question-answering skill processing model. For example, when the text message is "Do I look good in what I wear today?", the question-answering skill processing model will first analyze the text message. The analysis shows that the user's question intention is "evaluate the user's clothes", and then the camera can be turned on to track the user and collect images, and the collected images can be identified and analyzed to generate answer text. For example, it can be identified that the user is wearing red clothes, and then the answer text can be "You look very good today, and the red clothes set off your complexion very well." The generated answer text will be presented to the user in the form of voice and / or text.

[0150] In the embodiment of the present application, the question-answering skill processing model can generate targeted answers based on the user's specific questions, especially through multimodal processing (such as text combined with image analysis), to provide users with more personalized and accurate feedback. For example, when a user asks "Does my outfit look good today?", the processing model not only answers based on a fixed answer library, but also generates a more natural and closer answer based on the user's current clothing situation, which significantly improves the intelligence and personalization of the user experience.

[0151] In some embodiments, when the target skill type is the question-and-answer skill type, before executing S24, analyzing the text information by the target processing model corresponding to the target skill type to obtain the processing result corresponding to the text information, the voice instruction may be analyzed to determine whether the user is a minor. When determining that the user is a minor, inappropriate content may be filtered out during question-and-answering.

[0152] Optionally, the user's voice command information can be analyzed in any one or more of the following ways to determine whether the user is a minor: voiceprint analysis, inferring the user's age group by analyzing the frequency, intonation, timbre and other characteristics of the voice; matching with a preset voiceprint recognition database, for example, pre-setting a family member's voiceprint library, which can recognize and classify the voices of specific family members. When the voice of a minor is detected, it can automatically switch to child mode or perform corresponding content filtering; language and intonation analysis, analyzing the user's language usage and intonation characteristics to determine the age group; topic and content recognition, analyzing the keywords and topic content in the user's voice command information. For example, if the user frequently mentions content related to school, homework, toys, cartoons, etc., it can be inferred that the user is a minor. Minors; Behavioral pattern analysis, which can infer age groups based on user usage behaviors and patterns. For example, underage users may use voice assistants more frequently during specific time periods (such as after school or before dinner) and tend to access educational or entertainment content; Multi-round conversation analysis: Determine the user's age through multi-round conversations and context analysis. For example, in multi-round conversations, you can set some questioning strategies to guide users to provide age-related information (such as favorite games, cartoon characters, etc.), and then determine the user's age level through semantic analysis; Family account and user configuration: The preset voice assistant allows family members to use the same first terminal device and determine the user's age based on login information or user registration information. Parents can set up a sub-account through a family account and specify the account as child mode.

[0153] In the embodiment of the present application, through the comprehensive use of voiceprint analysis, voice content analysis, behavior pattern analysis and registration information management, the preset voice assistant can determine whether the user is a minor, and take corresponding measures to ensure the age appropriateness, safety and compliance of the content, thereby effectively improving the reliability of voice interaction.

[0154] Figure 5 A specific flow chart of a voice interaction method provided by an exemplary embodiment of the present application, Figure 5 As shown, the voice interaction method includes S501-S09:

[0155] S501: Receive a voice wake-up signal sent by a user.

[0156] S502: Determine whether the voice wake-up signal matches the wake-up word of the preset voice assistant. If so, execute S503; if not, return to execute S501.

[0157] S503: Determine the state of the preset voice assistant as a wake-up state.

[0158] S504: Detect whether the user issues a voice command.

[0159] S505: When it is detected that the user issues a voice command, the voice command information issued by the user is obtained.

[0160] S506: Recognize and process the voice command information to obtain text information corresponding to the voice command information.

[0161] S507: Identify the text information through a preset intent model, and obtain a target skill type corresponding to the text information as a local skill model.

[0162] S508: Analyze the text information through the local skill processing model of the first terminal device to obtain a first operation instruction corresponding to the text information.

[0163] S509: Execute the first operation instruction to obtain a processing result corresponding to the text information.

[0164] The specific process of the voice interaction method has been described in the above content and will not be repeated here.

[0165] Please refer to the following Figure 6 , which is a structural diagram of a voice interaction device provided by an exemplary embodiment of the present application. Figure 6 As shown, the above-mentioned voice interaction device 600 includes:

[0166] The acquisition module 601 is used to acquire the voice command information issued by the user;

[0167] The processing module 602 is used to identify and process the voice command information to obtain text information corresponding to the voice command information;

[0168] A determination module 603 is used to determine the target skill type corresponding to the text information based on the text information;

[0169] The analysis module 604 is used to analyze the above text information through the target processing model corresponding to the above target skill type to obtain the processing result corresponding to the above text information.

[0170] In a possible implementation, the determining module 603 includes:

[0171] The recognition unit is used to recognize the above text information through a preset intent model to obtain a target skill type corresponding to the above text information, and the above target skill type is any one of the following: cloud skill type, local skill type, and question-and-answer skill type.

[0172] In a possible implementation, when the target skill type is the local skill type, the target processing model corresponding to the target skill type is a local skill processing model;

[0173] The above analysis module 604 includes:

[0174] A first analyzing unit, configured to analyze the text information by using the local skill processing model of the first terminal device to obtain a first operation instruction corresponding to the text information;

[0175] The execution unit is used to execute the first operation instruction to obtain the processing result corresponding to the text information.

[0176] In a possible implementation, when the target skill type is the cloud skill type, the target processing model corresponding to the target skill type is a cloud skill processing model;

[0177] The above analysis module 604 includes:

[0178] A generating unit, used to generate corresponding request information based on the above text information;

[0179] A sending unit, configured to send the request information to a cloud device, so that the cloud device analyzes the text information included in the request information through the cloud skill processing model to obtain a processing result of the text information;

[0180] The receiving unit is used to receive the processing result corresponding to the above text information fed back by the above cloud device.

[0181] In a possible implementation, the request information is used to instruct the second terminal device to execute the second operation instruction, and the processing result corresponding to the text information is prompt information, and the prompt information is used to indicate whether the second terminal device executes the second operation instruction successfully or fails.

[0182] In a possible implementation, when the target skill type is the question-and-answer skill type, the target processing model corresponding to the target skill type is a question-and-answer skill processing model, and the processing result corresponding to the text information is playing the answer text corresponding to the text information;

[0183] The above analysis module 604 includes:

[0184] A second analysis unit is used to analyze the text information through the question-answering skill processing model to generate an answer text corresponding to the text information;

[0185] The playback unit is used to play the above answer text.

[0186] In a possible implementation, the above device further includes:

[0187] A receiving module, used to receive a voice wake-up signal sent by the user;

[0188] A judgment module is used to judge whether the above-mentioned voice wake-up signal matches the wake-up word of the preset voice assistant;

[0189] The determination module 603 is further configured to determine the state of the preset voice assistant as a wake-up state when the voice wake-up signal matches the wake-up word of the preset voice assistant;

[0190] The detection module is used to detect whether the user issues the voice command information when the preset voice assistant is in the wake-up state.

[0191] The division of the modules in the above-mentioned voice interaction device 600 is only for illustration. In other embodiments, the voice interaction device can be divided into different modules as needed to complete all or part of the functions of the above-mentioned voice interaction device. The implementation of each module in the voice interaction device provided in the embodiment of this specification can be in the form of a computer program. The computer program can be run on a terminal or a server. The program modules constituted by the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, all or part of the steps of the voice interaction method described in the embodiment of this specification are implemented.

[0192] See next Figure 7 , which is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Figure 7 As shown, the electronic device 700 may include: a processor 710 and a memory 720 , and may also include a user interface 730 , a network interface 740 and a communication bus 750 .

[0193] Among them, the processor 710 may include one or more processing cores. The processor 710 uses various interfaces and lines to connect various parts of the entire electronic device 700, and executes various functions and processes data of the electronic device 700 by running or executing instructions, programs, code sets or instruction sets stored in the memory 720, and calling data stored in the memory 720. Optionally, the processor 710 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 710 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes operating systems and applications, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 710, and it can be implemented separately through a chip.

[0194] Among them, the memory 720 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 720 includes a non-transitory computer-readable storage medium. The memory 720 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a receiving function, a control function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 720 may also be optionally at least one storage device located away from the aforementioned processor 710. As Figure 7 As shown, the memory 720 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0195] Optionally, the communication bus 750 is used to realize the connection and communication between these components. The user interface 730 may include a display screen (Display), a camera (Camera), and may also include a standard wired interface and a wireless interface; the network interface 740 may optionally include a standard wired interface and a wireless interface (such as a WIFI interface).

[0196] exist Figure 7 In the electronic device 700 shown, the processor 710 may be used to call program instructions stored in the memory 720 and specifically perform the following operations:

[0197] Obtain the voice command information issued by the user;

[0198] Recognize and process the voice command information to obtain text information corresponding to the voice command information;

[0199] Determine the target skill type corresponding to the text information based on the text information;

[0200] The above text information is analyzed by the target processing model corresponding to the above target skill type to obtain the processing result corresponding to the above text information.

[0201] In a possible implementation, determining the target skill type corresponding to the text information based on the text information includes:

[0202] The above text information is identified through a preset intent model to obtain a target skill type corresponding to the above text information, and the above target skill type is any one of the following: cloud skill type, local skill type, and question-and-answer skill type.

[0203] In a possible implementation, when the target skill type is the local skill type, the target processing model corresponding to the target skill type is a local skill processing model;

[0204] The above-mentioned text information is analyzed by the target processing model corresponding to the above-mentioned target skill type to obtain the processing result corresponding to the above-mentioned text information, including:

[0205] Analyze the text information by using the local skill processing model of the first terminal device to obtain a first operation instruction corresponding to the text information;

[0206] Execute the first operation instruction to obtain the processing result corresponding to the text information.

[0207] In a possible implementation, when the target skill type is the cloud skill type, the target processing model corresponding to the target skill type is a cloud skill processing model;

[0208] The above-mentioned text information is analyzed by the target processing model corresponding to the above-mentioned target skill type to obtain the processing result corresponding to the above-mentioned text information, including:

[0209] Generate corresponding request information based on the above text information;

[0210] Sending the request information to a cloud device, so that the cloud device analyzes the text information included in the request information through the cloud skill processing model to obtain a processing result of the text information;

[0211] Receive the processing result corresponding to the above text information fed back by the above cloud device.

[0212] In a possible implementation, the request information is used to instruct the second terminal device to execute the second operation instruction, and the processing result corresponding to the text information is prompt information, and the prompt information is used to indicate whether the second terminal device executes the second operation instruction successfully or fails.

[0213] In a possible implementation, when the target skill type is the question-and-answer skill type, the target processing model corresponding to the target skill type is a question-and-answer skill processing model, and the processing result corresponding to the text information is playing the answer text corresponding to the text information;

[0214] The above-mentioned text information is analyzed by the target processing model corresponding to the above-mentioned target skill type to obtain the processing result corresponding to the above-mentioned text information, including:

[0215] Analyze the text information through the question-answering skill processing model to generate an answer text corresponding to the text information;

[0216] Play the above answer text.

[0217] In a possible implementation, before obtaining the voice command information issued by the user, the method further includes:

[0218] Receiving a voice wake-up signal sent by the user;

[0219] Determine whether the voice wake-up signal matches the wake-up word of the preset voice assistant;

[0220] If yes, the state of the preset voice assistant is determined as the awake state;

[0221] When the state of the preset voice assistant is the wake-up state, detect whether the user issues the voice command information.

[0222] The embodiment of the present application also provides a computer-readable storage medium, which stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes one or more steps in the above embodiment. If the components of the above voice interaction device are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium.

[0223] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The above-mentioned computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, the above-mentioned process or function according to the embodiment of the present application is generated in whole or in part. The above-mentioned computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium or transmitted by the above-mentioned computer-readable storage medium. The above-mentioned computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The above-mentioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVD)), or semiconductor media (for example, solid state disks (SSD)), etc.

[0224] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes. In the absence of conflict, the technical features in this embodiment and the implementation scheme can be combined arbitrarily.

[0225] The above-mentioned embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the protection scope determined by the claims of the present application.

Claims

1. A voice interaction method, characterized in that: Applied to the first terminal device, including: Obtain the voice command information issued by the user; Recognize and process the voice command information to obtain text information corresponding to the voice command information; Determining a target skill type corresponding to the text information based on the text information; The text information is analyzed by a target processing model corresponding to the target skill type to obtain a processing result corresponding to the text information.

2. The method according to claim 1, characterized in that The determining, based on the text information, the target skill type corresponding to the text information includes: The text information is identified through a preset intent model to obtain a target skill type corresponding to the text information, where the target skill type is any one of the following: a cloud skill type, a local skill type, and a question-and-answer skill type.

3. The method according to claim 2, characterized in that In the case where the target skill type is the local skill type, the target processing model corresponding to the target skill type is the local skill processing model; The analyzing the text information by using the target processing model corresponding to the target skill type to obtain the processing result corresponding to the text information includes: Analyze the text information by using the local skill processing model of the first terminal device to obtain a first operation instruction corresponding to the text information; Execute the first operation instruction to obtain a processing result corresponding to the text information.

4. The method according to claim 2, characterized in that In the case where the target skill type is the cloud skill type, the target processing model corresponding to the target skill type is the cloud skill processing model; The analyzing the text information by using the target processing model corresponding to the target skill type to obtain the processing result corresponding to the text information includes: Generate corresponding request information based on the text information; Sending the request information to a cloud device, so that the cloud device analyzes the text information included in the request information through the cloud skill processing model to obtain a processing result of the text information; Receive the processing result corresponding to the text information fed back by the cloud device.

5. The method according to claim 4, characterized in that The request information is used to instruct the second terminal device to execute the second operation instruction. The processing result corresponding to the text information is prompt information, and the prompt information is used to indicate whether the second terminal device succeeds or fails in executing the second operation instruction.

6. The method according to claim 2, characterized in that In the case where the target skill type is the question-and-answer skill type, the target processing model corresponding to the target skill type is the question-and-answer skill processing model, and the processing result corresponding to the text information is playing the answer text corresponding to the text information; The analyzing the text information by using the target processing model corresponding to the target skill type to obtain the processing result corresponding to the text information includes: Analyze the text information through the question-answering skill processing model to generate an answer text corresponding to the text information; The answer text is played.

7. The method according to claim 1, characterized in that Before acquiring the voice command information issued by the user, the method further includes: Receiving a voice wake-up signal sent by the user; Determining whether the voice wake-up signal matches the wake-up word of the preset voice assistant; If yes, determining the state of the preset voice assistant as a wake-up state; When the state of the preset voice assistant is the awake state, detecting whether the user issues the voice command information.

8. A voice interaction device, characterized in that: include: An acquisition module is used to acquire the voice command information issued by the user; A processing module, used to identify and process the voice command information to obtain text information corresponding to the voice command information; A determination module, configured to determine a target skill type corresponding to the text information based on the text information; The analysis module is used to analyze the text information through the target processing model corresponding to the target skill type to obtain the processing result corresponding to the text information.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method according to any one of claims 1 to 7.

10. A computer storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the steps of the method according to any one of claims 1 to 7.

Images