Voice control method and apparatus of device, storage medium, and electronic device

Abstract

The application discloses a voice control method and device of equipment, a storage medium and an electronic device, and relates to the technical field of smart home / smart home, wherein the method comprises the following steps: extracting the acoustic characteristics of a first voice signal to obtain first acoustic characteristics, wherein the first voice signal is a voice signal collected by a voice device in the process of voice interaction; performing semantic analysis according to the first acoustic characteristics to obtain a target semantic intention corresponding to the first voice signal; in the case that the target semantic intention belongs to a group of preset intentions, performing voice emotion recognition according to the first acoustic characteristics to obtain a first emotion category corresponding to the first voice signal; fusing the first emotion category and the target semantic intention to obtain an updated target semantic intention, wherein the updated target semantic intention is a semantic intention with an emotion parameter; and controlling a controlled device to perform a device operation corresponding to the updated target semantic intention.

Description

Technical Field

[0001] This application relates to the field of smart home / smart family, and more specifically, to a voice control method and apparatus for a device, a storage medium, and an electronic device. Background Technology

[0002] Currently, voice devices can perform predetermined actions based on users' voice commands. For example, when a user requests the voice device to play a song, the voice device will play the specified song according to the user's instructions, or play songs from the playlist or song library according to the user's usage habits.

[0003] However, the voice control method of the aforementioned devices, which involves semantic parsing of user commands and controlling the device based on the parsing results, is simplistic and lacks flexibility. Long-term use of voice devices can lead to a dull user experience.

[0004] It is evident that the voice control methods for devices in related technologies suffer from poor device control flexibility due to the simplistic control approach. Summary of the Invention

[0005] This application provides a voice control method and apparatus for a device, a storage medium, and an electronic device to at least solve the problem of poor device control flexibility caused by the single control method in related technologies.

[0006] According to one aspect of the embodiments of this application, a voice control method for a device is provided, applied to a smart device, comprising: extracting acoustic features of a first speech signal to obtain first acoustic features, wherein the first speech signal is a speech signal collected by the voice device during voice interaction; performing semantic parsing based on the first acoustic features to obtain a target semantic intent corresponding to the first speech signal; if the target semantic intent belongs to a set of preset intents, performing voice emotion recognition based on the first acoustic features to obtain a first emotion category corresponding to the first speech signal; fusing the first emotion category with the target semantic intent to obtain an updated target semantic intent, wherein the updated target semantic intent is a semantic intent with emotion parameters; controlling a controlled device to perform a device operation corresponding to the updated target semantic intent, wherein the controlled device is a device matched with the updated target semantic intent.

[0007] In an exemplary embodiment, before controlling the controlled device to perform a device operation corresponding to the updated target semantic intent, the method further includes: determining operation execution parameters of the controlled device based on the first emotion category, wherein the operation execution parameters are device parameters used by the controlled device to perform the device operation.

[0008] In an exemplary embodiment, the step of extracting acoustic features from a first speech signal to obtain the first acoustic features includes: inputting the first speech signal into an acoustic feature extraction model without performing text parsing on the first speech signal, and obtaining the first acoustic features output by the acoustic feature extraction model, wherein the acoustic feature extraction model is a neural network model.

[0009] In an exemplary embodiment, the step of performing semantic parsing based on the first acoustic feature to obtain the target semantic intent corresponding to the first speech signal includes: inputting the first acoustic feature into a speech recognition model to obtain the target semantic intent, wherein the speech recognition model is used to identify the corresponding text vector feature based on the output acoustic feature, and perform semantic parsing based on the text vector feature to obtain the semantic intent corresponding to the output acoustic feature.

[0010] In an exemplary embodiment, the step of performing speech emotion recognition based on the first acoustic feature to obtain a first emotion category corresponding to the first speech signal when the target semantic intent belongs to a set of preset intents includes: when the target semantic intent belongs to the set of preset intents, inputting the first acoustic feature into a speech emotion prediction model to obtain a target emotion label output by the speech emotion prediction model, wherein the speech emotion prediction model is used to extract emotion features from the input acoustic features and predict an emotion label corresponding to the input acoustic features from a set of emotion labels based on the extracted emotion features, and the target emotion label corresponds to the first emotion category.

[0011] In an exemplary embodiment, fusing the first emotion category with the target semantic intent to obtain an updated target semantic intent includes: fusing the first emotion category as a modifier of the object in the target semantic intent into the target semantic intent to obtain an updated target semantic intent.

[0012] In an exemplary embodiment, after performing semantic parsing based on the first acoustic feature, the method further includes: if the target semantic intent does not belong to the set of preset intents, controlling the controlled device to perform a device operation corresponding to the target semantic intent.

[0013] In one exemplary embodiment, the method further includes: when the voice device is woken up by abnormal voice, extracting acoustic features of a second voice signal to obtain second acoustic features, wherein the second voice signal is a voice signal collected after the voice device is woken up by the abnormal voice; performing semantic parsing based on the second acoustic features to obtain a current dialogue scenario corresponding to the second voice signal; performing voice emotion recognition based on the second acoustic features to obtain a second emotion category corresponding to the second voice signal; and controlling the voice device to perform voice interaction with objects in the current dialogue scenario based on the current dialogue scenario and the second emotion category.

[0014] According to another aspect of the embodiments of this application, a voice control device for a smart device is also provided, comprising: a first extraction unit, configured to extract acoustic features of a first speech signal to obtain first acoustic features, wherein the first speech signal is a speech signal collected by the voice device during voice interaction; a first parsing unit, configured to perform semantic parsing based on the first acoustic features to obtain a target semantic intent corresponding to the first speech signal; a first recognition unit, configured to perform voice emotion recognition based on the first acoustic features when the target semantic intent belongs to a set of preset intents to obtain a first emotion category corresponding to the first speech signal; a fusion unit, configured to fuse the first emotion category with the target semantic intent to obtain an updated target semantic intent, wherein the updated target semantic intent is a semantic intent with emotion parameters; and a first control unit, configured to control a controlled device to perform a device operation corresponding to the updated target semantic intent, wherein the controlled device is a device matching the updated target semantic intent.

[0015] In one exemplary embodiment, the apparatus further includes: a determining unit, configured to determine, based on the first emotion category, operation execution parameters of the controlled device before the controlled device performs a device operation corresponding to the updated target semantic intent, wherein the operation execution parameters are device parameters used by the controlled device to perform the device operation.

[0016] In an exemplary embodiment, the first extraction unit includes: a first input module, configured to input the first speech signal into an acoustic feature extraction model without performing text parsing on the first speech signal, to obtain the first acoustic feature output by the acoustic feature extraction model, wherein the acoustic feature extraction model is a neural network model.

[0017] In an exemplary embodiment, the first parsing unit includes: a second input module, configured to input the first acoustic feature into a speech recognition model to obtain the target semantic intent, wherein the speech recognition model is configured to identify the corresponding text vector feature based on the output acoustic feature, and perform semantic parsing based on the text vector feature to obtain the semantic intent corresponding to the output acoustic feature.

[0018] In an exemplary embodiment, the first recognition unit includes: a third input module, configured to input the first acoustic feature into a speech emotion prediction model when the target semantic intent belongs to the set of preset intents, to obtain a target emotion label output by the speech emotion prediction model, wherein the speech emotion prediction model is configured to extract emotion features from the input acoustic features, and predict an emotion label corresponding to the input acoustic features from a set of emotion labels based on the extracted emotion features, and the target emotion label corresponds to the first emotion category.

[0019] In an exemplary embodiment, the fusion unit includes a fusion module, configured to fuse the first emotion category as a modifier of the object in the target semantic intent into the target semantic intent, thereby obtaining an updated target semantic intent.

[0020] In one exemplary embodiment, the apparatus further includes: a second control unit, configured to, after semantic parsing based on the first acoustic feature, control the controlled device to perform a device operation corresponding to the target semantic intent when the target semantic intent does not belong to the set of preset intents.

[0021] In one exemplary embodiment, the apparatus further includes: a second extraction unit, configured to extract acoustic features of a second speech signal to obtain second acoustic features when the voice device is awakened by abnormal speech, wherein the second speech signal is a speech signal collected after the voice device is awakened by the abnormal speech; a second parsing unit, configured to perform semantic parsing based on the second acoustic features to obtain a current dialogue scenario corresponding to the second speech signal; a second recognition unit, configured to perform speech emotion recognition based on the second acoustic features to obtain a second emotion category corresponding to the second speech signal; and a third control unit, configured to control the voice device to perform voice interaction with objects in the current dialogue scenario based on the current dialogue scenario and the second emotion category.

[0022] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the voice control method of the above-described device when it is run.

[0023] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the voice control method of the device through the computer program.

[0024] In this embodiment, a method is adopted to control the corresponding device to perform device operations by combining the emotion category and semantic intent of the speech signal. First acoustic features are obtained by extracting the acoustic features of the first speech signal, where the first speech signal is the speech signal collected by the voice device during voice interaction. Semantic parsing is performed based on the first acoustic features to obtain the target semantic intent corresponding to the first speech signal. If the target semantic intent belongs to a set of preset intents, speech emotion recognition is performed based on the first acoustic features to obtain the first emotion category corresponding to the first speech signal. The first emotion category is fused with the target semantic intent to obtain an updated target semantic intent, where the updated target semantic intent is a semantic intent with emotion parameters. The controlled device is then controlled to perform device operations corresponding to the updated target semantic intent, where the controlled device is the device that performs the operation corresponding to the updated target semantic intent. The target semantic intent matching device obtains the semantic intent corresponding to the speech signal by performing semantic analysis on the acoustic features of the speech signal. Based on the semantic intent, it determines whether it is necessary to combine the user's emotion to generate device operation. If it is necessary to combine the user's emotion to generate device operation, it performs speech emotion recognition on the acoustic features of the speech signal to obtain the emotion category corresponding to the speech signal. It generates a semantic intent with emotion parameters by combining the semantic intent of the speech signal and the emotion category. Based on the semantic intent with emotion parameters, it determines the device to perform the operation and the corresponding device operation, and controls the corresponding device to perform the corresponding device operation. This can achieve the purpose of generating device operation based on user emotion, thereby improving the technical effect of improving the flexibility of device control. This solves the problem of poor device control flexibility caused by the single control method in the voice control method of related technologies. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1This is a schematic diagram of the hardware environment of a voice control method for a device according to an embodiment of this application;

[0028] Figure 2 This is a flowchart illustrating an optional voice control method for a device according to an embodiment of this application.

[0029] Figure 3 This is a schematic diagram of an optional voice control method for a device according to an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of another optional voice control method for a device according to an embodiment of this application;

[0031] Figure 5 This is a schematic diagram of another optional voice control method for a device according to an embodiment of this application;

[0032] Figure 6 This is a structural block diagram of a voice control device for an optional device according to an embodiment of this application;

[0033] Figure 7 This is a structural block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] According to one aspect of the embodiments of this application, a voice control method for a device is provided. This voice control method is widely used in whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned voice control method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0037] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0038] The voice control method of the device in this embodiment can be executed by server 104, terminal device 102, or jointly by server 104 and terminal device 102. Alternatively, the voice control method of the device in this embodiment can be executed by a client installed on the terminal device 102.

[0039] Taking the voice control method of the device in this embodiment as an example, which is executed by a voice device (an example of terminal device 102), Figure 2 This is a flowchart illustrating an optional voice control method for a device according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this method may include the following steps:

[0040] Step S202: Extract the acoustic features of the first speech signal to obtain the first acoustic features, wherein the first speech signal for voice control of the device is the speech signal collected by the voice device during voice interaction.

[0041] The voice control method for the device in this embodiment can be applied to scenarios where the device is controlled based on voice signals collected by the voice device. The voice device (i.e., the voice-enabled device) can be an intelligent voice device that can respond to and interact with the user during voice interaction, collect voice data, and control the corresponding device to perform corresponding device operations or respond to the user's interactive voice to meet the user's needs. The device type of the voice device can be a smart home device, such as a smart speaker, a smart washing machine, or a smart air conditioner.

[0042] Voice devices can collect the voice signals emitted by users in real time during voice interaction, perform voice recognition on the voice signals, process the voice recognition results through NLP (Natural Language Processing) technology, and obtain the execution actions that respond to and adapt to the user's voice commands. The execution actions will be fed back to the voice network device to make responses to the user's voice commands.

[0043] For example, when a user says "Open the window for me," the voice device can determine whether there are curtains in the room and how many curtains can be controlled, responding and interacting with the user to ultimately fulfill their command. Furthermore, the voice-enabled device has a companion function; for instance, when a user needs to play music, control ambient lighting, or tell a story, the voice device can directly parse the user's semantics, that is, to a certain extent expand and understand the literal meaning of the user's command, responding and executing it on the device side.

[0044] Most current smart home voice-controlled devices only support predetermined actions based on the speaker's voice commands. That is, they only perform NLP parsing and device control based on the literal meaning of the user's instructions. For example, when a user asks the voice-controlled device to play a song, it will play the specified song or a song from the playlist according to the user's explicit needs or usage habits. This level of smart home companionship cannot bring users real pleasure or connect with their emotions. Over time, users will feel bored with the intelligence of the voice-controlled device, and the interaction with the voice-controlled device cannot fully meet users' growing requirements for the richness of AI (Artificial Intelligence) voice-controlled devices.

[0045] To at least partially address the aforementioned issues, this embodiment extracts acoustic features from the speech signal. Semantic analysis of these features yields the corresponding semantic intent. Based on this intent, it is determined whether to integrate the user's emotional data generation device operation. If integration is necessary, speech emotion recognition is performed on the acoustic features to obtain the corresponding emotion category. The semantic intent and emotion category are combined to generate a semantic intent with emotional parameters, controlling the corresponding device to perform the appropriate operation. This approach overcomes the shortcomings of traditional single-dimensional analysis from user speech to literal meaning, which lacks integration of user emotional information. By introducing user emotional dimension information and using user emotion as a crucial information dimension for multi-dimensional information fusion, the user command execution generation model can better reflect the user's genuine feelings, providing deeper and more engaging feedback on user needs. This significantly enhances the user experience of smart home companion products and increases user stickiness with smart voice devices.

[0046] During voice interaction with a user, a voice device can acquire a voice signal to obtain a first voice signal. By extracting acoustic features from the first voice signal, the first acoustic features of the first voice signal can be obtained. Acoustic feature extraction of the first voice signal can be performed by the voice device itself or by calling an acoustic feature extraction model in the cloud.

[0047] Optionally, the implementation scenarios of the above method may include multimodal intelligent voice network devices with sound reception and broadcasting functions. The core module of the multimodal voice network device that combines user emotion recognition to achieve companionship may include, but is not limited to, a voice emotion feature extraction model (i.e., a voice emotion model or a user emotion recognition model), a voice acoustic feature extraction model, a voice recognition model, and a voice parsing companionship model that combines emotion features and voice recognition. The voice signal can be processed by the corresponding model in the voice network device, or the voice signal can be processed by calling the corresponding model in the cloud.

[0048] Step S204: Perform semantic parsing based on the first acoustic features to obtain the target semantic intent corresponding to the first speech signal.

[0049] By performing semantic parsing on the first acoustic features, the target semantic intent corresponding to the first speech signal can be obtained. Here, the target semantic intent is the user intent corresponding to the first speech signal. Semantic parsing of the first acoustic features can be performed by the voice device or by calling a semantic parsing model in the cloud.

[0050] Step S206: If the target semantic intent belongs to a set of preset intents, perform speech emotion recognition based on the first acoustic features to obtain the first emotion category corresponding to the first speech signal.

[0051] For some target semantic intents, the user intent they represent is clear, or the target voice intent already includes the emotional dimension of the need. Therefore, it is not necessary to combine the emotional category corresponding to the voice signal to determine the device operation that matches the target voice intent. So after obtaining the target semantic intent corresponding to the first voice signal, it is necessary to determine whether it is necessary to combine the emotional category corresponding to the first voice signal to generate the device operation.

[0052] In this embodiment, it can be determined whether device operation needs to be generated by combining the user's emotion and the target semantic intent contained in the voice signal by judging whether the target semantic intent belongs to a set of preset intents. Here, preset intents may include, but are not limited to, intents without a clear emotion category, intents that can be combined with emotion categories to determine device operation, and other intents that require combining emotion categories to determine device operation.

[0053] When the target semantic intent belongs to a set of preset intents, it can be determined that it is necessary to combine the user emotion generation device operation contained in the speech signal. Speech emotion recognition can be performed on the first acoustic features of the first speech signal to determine the emotion category corresponding to the first speech signal, thus obtaining the first emotion category. Here, the emotion category can be used to identify the type of user emotion corresponding to the first speech signal. The type of emotion category can be an emotion label obtained by classifying emotions, such as "joy", "sad", "happy", etc.

[0054] Step S208: The first sentiment category is fused with the target semantic intent to obtain the updated target semantic intent, wherein the updated target semantic intent is a semantic intent with sentiment parameters.

[0055] After obtaining the first emotion category and target semantic intent corresponding to the first speech signal, the first emotion category and target semantic intent can be fused together to update the target semantic intent. The updated target semantic intent is a semantic intent with emotion parameters, that is, a semantic intent that combines the user's emotional needs.

[0056] Optionally, in such Figure 3The multimodal voice network device with companionship capabilities, which combines user emotion recognition, is shown on the left. The left side represents the speaker emotion feature extraction model (i.e., the voice emotion prediction model). Its training labels are the emotion tags of the input audio, such as {'joy', 'sad', 'happy'}. The hidden layer information of this model is extracted as a feature vector representing the user's emotional information. This feature vector (embedding) is then merged with the text feature vectors from the speech recognition and semantic intent recognition results to form a new feature vector. This new feature vector is input into the right side's integrated emotion recognition speech semantic parsing model (i.e., the speech parsing companionship model). This generates semantic execution content with emotion parameters that conform to the user's emotional characteristics. This allows the integration of the voice emotion model into the semantic recognition model (i.e., the speech recognition model). By adding speech-based emotion recognition information to the semantic parsing and generation model, a user emotion-based voice semantic parsing model is obtained, thereby improving the personalized AI voice companionship and interaction capabilities of the user end. In the speech semantic analysis model, by incorporating speaker emotion discrimination information, the voice-enabled smart device can respond most appropriately based on the user's different emotions. It provides warm comfort and encouragement when the user is frustrated or sad, and adds to the joy when the user is happy. This optimizes the intelligence of voice interaction in smart home products and the companionship capabilities of AI voice, making smart home products feel more like family members, full of humanistic care. For the interaction between a single user and a voice device, multimodal emotion recognition can combine the user's emotions to provide more considerate and intelligent services. Taking intelligent poetry generation as an example, when a user happily says, "The sun is shining brightly today, and the grass is sprouting. Write me a poem!", the speech analysis companionship model can combine the user's happy emotions to create poetry, and the voice-enabled smart device terminal can convert the creation into speech and play it back, thus allowing the user to receive a multimodal response that reflects their emotions.

[0057] Step S210: Control the controlled device to perform device operations corresponding to the updated target semantic intent, wherein the controlled device is the device that matches the updated target semantic intent.

[0058] After obtaining the updated target semantic intent, the device matching the updated target semantic intent can be determined, that is, the controlled device. Here, the controlled device is the device that executes the device operation corresponding to the updated target semantic intent. The controlled device can be a voice device or other smart devices. The device operation corresponding to the updated target semantic intent can be determined based on the device operation corresponding to the user instruction determined by the updated target semantic intent.

[0059] Through steps S202 to S210, acoustic features of the first speech signal are extracted to obtain the first acoustic features, wherein the first speech signal is the speech signal collected by the voice device during voice interaction; semantic parsing is performed based on the first acoustic features to obtain the target semantic intent corresponding to the first speech signal; if the target semantic intent belongs to a set of preset intents, voice emotion recognition is performed based on the first acoustic features to obtain the first emotion category corresponding to the first speech signal; the first emotion category is fused with the target semantic intent to obtain the updated target semantic intent, wherein the updated target semantic intent is a semantic intent with emotion parameters; the controlled device is controlled to perform device operations corresponding to the updated target semantic intent, wherein the controlled device is a device that matches the updated target semantic intent. This solves the problem of poor device control flexibility caused by the single control method in the voice control method of the device in the related technology, and improves the flexibility of device control.

[0060] In one exemplary embodiment, before controlling the controlled device to perform device operations corresponding to the updated target semantic intent, the method further includes:

[0061] S11, Based on the first emotion category, determine the operation execution parameters of the controlled device, wherein the operation execution parameters are the device parameters used by the controlled device to perform device operations.

[0062] Upon receiving the updated semantic intent, the voice device can determine the device operation and operation execution parameters of the controlled device based on the updated semantic intent and the first emotion category of the first voice signal. That is, the device parameters used by the controlled device to perform the device operation. Here, the operation execution parameters are device parameters that correspond to the first emotion category and meet the user's emotional needs.

[0063] For example, when a user gives commands such as "play a song", "control the ambient lights", or "tell a story", the corresponding device operation parameters can be determined based on their emotional category. That is, the type of song to play, the brightness and number of ambient lights to be adjusted, and the type of story to be told can be selected.

[0064] In this embodiment, by combining the emotion category of the voice signal to determine the operation execution parameters of the controlled device, the user's emotional needs can be met, thereby improving the flexibility of the device's voice control.

[0065] In one exemplary embodiment, the acoustic features of the first speech signal are extracted to obtain the first acoustic features, including:

[0066] S21, without performing text parsing on the first speech signal, the first speech signal is input into the acoustic feature extraction model to obtain the first acoustic feature output by the acoustic feature extraction model, wherein the acoustic feature extraction model is a neural network model.

[0067] When extracting the acoustic features of the first speech signal, it is not necessary to perform text parsing on the speech signal. The acoustic features of the speech signal can be directly extracted through the acoustic feature extraction model. The first speech signal is input into the acoustic feature extraction model to obtain the first acoustic features output by the acoustic feature extraction model. Here, the type of acoustic feature extraction model can be a neural network model.

[0068] For example, after receiving a user's voice query (voice command), the voice device can call the acoustic feature extraction model through the cloud, using the user's voice signal as the input to the acoustic feature extraction model. The acoustic feature extraction model can extract acoustic signal features through a pre-trained neural network model without performing text parsing on the user's voice signal.

[0069] Optionally, a large amount of speech data can be used as a training set, and the acoustic features corresponding to the speech data can be used as training labels to train the neural network model, so that the neural network model has the function of extracting acoustic features from speech signals, and a pre-trained neural network model can be obtained for extracting acoustic features from speech signals.

[0070] In this embodiment, by inputting the speech signal into the acoustic feature extraction model, the acoustic features of the speech signal can be obtained, which can improve the accuracy of speech signal processing.

[0071] In an exemplary embodiment, semantic parsing is performed based on the first acoustic feature to obtain the target semantic intent corresponding to the first speech signal, including:

[0072] S31, the first acoustic feature is input into the speech recognition model to obtain the target semantic intent. The speech recognition model is used to identify the corresponding text vector features based on the output acoustic features, and to perform semantic parsing based on the text vector features to obtain the semantic intent corresponding to the output acoustic features.

[0073] In this embodiment, semantic parsing of a speech signal can be achieved by inputting the acoustic features of the speech signal into a speech recognition model to obtain the semantic intent of the speech signal. The speech recognition model can identify the corresponding text vector features based on the input acoustic features, and output the semantic intent corresponding to the speech signal by parsing the text vector features.

[0074] The first acoustic feature of the first speech signal is input into the speech recognition model. The text vector feature corresponding to the first acoustic feature is identified through the first speech signal. The text vector feature corresponding to the first acoustic feature is then parsed to obtain the target semantic intent output by the speech recognition model.

[0075] For example, a voice-enabled device can call a pre-trained speech recognition model through the cloud, use the speech acoustic features of the speech signal as input to the speech recognition model, perform semantic parsing and intent recognition on the speech acoustic features, and obtain the text vector features of the user query and the user intent.

[0076] In this embodiment, by performing semantic analysis on the speech signal using a speech recognition model, the semantic intent corresponding to the speech signal can be obtained, thereby improving the reliability of the device's voice control.

[0077] In an exemplary embodiment, when the target semantic intent belongs to a set of preset intents, speech emotion recognition is performed based on first acoustic features to obtain a first emotion category corresponding to the first speech signal, including:

[0078] S41, when the target semantic intent belongs to a set of preset intents, the first acoustic feature is input into the speech emotion prediction model to obtain the target emotion label output by the speech emotion prediction model. The speech emotion prediction model is used to extract emotion features from the input acoustic features and predict the emotion label corresponding to the input acoustic features from a set of emotion labels based on the extracted emotion features. The target emotion label corresponds to the first emotion category.

[0079] When the target semantic intent corresponding to the first speech signal belongs to a set of preset intents, the first acoustic feature of the first speech signal can be input into the speech emotion prediction model. The speech emotion prediction model determines the target emotion label corresponding to the first speech signal. Here, the target emotion label corresponds to the first emotion category and can be a label used to identify the emotion category corresponding to the speech signal. The target emotion label can be an emotion label in a preset set of emotion labels. The preset set of emotion labels can include labels corresponding to some common emotion types, such as "joy", "sad", "happy", etc.

[0080] Speech emotion prediction models are used to predict the emotion category of speech signals. They can extract the emotion features corresponding to the speech signal from the acoustic features of the speech signal, and select the emotion label corresponding to the input acoustic features from a set of emotion labels based on the extracted emotion features.

[0081] For example, a device can call a speech emotion feature extraction model through the cloud, take the speech acoustic features as input, and finally obtain speech emotion tags, such as "joy", "sad", "happy", etc.

[0082] Speech sentiment prediction models can incorporate sentiment discrimination information, and their model structures are highly variable. They can be pre-trained wav2vec 2.0 models (an unsupervised speech pre-training model) or other sequence learning models. For example... Figure 4 As shown, the front end of the speech emotion feature extraction model is a pre-trained wav2vec 2.0 model. When performing emotion prediction, the output of the last layer is used as the emotion feature. The pre-trained wav2vec 2.0 model is based on an encoding and decoding model structure. Before using the speech emotion prediction model, the initial speech emotion prediction model can be pre-trained. The training set can be speech data, and the training labels can be the emotion labels corresponding to the speech data. The speech signal is input into the trained speech emotion prediction model to obtain the output of the last layer (hidden layer), that is, the emotion features of the speech signal, and the emotion label is matched according to the emotion features of the speech signal.

[0083] In this embodiment, by inputting the acoustic features of the speech signal into the speech emotion prediction model when the semantic intent of the speech signal belongs to a preset intent, the emotion label of the speech signal can be obtained. Based on the emotion label of the speech signal, device operation can be generated, which can improve the accuracy of the device's voice control method.

[0084] In one exemplary embodiment, fusing a first emotion category with a target semantic intent to obtain an updated target semantic intent includes:

[0085] S51, the first sentiment category is integrated into the target semantic intention as a modifier of the object, resulting in the updated target semantic intention.

[0086] In this embodiment, when fusing the first emotion category with the target semantic intent, the first emotion category is usually used as an adjective to modify, limit, or describe the object in the target semantic intent. Therefore, the first emotion category can be used as an adjective to the object in the target semantic intent and fused into the target semantic intent to obtain the updated semantic intent.

[0087] For example, if a user's emotional trait is "happy" and their intention is "play a song", the voice parsing companion model can use "happy" as a modifier of the object in "play a song" to obtain the final semantic execution content "play a song that brings a happy mood".

[0088] Optionally, the first emotion category and the target semantic intent can be fused through a voice parsing companion model. The first emotion category and the target semantic intent are input into the voice parsing companion model, and the first emotion category is used as the modifier of the object in the target semantic intent by the voice parsing companion model, so as to obtain the target semantic intent that has been fused with the first emotion category.

[0089] In this embodiment, by using emotion type as a modifier of the object in the target semantic intent, and by fusing emotion type and semantic intent, a semantic intent incorporating emotion features can be obtained, which can improve the reliability of voice devices.

[0090] In one exemplary embodiment, after semantic parsing based on the first acoustic feature, the method further includes:

[0091] S61, if the target semantic intent does not belong to a set of preset intents, control the controlled device to perform the device operation corresponding to the target semantic intent.

[0092] It should be noted that for user commands with clear user needs or clear emotional needs, after obtaining their semantic intent, the corresponding device operation can be directly executed based on the user's needs, without the need for emotion recognition of the voice signal. For example, when a user issues a voice command such as "play the first song in the playlist," the user has a clear need that is unrelated to the user's emotions, or when issuing a voice command such as "play a happy song," the user has a clear emotional attitude. It is not necessary to combine the user's emotional characteristics to generate device operations; the corresponding device operation can be generated directly based on the voice signal.

[0093] After performing speech parsing on the first speech signal based on the first acoustic features, if the target semantic intent of the first speech signal does not belong to a set of preset intents, the target semantic intent may be a semantic intent unrelated to the user's emotion, such as cooperation, or a semantic intent containing emotional words. It is not necessary to determine the device operation corresponding to the target semantic intent based on the emotion category of the first speech signal. That is, it is not necessary to perform speech emotion recognition on the speech signal. The controlled device can be controlled to perform the device operation corresponding to the target semantic intent based on the target semantic intent of the first speech signal. Controlling the controlled device to perform the device operation corresponding to the target semantic intent is similar to the above, and will not be elaborated here.

[0094] In this embodiment, by directly controlling the controlled device to perform device operations corresponding to the target semantic intent when the semantic intent of the voice signal does not belong to the preset intent, unnecessary emotion recognition can be reduced and the resource consumption of the voice device can be reduced.

[0095] In one exemplary embodiment, the above method further includes:

[0096] S71, when the voice device is woken up by abnormal voice, extract the acoustic features of the second voice signal to obtain the second acoustic features, wherein the second voice signal is the voice signal collected after the voice device is woken up by abnormal voice.

[0097] S72, perform semantic parsing based on the second acoustic features to obtain the current dialogue scenario corresponding to the second speech signal;

[0098] S73, based on the second acoustic features, perform speech emotion recognition to obtain the second emotion category corresponding to the second speech signal;

[0099] S74, based on the current dialogue scenario and the second emotion category, controls the voice device to perform voice interaction with objects in the current dialogue scenario.

[0100] In family settings, conflicts may arise among family members. If these conflicts are not addressed promptly, such as by intervening, mediating, or reconciling, they can escalate. Therefore, in this embodiment, when the voice device detects abnormal speech, it can engage in voice interaction with the user, taking into account the dialogue context and the user's emotions, to alleviate conflicts in multi-person scenarios.

[0101] In this embodiment, the abnormal voice can be a voice signal with contradictions and conflicts, such as arguing or scolding. After the voice device is awakened by the abnormal voice, it can collect the voice signal to obtain a second voice signal, and extract the acoustic features of the second voice signal to obtain a second acoustic feature. By analyzing the second acoustic feature, the dialogue scenario corresponding to the second voice signal can be determined, and the current dialogue scenario can be obtained.

[0102] By performing voice emotion recognition on the second emotion feature, the second emotion category corresponding to the second voice signal can be determined. The process of performing voice emotion recognition on the second emotion feature is similar to that described above and will not be repeated here. The voice device can then interact with objects in the current dialogue scenario based on the current dialogue context and the second emotion category.

[0103] In family settings, conflicts often arise among family members. When these conflicts become intense, a moment of persuasion or de-escalation is usually needed to ease tensions. For multi-person scenarios, multimodal voice-enabled devices that combine user emotion recognition and companionship capabilities can be used to attempt to mitigate conflicts between family members. The process of multimodal voice-enabled devices handling conflict events is as follows: Figure 5 As shown.

[0104] For example, in a family setting, when a couple is arguing, the multimodal voice network device is immediately activated upon detecting the abnormal sounds of "arguing." It then initiates the emotion recognition module, using audio emotion recognition and semantic understanding to identify the emotion category (argument) and the dialogue scenario category (couple arguing). Next, it analyzes conflict mitigation strategies and expresses these strategies through voice TTS (Text-to-Speech) and image information. For instance, it might broadcast the following: "Stop arguing! You've woken me up. You've been together for so long, have you forgotten your promises?" The device also displays a distressed and helpless expression on the screen.

[0105] For example, when parents are helping their children with homework, if the parent yells at the child while explaining, the multimodal voice device will be activated by the abnormal sound of "yelling". It will then start the emotion recognition module, identify the emotion category (yelling) and the dialogue scenario category (helping the child with homework) through audio emotion recognition and semantic understanding, and then perform conflict mitigation strategy analysis. It will express the mitigation strategy through voice TTS, image information and other means (broadcasting the following: "Your child has been doing homework for more than an hour and is very tired. The math olympiad problems are indeed quite difficult. Please be patient. I can help you search for math olympiad tutoring videos"). If the user says "Okay, go ahead and look for them", the multimodal voice device can play the relevant content.

[0106] In this embodiment, by combining the emotion tags of the voice signal and the dialogue scenario to formulate a strategy to alleviate conflict after the voice device recognizes abnormal voice, the conflict in the dialogue scenario can be effectively alleviated, thereby improving the reliability of the device's voice control.

[0107] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0109] According to another aspect of the embodiments of this application, a voice control device for implementing the voice control method of the above-described device is also provided, which can be applied to smart devices. Figure 6 This is a structural block diagram of a voice control device for an optional device according to an embodiment of this application, such as... Figure 6 As shown, the device may include:

[0110] The first extraction unit 602 is used to extract the acoustic features of the first speech signal to obtain the first acoustic features, wherein the first speech signal is the speech signal collected by the speech device during the speech interaction process.

[0111] The first parsing unit 604 is connected to the first extraction unit 602 and is used to perform semantic parsing based on the first acoustic features to obtain the target semantic intent corresponding to the first speech signal.

[0112] The first recognition unit 606 is connected to the first parsing unit 604 and is used to perform speech emotion recognition based on the first acoustic features when the target semantic intent belongs to a set of preset intents, so as to obtain the first emotion category corresponding to the first speech signal.

[0113] The fusion unit 608, connected to the first recognition unit 606, is used to fuse the first emotion category with the target semantic intent to obtain an updated target semantic intent, wherein the updated target semantic intent is a semantic intent with emotion parameters.

[0114] The first control unit 610, connected to the fusion unit 608, is used to control the controlled device to perform device operations corresponding to the updated target semantic intent, wherein the controlled device is the device that matches the updated target semantic intent.

[0115] It should be noted that the first extraction unit 602 in this embodiment can be used to perform the above step S202, the first parsing unit 604 in this embodiment can be used to perform the above step S204, the first identification unit 606 in this embodiment can be used to perform the above step S206, the fusion unit 608 in this embodiment can be used to perform the above step S208, and the first control unit 610 in this embodiment can be used to perform the above step S210.

[0116] Through the above modules, acoustic features of the first speech signal are extracted to obtain the first acoustic features, wherein the first speech signal is the speech signal collected by the voice device during voice interaction; semantic parsing is performed based on the first acoustic features to obtain the target semantic intent corresponding to the first speech signal; if the target semantic intent belongs to a set of preset intents, speech emotion recognition is performed based on the first acoustic features to obtain the first emotion category corresponding to the first speech signal; the first emotion category is fused with the target semantic intent to obtain the updated target semantic intent, wherein the updated target semantic intent is a semantic intent with emotion parameters; the controlled device is controlled to perform device operations corresponding to the updated target semantic intent, wherein the controlled device is a device that matches the updated target semantic intent. This solves the problem of poor device control flexibility caused by the single control method in the related technology, and improves the flexibility of device control.

[0117] In one exemplary embodiment, the above-described apparatus further includes:

[0118] The determining unit is used to determine the operation execution parameters of the controlled device according to the first emotion category before controlling the controlled device to perform the device operation corresponding to the updated target semantic intent. The operation execution parameters are the device parameters used by the controlled device to perform the device operation.

[0119] In one exemplary embodiment, the first extraction unit includes:

[0120] The first input module is used to input the first speech signal into the acoustic feature extraction model without performing text parsing on the first speech signal, and to obtain the first acoustic feature output by the acoustic feature extraction model, wherein the acoustic feature extraction model is a neural network model.

[0121] In one exemplary embodiment, the first parsing unit includes:

[0122] The second input module is used to input the first acoustic features into the speech recognition model to obtain the target semantic intent. The speech recognition model is used to identify the corresponding text vector features based on the output acoustic features, and to perform semantic parsing based on the text vector features to obtain the semantic intent corresponding to the output acoustic features.

[0123] In one exemplary embodiment, the first identification unit includes:

[0124] The third input module is used to input the first acoustic features into the speech emotion prediction model when the target semantic intent belongs to a set of preset intents, and obtain the target emotion label output by the speech emotion prediction model. The speech emotion prediction model is used to extract emotion features from the input acoustic features and predict the emotion label corresponding to the input acoustic features from a set of emotion labels based on the extracted emotion features. The target emotion label corresponds to the first emotion category.

[0125] In one exemplary embodiment, the fusion unit includes:

[0126] The fusion module is used to merge the first sentiment category as a modifier of the object in the target semantic intent into the target semantic intent, so as to obtain the updated target semantic intent.

[0127] In one exemplary embodiment, the above-described apparatus further includes:

[0128] The second control unit is used to control the controlled device to perform device operations corresponding to the target semantic intent after semantic parsing based on the first acoustic features, provided that the target semantic intent does not belong to a set of preset intents.

[0129] In one exemplary embodiment, the above-described apparatus further includes:

[0130] The second extraction unit is used to extract the acoustic features of the second speech signal when the voice device is woken up by abnormal speech, and obtain the second acoustic features, wherein the second speech signal is the speech signal collected after the voice device is woken up by abnormal speech.

[0131] The second parsing unit is used to perform semantic parsing based on the second acoustic features to obtain the current dialogue scenario corresponding to the second speech signal;

[0132] The second recognition unit is used to perform speech emotion recognition based on the second acoustic features to obtain the second emotion category corresponding to the second speech signal.

[0133] The third control unit is used to control the voice device to perform voice interaction with objects in the current dialogue scenario based on the current dialogue scenario and the second emotion category.

[0134] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of a device, can operate in situations such as... Figure 1The hardware environment shown can be implemented through software or hardware, and the hardware environment includes the network environment.

[0135] According to another aspect of the embodiments of this application, a storage medium is also provided, which can be located on a smart device. Optionally, in this embodiment, the storage medium can be used to execute program code for the voice control method of any of the devices described in the embodiments of this application.

[0136] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0137] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0138] S1, extract the acoustic features of the first speech signal to obtain the first acoustic features, wherein the first speech signal is the speech signal collected by the speech device during the speech interaction process.

[0139] S2, perform semantic parsing based on the first acoustic features to obtain the target semantic intent corresponding to the first speech signal;

[0140] S3, when the target semantic intent belongs to a set of preset intents, perform speech emotion recognition based on the first acoustic features to obtain the first emotion category corresponding to the first speech signal;

[0141] S4, the first sentiment category is fused with the target semantic intent to obtain the updated target semantic intent, wherein the updated target semantic intent is a semantic intent with sentiment parameters;

[0142] S5, control the controlled device to perform device operations corresponding to the updated target semantic intent, wherein the controlled device is the device that matches the updated target semantic intent.

[0143] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0144] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0145] According to another aspect of the embodiments of this application, an electronic device for implementing the voice control method of the above-described device is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0146] Figure 7This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 7 As shown, it includes a processor 702, a communication interface 704, a memory 707, and a communication bus 708. The processor 702, communication interface 704, and memory 707 communicate with each other via the communication bus 708.

[0147] Memory 707 is used to store computer programs;

[0148] When processor 702 executes a computer program stored in memory 707, it performs the following steps:

[0149] S1, extract the acoustic features of the first speech signal to obtain the first acoustic features, wherein the first speech signal is the speech signal collected by the speech device during the speech interaction process.

[0150] S2, perform semantic parsing based on the first acoustic features to obtain the target semantic intent corresponding to the first speech signal;

[0151] S3, when the target semantic intent belongs to a set of preset intents, perform speech emotion recognition based on the first acoustic features to obtain the first emotion category corresponding to the first speech signal;

[0152] S4, the first sentiment category is fused with the target semantic intent to obtain the updated target semantic intent, wherein the updated target semantic intent is a semantic intent with sentiment parameters;

[0153] S5, control the controlled device to perform device operations corresponding to the updated target semantic intent, wherein the controlled device is the device that matches the updated target semantic intent.

[0154] Optionally, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic device and other devices.

[0155] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0156] As an example, the memory 707 described above may include, but is not limited to, the first extraction unit 602, the first parsing unit 604, the first recognition unit 606, the fusion unit 608, and the first control unit 610 of the voice control device of the aforementioned device. Furthermore, it may include, but is not limited to, other module units of the voice control device of the aforementioned device, which will not be elaborated upon in this example.

[0157] The processor mentioned above can be a general-purpose processor, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0158] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0159] Those skilled in the art will understand that Figure 7 The structure shown is for illustrative purposes only. The device implementing the voice control method of the above device can be a terminal device, such as a smartphone (e.g., Android phone, iOS phone), tablet computer, handheld computer, mobile internet device (MID), PAD, etc. Figure 7 This does not limit the structure of the aforementioned electronic device. For example, the electronic device may also include components that are more... Figure 7 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 7 The different configurations shown.

[0160] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0161] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0162] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0163] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0165] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0166] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or at least two units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0167] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A voice control method of a device, characterized by, The method comprises: extracting acoustic features of a first speech signal to obtain first acoustic features, wherein the first speech signal is a speech signal collected by a speech device during speech interaction; performing semantic analysis according to the first acoustic features to obtain a target semantic intent corresponding to the first speech signal; in the case where the target semantic intent belongs to a group of preset intents, performing speech emotion recognition according to the first acoustic features to obtain a first emotion category corresponding to the first speech signal; fusing the first emotion category and the target semantic intent to obtain an updated target semantic intent, wherein the updated target semantic intent is a semantic intent with an emotion parameter; controlling a controlled device to perform a device operation corresponding to the updated target semantic intent, wherein the controlled device is a device matched with the updated target semantic intent.

2. The method of claim 1, wherein, Before the controlled device performs the device operation corresponding to the updated target semantic intent, the method further comprises: determining an operation execution parameter of the controlled device according to the first emotion category, wherein the operation execution parameter is a device parameter used by the controlled device to perform the device operation.

3. The method of claim 1, wherein, The method of extracting acoustic features of a first speech signal to obtain first acoustic features comprises: without text analysis of the first speech signal, inputting the first speech signal into an acoustic feature extraction model to obtain the first acoustic features output by the acoustic feature extraction model, wherein the acoustic feature extraction model is a neural network model.

4. The method of claim 1, wherein, The method of performing semantic analysis according to the first acoustic features to obtain a target semantic intent corresponding to the first speech signal comprises: inputting the first acoustic features into a speech recognition model to obtain the target semantic intent, wherein the speech recognition model is used to identify corresponding text vector features based on output acoustic features, and perform semantic analysis according to the text vector features to obtain a semantic intent corresponding to the output acoustic features.

5. The method of claim 1, wherein, The method of performing speech emotion recognition according to the first acoustic features to obtain a first emotion category corresponding to the first speech signal in the case where the target semantic intent belongs to a group of preset intents comprises: in the case where the target semantic intent belongs to the group of preset intents, inputting the first acoustic features into a speech emotion prediction model to obtain a target emotion label output by the speech emotion prediction model, wherein the speech emotion prediction model is used to extract emotion features from input acoustic features, and predict an emotion label corresponding to the input acoustic features from a group of emotion labels based on the extracted emotion features, the target emotion label corresponding to the first emotion category.

6. The method of claim 1, wherein, The method of fusing the first emotion category and the target semantic intent to obtain an updated target semantic intent comprises: fusing the first emotion category as a subject's attribute in the target semantic intent into the target semantic intent to obtain the updated target semantic intent.

7. The method of claim 1, wherein, After the semantic analysis according to the first acoustic feature, the method further comprises: In the case where the target semantic intention does not belong to the group of preset intentions, the controlled device is controlled to perform a device operation corresponding to the target semantic intention.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: In the case where the voice device is woken up by abnormal voice, acoustic features of a second voice signal are extracted to obtain second acoustic features, wherein the second voice signal is a voice signal collected after the voice device is woken up by the abnormal voice; According to the second acoustic features, semantic analysis is performed to obtain a current conversation scene corresponding to the second voice signal; According to the second acoustic features, voice emotion recognition is performed to obtain a second emotion category corresponding to the second voice signal; According to the current conversation scene and the second emotion category, the voice device is controlled to perform voice interaction with an object in the current conversation scene.

9. A voice control apparatus of a device, characterized by, Comprise: A first extraction unit is configured to extract acoustic features of a first voice signal to obtain first acoustic features, wherein the first voice signal is a voice signal collected by a voice device in the process of voice interaction; A first analysis unit is configured to perform semantic analysis according to the first acoustic features to obtain a target semantic intention corresponding to the first voice signal; A first identification unit is configured to, in the case where the target semantic intention belongs to a group of preset intentions, perform voice emotion recognition according to the first acoustic features to obtain a first emotion category corresponding to the first voice signal; A fusion unit is configured to fuse the first emotion category and the target semantic intention to obtain an updated target semantic intention, wherein the updated target semantic intention is a semantic intention with an emotion parameter; A first control unit is configured to control a controlled device to perform a device operation corresponding to the updated target semantic intention, wherein the controlled device is a device matching the updated target semantic intention.

10. A computer readable storage medium, characterized in that, The computer-readable storage medium comprises a stored program, wherein the program performs the method of any one of claims 1-8 when executed. 11.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1-8 through the computer program.

Images