Orientation determination method and related equipment

Abstract

The embodiment of the invention provides an orientation determination method and related equipment, and relates to the field of voice. In the method, a first feature of a first device is obtained. The first feature is obtained according to M single-channel voice signals corresponding to M audio acquisition units in the first equipment. M is an integer, and the M audio acquisition units perform audio acquisition for the same sound source. The first feature comprises a feature of direct sound and / or reflected sound sensitive to orientation in a sound signal generated by the sound source. A first orientation angle between the sound source and the first device is determined based on the first feature. According to the method, the first feature is calculated according to the single-channel voice signal of the first device, the orientation angle can be determined according to the first feature, a deep learning model does not need to be pre-trained, and the orientation determination method is low in cost, high in generalization and small in occupied computing power resource.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of speech, and in particular to a method for determining a direction and related devices. Background Art

[0002] With the popularity of smart devices, the demand for human-computer interaction continues to increase. Users can interact with the computer through voice, gestures and other control methods. For voice human-computer interaction scenarios, in the absence of other sensors such as cameras, it is generally necessary to use the user's audio to determine the user's orientation. Based on the user's orientation, it is helpful to infer the user's interaction intention, determine the focus device, and improve the interactive experience of the voice assistant.

[0003] The method of direction detection based on audio requires the collection of multi-directional audio data in the early stage, and then the model learns information related to the direction to judge the user's direction. The models used include support vector machine (SVM), random forest, convolutional neural network (CNN), etc. The above detection methods face problems such as high data collection cost, limited model generalization performance, high computing power requirements, and difficult terminal deployment. Summary of the invention

[0004] The present application provides a method for determining a direction and related equipment, which do not require pre-training, have strong generalization, low computing power usage, and low cost for determining a direction.

[0005] In a first aspect, a method for determining an orientation is provided. The method may be executed by an orientation determining device itself, or by a chip in the orientation determining device.

[0006] The above-mentioned direction determination method includes the following steps: obtaining a first feature of the first device. The above-mentioned first feature is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the first device. M is an integer, and the M audio acquisition units perform audio acquisition for the same sound source. The above-mentioned first feature includes the features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source. Determine a first direction angle between the sound source and the first device based on the first feature.

[0007] It can be seen that in this solution, the first feature is calculated according to the single-channel voice signal of the first device, and the direction angle can be determined based on the first feature. There is no need to pre-train a deep learning model. The direction determination method has low cost, strong generalization, and occupies less computing power resources.

[0008] In a possible implementation of the first aspect, the obtaining of the first feature of the first device specifically includes the following steps: obtaining M single-channel voice signals from the first device. Obtaining a second feature based on the M single-channel voice signals, the second feature being a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction. Obtaining a first feature based on the second feature, the first feature being obtained according to high-frequency energy, low-frequency energy, or the difference between high-frequency energy and low-frequency energy in the second feature.

[0009] In this scheme, M single-channel voice signals are first obtained from the first device, and then the second feature is obtained based on the above M single-channel voice signals. Finally, the first feature is obtained according to the high-frequency energy, low-frequency energy or the difference between high-frequency energy and low-frequency energy in the second feature. Since the energy is normalized, this scheme is not easily affected by factors such as the distance and orientation between the sound source and the first device when determining the direction angle, and the direction angle has high accuracy.

[0010] In a possible implementation of the first aspect, the second feature is obtained based on M single-channel voice signals, specifically including the following steps: each single-channel voice signal is processed as follows to obtain a first spectrum corresponding to the single-channel voice signal. The single-channel voice signal is framed and windowed to obtain Q-segment voice signals, where Q is greater than or equal to two. The Q-segment voice signals are Fourier transformed to obtain a time-frequency spectrum. The time-frequency spectrum is signal modulated to obtain a second spectrum. The second spectrum is filtered to obtain the first spectrum. The second feature is obtained based on the M first spectra.

[0011] In a possible implementation manner of the first aspect, the obtaining of the first feature of the first device specifically includes the following steps: obtaining a second feature from the first device, where the second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction. The first feature is obtained based on the second feature, where the first feature is obtained according to high-frequency energy, low-frequency energy, or the difference between high-frequency energy and low-frequency energy in the second feature.

[0012] In this solution, the second feature of the direct sound and / or reflected sound that is sensitive to direction in the sound signal is first obtained from the first device, and then the first feature is obtained based on the high-frequency energy, low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature. The second feature is first processed in the first device, which can effectively reduce the computing power occupied by this solution.

[0013] In a possible implementation of the first aspect, the determining of the first orientation angle between the sound source and the first device based on the first feature specifically includes the following steps: determining the second orientation angle according to the first feature and a preset correspondence, wherein the preset correspondence is a correspondence between different first features and the second orientation angle. Determining the first orientation angle based on the second orientation angle and a third orientation angle. The third orientation angle is determined based on an image captured by a camera of the first device.

[0014] In this solution, the second orientation angle can be determined by using the preset correspondence and the first feature. In order to improve the accuracy of the orientation angle between the sound source and the first device, the present application also determines the third orientation angle based on the image captured by the camera of the first device, and then determines the first orientation angle by combining the second orientation angle and the third orientation angle.

[0015] In a possible implementation of the first aspect, the above-mentioned orientation determination method further includes the following steps: when the first orientation angle falls into the first range and the first command word table includes the first command word, the first device is controlled to enter a wake-up state and perform a first operation corresponding to the first command word. The above-mentioned first command word is a command word contained in the text corresponding to the sound signal. When the first orientation angle falls into the second range and the second command word table includes the first command word, the first device is controlled to enter a wake-up state and perform the first operation. The number of command words in the above-mentioned first command word table is greater than the number of command words in the second command word table, and the absolute value of any orientation angle in the above-mentioned second range is less than the absolute value of any orientation angle in the first range.

[0016] In this solution, two orientation angle ranges, the first range and the second range, and the first command word table and the second command word table with different numbers of command words are used to determine whether to wake up the first device and whether to perform a voice interaction response. The difficulty of entering the wake-up state is adjusted according to the orientation angle, which can reduce the risk of false wake-up, ensure the accuracy of voice interaction, and enhance the user's voice interaction experience.

[0017] In a possible implementation of the first aspect, the orientation determination method further includes the following steps: when the first orientation angle is zero, controlling the first device to enter a wake-up state. When the first orientation angle is zero, it is considered that the user is interacting with the first device, and the first device can be controlled to enter a wake-up state without the user saying a wake-up word, thereby simplifying the voice interaction process and improving the user's voice interaction experience.

[0018] In a possible implementation of the first aspect, after controlling the first device to enter the awake state, the direction determination method further includes the following steps: controlling the first device to perform a first operation corresponding to a first command word. The first command word is a command word contained in the text corresponding to the sound signal.

[0019] In this solution, after the first device is controlled to enter the wake-up state without the user saying the wake-up word, the first device is controlled to execute the first operation corresponding to the command word contained in the text corresponding to the sound signal, in response to the user's voice interaction command.

[0020] In a second aspect, the present application also provides a focus device determination method, which can be executed by a focus device determination apparatus, or by a chip in the focus device determination apparatus.

[0021] The focus device determination method includes the following steps: obtaining N first features corresponding to N first devices. The first features of the first devices are obtained based on M single-channel voice signals corresponding to M audio acquisition units in the first devices. The N first devices perform audio acquisition for the same sound source. The first features include features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source. N is greater than or equal to two, and M is an integer. The focus device is determined from the N first devices based on the N first features.

[0022] In this solution, the corresponding first feature is calculated according to the single-channel voice signal of each first device, and the focus device can be determined from the N first devices based on the N first features. This method does not require pre-training of the deep learning model. The focus device determination method has low cost, strong generalization, and occupies less computing resources.

[0023] In a possible implementation of the second aspect, the above-mentioned determining the focus device from N first devices based on N first features specifically includes the following steps: determining the first device corresponding to at least one first feature with a higher feature value among the N first features as the focus device.

[0024] In this scheme, the first device corresponding to the first feature with the highest feature value among the N first features can be determined as the focus device, or the first devices corresponding to the x first features with the highest feature values ​​among the N first features can be determined as the focus devices, where x is greater than or equal to two.

[0025] In a possible implementation of the second aspect, the above-mentioned acquisition of N first features corresponding to N first devices specifically includes the following steps: acquiring M single-channel voice signals from the first device. Determining a second feature of the first device based on the M single-channel voice signals. The above-mentioned second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction. The first feature is obtained based on the second feature, and the first feature is obtained according to the high-frequency energy, low-frequency energy, or the difference between high-frequency energy and low-frequency energy in the second feature.

[0026] In this scheme, M single-channel voice signals are first obtained from the first device, and then the second feature is obtained based on the above M single-channel voice signals. Finally, the first feature is obtained according to the high-frequency energy, low-frequency energy or the difference between high-frequency energy and low-frequency energy in the second feature. Since the energy is normalized, this scheme is not easily affected by factors such as the distance and orientation between the sound source and the first device when determining the focal device, and the focal device has high accuracy.

[0027] In a possible implementation of the second aspect, the above-mentioned determination of the second feature of the first device based on M single-channel voice signals specifically includes the following steps: performing the following processing on each single-channel voice signal to obtain a first spectrum corresponding to the single-channel voice signal. The single-channel voice signal is framed and windowed to obtain Q-segment voice signals, where Q is greater than or equal to two. Fourier transform is performed on the Q-segment voice signals to obtain a time-frequency spectrum. Signal modulation is performed on the time-frequency spectrum to obtain a second spectrum. The second spectrum is filtered to obtain the first spectrum. The second feature is obtained based on the M first spectra.

[0028] In a possible implementation of the second aspect, the obtaining of the N first features corresponding to the N first devices specifically includes the following steps: obtaining a second feature from the first device. The second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction. The first feature is obtained based on the second feature, and the first feature is obtained according to the high-frequency energy, low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature.

[0029] In this solution, the second feature of the direct sound and / or reflected sound that is sensitive to direction in the sound signal is first obtained from the first device, and then the first feature is obtained based on the high-frequency energy, low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature. The second feature is first processed in the first device, which can effectively reduce the computing power occupied by this solution.

[0030] In a possible implementation manner of the second aspect, the focus device determination method further includes the following steps: controlling the focus device to enter a wake-up state.

[0031] In this solution, the focus device can be controlled to enter the wake-up state without the user saying the wake-up word, which simplifies the voice interaction process and improves the user's voice interaction experience.

[0032] In a possible implementation of the second aspect, after controlling the focus device to enter the awake state, the focus device determination method further includes the following steps: controlling the focus device to execute a first operation corresponding to a first command word. The first command word is a command word contained in the text corresponding to the sound signal.

[0033] In this solution, after the focus device is controlled to enter the wake-up state without the user saying the wake-up word, the focus device is controlled to execute the first operation corresponding to the command word contained in the text corresponding to the sound signal, in response to the user's voice interaction command.

[0034] In a third aspect, the present application also provides a direction determination device, including an acquisition module and a determination module.

[0035] The acquisition module is used to acquire a first feature of the first device. The above-mentioned first feature is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the first device. M is an integer. The M audio acquisition units perform audio acquisition for the same sound source. The first feature includes features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source.

[0036] A determination module is used to determine a first orientation angle between a sound source and a first device based on a first feature.

[0037] In this solution, the direction determination device calculates the first feature based on the single-channel voice signal of the first device, and can determine the direction angle based on the first feature. There is no need to pre-train a deep learning model. The direction determination method has low cost, strong generalization, and occupies less computing power resources.

[0038] In a possible implementation of the third aspect, the orientation determination device further includes a processing module.

[0039] The processing module is used to control the first device to enter a wake-up state and execute a first operation corresponding to the first command word when the first orientation angle falls within the first range and the first command word table includes a first command word. The first command word is a command word included in the text corresponding to the sound signal.

[0040] The processing module is further configured to control the first device to enter a wake-up state and perform a first operation when the first orientation angle falls within the second range and the second command word list includes the first command word. The number of command words in the first command word list is greater than the number of command words in the second command word list, and the absolute value of any orientation angle in the second range is less than the absolute value of any orientation angle in the first range.

[0041] In a possible implementation manner of the third aspect, the orientation determination device further includes:

[0042] The processing module is used to control the first device to enter a wake-up state when the first orientation angle is zero.

[0043] In a possible implementation of the third aspect, after controlling the first device to enter a wake-up state, the processing module is also used to control the first device to perform a first operation corresponding to a first command word, where the first command word is a command word contained in a text corresponding to the sound signal.

[0044] In a fourth aspect, the present application also provides a focus device determination apparatus, comprising an acquisition module and a determination module.

[0045] An acquisition module is used to acquire N first features corresponding to N first devices. The first feature of the above-mentioned first device is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the first device. The N first devices perform audio acquisition for the same sound source. The first feature includes features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source. N is greater than or equal to two, and M is an integer.

[0046] A determination module is used to determine a focus device from N first devices based on the N first features.

[0047] In this solution, the focus device determination device calculates the corresponding first feature based on the single-channel voice signal of each first device, and can determine the focus device from N first devices based on the N first features. This method does not require pre-training of a deep learning model. The focus device determination method has low cost, strong generalization, and occupies less computing resources.

[0048] In a possible implementation of the fourth aspect, the focus device determination apparatus further includes a processing module.

[0049] The processing module is used to control the focus device to enter the awake state.

[0050] In a possible implementation of the fourth aspect, after the focus device is controlled to enter the wake-up state, the processing module is also used to control the focus device to execute a first operation corresponding to a first command word, where the first command word is a command word contained in the text corresponding to the sound signal.

[0051] In a fifth aspect, the present application also provides an electronic device, comprising a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute the method described in any one of the first aspect or the second aspect.

[0052] In a sixth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method as described in any one of the first aspect or the second aspect.

[0053] In a seventh aspect, the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in any one of the first aspect or the second aspect.

[0054] In an eighth aspect, the present application further provides a chip, comprising a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to execute the method described in any one of the first aspect or the second aspect.

[0055] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method described in any one of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The following is an introduction to the drawings used in the embodiments of the present application.

[0057] Figure 1A is a schematic diagram of a first device provided in an embodiment of the present application;

[0058] Figure 1B It is a flowchart of a sound direction-based rejection strategy provided in an embodiment of the present application;

[0059] Figure 1C It is a flowchart of another sound direction-based rejection strategy provided in an embodiment of the present application;

[0060] Figure 1D It is a schematic diagram of a voice interaction reference process based on sound direction provided in an embodiment of the present application;

[0061] Figure 1E It is a schematic diagram of a process for generating a target object list provided in an embodiment of the present application;

[0062] Figure 1F is a schematic diagram of an azimuth angle and an orientation angle provided in an embodiment of the present application;

[0063] Figure 1G is a schematic diagram of an active recommendation reference process based on sound direction provided in an embodiment of the present application;

[0064] Figure 1H It is a schematic diagram of a command action object determination process provided by an embodiment of the present application;

[0065] Fig. 1I is a schematic diagram of a speaker enhancement process based on sound direction provided in an embodiment of the present application;

[0066] Figure 1J is a schematic diagram of a cockpit sound zone provided in an embodiment of the present application;

[0067] Figure 1K It is a schematic diagram of a human-to-human interaction assistant orientation enhancement process provided by an embodiment of the present application;

[0068] Figure 1L is a schematic diagram of a process for calculating an angle difference provided in an embodiment of the present application;

[0069] Figure 1M is a schematic diagram of a visible and audible process based on sound direction provided by an embodiment of the present application;

[0070] Figure 1N is a schematic diagram of the system architecture of a method for determining a direction provided in an embodiment of the present application;

[0071] Fig.1O is a schematic diagram of the system architecture of another method for determining a direction provided in an embodiment of the present application;

[0072] Figure 1P is a schematic diagram of the system architecture of a focus device determination method provided by an embodiment of the present application;

[0073] Figure 1Q is a schematic diagram of the system architecture of another focus device determination method provided by an embodiment of the present application;

[0074] Figure 2 is a flow chart of a method for determining a direction provided in an embodiment of the present application;

[0075] Figure 3A is a schematic diagram of a process of obtaining a first spectrum provided in an embodiment of the present application;

[0076] Figure 3B This is a schematic diagram of a process for waking up a first device provided in an embodiment of the present application;

[0077] Figure 3C This is a schematic diagram of a user experience dynamic effect provided by an embodiment of the present application;

[0078] Figure 3D This is another schematic diagram of user experience dynamic effect provided by an embodiment of the present application;

[0079] Figure 4 is a flow chart of a method for determining a focus device provided in an embodiment of the present application;

[0080] Figure 5 is a schematic diagram of the structure of a device provided in an embodiment of the present application;

[0081] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0083] Due to the applications involved in the embodiments of the present application, in order to facilitate understanding, the relevant terms and other related concepts involved in the embodiments of the present application are first introduced below.

[0084] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0085] The "at least one" mentioned in the embodiments of the present application refers to one or more, and "multiple" refers to two or more. "At least one of the following" or its similar expression refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, c can be single or multiple. "And / or" describes the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can be represented by: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are an "or" relationship. The serial numbers of the steps of the embodiments of the present application (such as step S1, step S21, etc.) are only for distinguishing different steps, and do not limit the order of execution between the steps.

[0086] Furthermore, unless otherwise specified, the ordinal numbers such as "first" and "second" used in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. For example, the first device and the second device are only for the convenience of description, and do not represent the difference in structure, importance, etc. between the first device and the second device. In some embodiments, the first device and the second device can also be the same device.

[0087] In the above embodiments, the term "when..." can be interpreted as meaning "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modification, equivalent substitution, improvement, etc. made within the concept and principle of the present application shall be included in the protection scope of the present application.

[0088] The orientation angle between the device and the user refers to the angle between the normal of the user's face and the first line, which is equivalent to the head posture. The first line is the line between the device and the user.

[0089] With the popularity of smart devices, the demand for human-computer interaction continues to increase. Users can interact with the computer through voice, gestures and other control methods. For voice human-computer interaction scenarios, in the absence of other sensors such as cameras, it is generally necessary to use the user's audio to determine the user's orientation. Based on the user's orientation, it is helpful to infer the user's interaction intention, determine the focus device, and improve the interactive experience of the voice assistant.

[0090] The method of detecting the direction based on audio requires the collection of multi-directional audio data in the early stage, and then the model learns information related to the direction to judge the user's direction. The above detection method faces problems such as high data collection cost, limited model generalization performance, high computing power requirements, and difficulty in terminal deployment.

[0091] Therefore, an embodiment of the present application provides a method for determining a direction. By calculating a first feature based on a single-channel voice signal of a first device, the direction angle can be determined based on the first feature. No pre-training of a deep learning model is required. The method for determining a direction has low cost, strong generalization, and occupies little computing power resources.

[0092] In addition, an embodiment of the present application also provides a focus device determination method, which calculates a corresponding first feature based on the single-channel voice signal of each of the N first devices, and can determine the focus device from the N first devices based on the N first features. This method does not require pre-training of a deep learning model, and the focus device determination method has low cost, strong generalization, and occupies less computing power resources.

[0093] The first device in the direction determination method and the focus device determination method of the embodiment of the present application refers to a device with an audio acquisition unit (such as a microphone, etc.), and the number of audio acquisition units can be 1, 2, 3, etc., and the specific number of audio acquisition units is not limited. Figure 1A , Figure 1A : is a schematic diagram of a first device provided in an embodiment of the present application; the first device includes but is not limited to a mobile phone, a large screen, a speaker, a central control screen, a TV, a sports watch, a Bluetooth headset, a tablet computer, a car computer, etc. The above-mentioned first device has a voice interaction function to respond to the user's voice interaction command.

[0094] Exemplarily, when there is only one first device in the environment (or scene, or space) where the user is located, the orientation angle between the first device and the user can be determined by using the above orientation determination method.

[0095] Again illustratively, reference Figure 1AWhen there are N first devices (N is greater than or equal to two) in the environment (or scene, or space) where the user is located, the focus device is determined from the N first devices using the focus device determination method to respond to the user's voice interaction command, thereby avoiding "one call and a hundred responses" and greatly improving the voice interaction experience.

[0096] The orientation determination method and focus device determination method of the embodiments of the present application are applicable to privacy scenarios without camera coverage, such as at home, in the back seat of a car, and other scenarios without cameras. During the voice interaction between the user and the device, the user orientation is judged through audio, or the focus device is determined, thereby improving the interactive experience of the voice assistant.

[0097] The direction determination method and focus device determination method of the embodiments of the present application are also applicable to scenes covered by cameras, and can assist the camera in direction determination and focus device determination, thereby improving the accuracy of direction and focus device.

[0098] The orientation determination method of the embodiment of the present application can be executed by the orientation determination device itself, or by a chip in the orientation determination device. The orientation determination device can be the first device and / or other main control device (or central control device) with data processing capability.

[0099] The focus device determination method of the embodiment of the present application can be executed by the focus device determination device itself, or by a chip in the focus device determination device. The focus device determination device can be a first device and / or a main control device (or central control device) with data processing capability.

[0100] The following describes application scenarios related to the orientation determination method or focus device determination method of the embodiments of the present application:

[0101] The first scenario is a rejection strategy based on the sound direction. For example, refer to Figure 1B , Figure 1B : is a flowchart of a sound direction-based rejection strategy provided by an embodiment of the present application; specifically, it includes the following steps:

[0102] S111. The direction determining device obtains a first feature of the device.

[0103] The first feature is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the device. M is an integer. The M audio acquisition units perform audio acquisition for the same sound source, and one audio acquisition unit performs audio acquisition to obtain a single-channel voice signal. The above-mentioned first feature includes the characteristics of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source.

[0104] S112. The direction determining device determines a direction angle between the device and the sound source according to the first feature.

[0105] For a specific description, please refer to the following description of step 202, which will not be repeated here.

[0106] S113: If the direction corner falls into the third range, the direction determination device controls the device to perform voice recognition on the sound signal generated by the sound source.

[0107] Specifically, the third range indicates that the user is facing the device to interact. For example, the third range is -45° to 45°. At this time, the direction determination device controls the device to respond to the sound signal sent by the user, and performs voice recognition on the sound signal. When the text corresponding to the sound signal includes the command word of the device, the device performs the operation corresponding to the command word. Otherwise, when the text corresponding to the sound signal does not include the command word of the device, the device does not perform the operation.

[0108] S114: If the orientation corner falls into the fourth range, the orientation determination device controls the device not to respond to the sound signal generated by the sound source.

[0109] Specifically, the fourth range indicates that the user is not interacting with the device. At this time, the control device does not respond to the sound signal sent by the user, that is, it refuses to recognize the sound signal.

[0110] For example, the user sitting in the main driver's seat says to the co-driver's seat: "It's too hot, open the window", and the car computer determines that the orientation angle is not zero based on the first feature, so it can be determined that the user is not speaking the command to the central control screen in the car (the central control screen is a display screen connected to the car computer, that is, a display screen for human-computer interaction). The car computer determines that the main driver user is not interacting with the car computer at this time, so it rejects this command. When the user sitting in the main driver's seat says to the central control screen: "It's too hot, open the window", the car computer determines that the orientation angle at this time is zero based on the first feature, and the orientation angle falls into the third range, so it can be determined that the user is speaking the command to the central control screen in the car. The car computer determines that the main driver user wants to interact with the car computer at this time, so the car computer responds to the user's voice command and opens the window next to the main driver's seat.

[0111] Further, Figure 1B In the embodiment, the direction determination device may also perform speech recognition on the sound signal to obtain corresponding semantics, and combine the above-mentioned speech and the direction angle to determine whether the device rejects the sound signal.

[0112] Again illustratively, reference Figure 1C , Figure 1C : is a flow chart of another sound direction-based rejection strategy provided by an embodiment of the present application; specifically, it includes the following steps:

[0113] S121. The focus device determining apparatus obtains N first features corresponding to N devices.

[0114] The first feature is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the device. M is an integer. The M audio acquisition units perform audio acquisition for the same sound source, and one audio acquisition unit performs audio acquisition to obtain a single-channel voice signal. The above-mentioned first feature includes the characteristics of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source.

[0115] S122. The focus device determining means determines a focus device from N devices based on the N first features.

[0116] For a specific description, please refer to the following description of step 402, which will not be repeated here.

[0117] S123: The focus device determining device controls the focus device to respond to the sound signal generated by the sound source.

[0118] Specifically, when the text corresponding to the sound signal includes the command word of the focus device, the focus device performs the operation corresponding to the command word. Otherwise, when the text corresponding to the sound signal does not include the command word of the focus device, the focus device does not perform the operation.

[0119] S124: The focus device determining device controls the non-focus devices not to respond to the sound signal generated by the sound source.

[0120] Specifically, the focus device determining means controls the focus device not to respond to the sound signal sent by the user, that is, to reject the sound signal.

[0121] The second scenario is voice interaction reference based on sound direction. For example, refer to Figure 1D , Figure 1D : is a schematic diagram of a voice interaction reference process based on sound direction provided in an embodiment of the present application; specifically, it includes the following steps:

[0122] 1) The device receives a voice signal, and the orientation detection module performs orientation detection to determine the user's orientation angle β.

[0123] 2) The sound zone positioning module performs sound zone positioning to determine the user's position coordinates, such as locating the user at the main driving position.

[0124] 3) The speech recognition module performs speech recognition to extract keyword information related to the object of the command.

[0125] 4) Perform semantic understanding on the keyword information to determine whether there is any ambiguity in reference. For example, if a user says "increase the angle of the driver's rearview mirror", there is only one driver's rearview mirror in the car, so there is no ambiguity. The target can be directly determined to be the driver's rearview mirror and a response can be given.

[0126] 5) For another example, the user says "increase the angle of the rearview mirror", the keyword is rearview mirror, and there are two rearview mirrors on the car. In this case, there is ambiguity, and the target object list needs to be generated in combination with the user's orientation angle and position coordinates.

[0127] 6) Determine whether there is only one candidate object in the target object list. If so, designate the candidate object as the action object, and the vehicle computer responds to execute the control operation corresponding to the voice signal, such as increasing the rearview mirror angle.

[0128] If the target object list is empty or there are multiple candidate objects, the car computer guides the user to enter the second round of interaction through voice or other means to clarify the object of the command. For example, the car computer asks the user by voice or text output on the screen: "Do you want to adjust the angle of the driver's rearview mirror or the passenger's rearview mirror?"

[0129] Figure 1D The illustrated process may be executed by a direction determining device.

[0130] For example, when the car computer in the car has been awakened, the user sitting in the main driver's seat says to the rearview mirror on the passenger side: "Adjust the angle of the rearview mirror a little wider", and the car computer executes Figure 1D In the process shown, the vehicle computer recognizes the command, combines the position information of the two rearview mirrors in the vehicle and the user's orientation, and infers that the user wishes to change the passenger side rearview mirror. The voice assistant responds: "OK, the angle of the passenger side rearview mirror has been increased."

[0131] In another example of a voice interaction reference scenario based on sound direction, the user sitting in the main driver's seat says to the rearview mirror on the passenger side: "XXX, adjust the rearview mirror angle a little wider." "XXX" is the wake-up word of the car computer, such as "Xiaoyi Xiaoyi". At this time, the car computer executes Figure 1D In the process shown, the vehicle computer recognizes the command, combines the position information of the two rearview mirrors in the vehicle and the user's orientation, and infers that the user wishes to change the passenger side rearview mirror. The voice assistant responds: "OK, the angle of the passenger side rearview mirror has been increased."

[0132] The target object list is generated by combining the user position coordinates, the orientation angle and the coordinate information of the candidate object to determine whether to add the object to the target object list. The target object list is initialized as an empty list. For the candidate object i, the method for determining whether to add it to the target object list is as follows. Figure 1E , Figure 1E This is a flow chart of generating a target object list provided by an embodiment of the present application; it specifically includes the following steps:

[0133] 1) The vehicle computer has a built-in database of spatial coordinate information of physical modules (such as windows, rearview mirrors, lights, seats, etc.) that can be controlled by voice. Figure 1DThe keyword information (such as rearview mirror) in the above spatial coordinate information database can obtain the two-dimensional coordinate information t of the candidate object i. i =[x i ,y i ].

[0134] 2)Reference Figure 1F , Figure 1F is a schematic diagram of an azimuth and orientation angle provided in an embodiment of the present application; combined with the two-dimensional coordinate information t of the candidate object i i and the user's position coordinates b = [b1, b2], calculate the azimuth of candidate object i where tan -1 is the inverse tangent function.

[0135] 3) According to the azimuth angle α i Calculate the angle difference with the user's orientation angle β The angle difference Compared with the preset angle threshold θ, if If the candidate object i is not in the area directly in front of the user, the target object list will not be updated; otherwise, the candidate object i will be added to the target object list.

[0136] The third scenario is active recommendation based on the sound direction. Figure 1G , Figure 1G : is a schematic diagram of a process of actively recommending references based on sound direction provided by an embodiment of the present application; specifically, it includes the following steps:

[0137] 1) The device receives a voice signal, and the orientation detection module performs orientation detection to determine the user's orientation angle.

[0138] 2) The speech recognition module performs speech recognition to extract keyword information related to the object of the command.

[0139] 3) Understand the semantics of the keyword information and determine whether there is any ambiguity in reference by combining it with the map information near the user's geographical location. For example, if the user says "what is the name of this community", and there is only one residential area near the user's location (such as on the left side of the road), there is no ambiguity at this time, and it can be directly determined that the user is referring to the community on the left side of the road, and the car computer can respond directly.

[0140] 4) If there is more than one cell near the user's location, there is ambiguity and the command action object determination process needs to be entered.

[0141] When judging the target of a command, the command target is determined by combining the user's orientation angle and the position information of the candidate object. If the target can be determined (such as the community on the left side of the road), the vehicle computer will respond ("The community on the right side of the road is xx community").

[0142] If the target cannot be determined, the vehicle computer guides the user into two-round interaction through voice or other means to clarify the target of the command. For example, the vehicle computer asks the user through voice or text on the screen: "Do you want to inquire about the community on the left side of the road or the community on the right side of the road?"

[0143] Figure 1G The illustrated process may be executed by a direction determining device.

[0144] refer to Figure 1H , Figure 1H : is a schematic diagram of a command action object determination process provided by an embodiment of the present application; the command action object determination process includes the following steps:

[0145] 1) Determine the reference area in combination with the user's orientation angle. For example, if β < -45, the reference area is the area to the left of the user's location. If β > 45, the reference area is the area to the right of the user's location. If -45 < β < 45, the reference area is the area directly in front of the user's location.

[0146] 2) Combine the map information near the user's geographical location and the above-mentioned reference area to determine whether there is a candidate object in the reference area (for example, if the user asks for the name of a community, determine whether there is a residential area in the reference area). If the candidate object exists and only exists, then the object is set as the command action object; if the candidate object does not exist or there are multiple candidate objects, then the command action object cannot be determined.

[0147] For example, when the car computer in the car has been awakened, the user sitting in the main driving seat looks out the left window and says: "What is the name of this community?" The car computer executes Figure 1G In the process shown, the vehicle computer recognizes the command and determines based on the user's direction that the cell the user is asking about is the cell on the left side of the road rather than the cell on the right side, and gives an answer.

[0148] In another example of the active recommendation based on the sound direction, the user sitting in the main driving seat looks out the left window and says: "XXX, what is the name of this community?" "XXX" is the wake-up word of the car computer, such as "Xiaoyi Xiaoyi". At this time, the car computer executes Figure 1G In the process shown, the vehicle computer recognizes the command and determines based on the user's direction that the cell the user is asking about is the cell on the left side of the road rather than the cell on the right side, and gives an answer.

[0149] The fourth scenario is speaker enhancement based on sound direction. Fig. 1I , Fig. 1I : is a schematic diagram of a speaker enhancement process based on sound direction provided in an embodiment of the present application; specifically, it includes the following steps:

[0150] 1) The device receives a voice signal, and the orientation detection module performs orientation detection to determine the user's orientation angle.

[0151] 2) The sound zone positioning module performs sound zone positioning to determine the user's position.

[0152] 3) Determine whether the user is facing the boundary based on the user's orientation angle and position. The determination method is as follows: For a general car cabin, there are usually two or three rows of seats, with two or three seats in each row. The boundary here is defined as the intersection of two seats in the same row. If the user sits on the left side (refer to Figure 1J , taking three rows of seats and two seats in each row as an example, the left position is Figure 1J If the user is sitting on the right side (refer to the left position of the 1st row, 2nd row, and 3rd row), when the user's orientation angle β is greater than 45 degrees, the user is considered to be facing the boundary, otherwise the user is considered not to be facing the boundary. Figure 1J , for example, if there are three rows of seats and two seats in each row, the right position is Figure 1J When the user's orientation angle β is less than -45 degrees, it is considered that the user is facing the boundary, otherwise it is considered that the user is not facing the boundary. If it is detected that the user is facing the boundary, the next step is entered, otherwise only the sound zone where the user is located is enhanced.

[0153] 4) The signal-to-noise ratio detection module detects the signal-to-noise ratio (SNR) in the current cabin. If the SNR in the current cabin is greater than the preset threshold SNR0, it is considered that the cabin is relatively quiet at this time. At this time, the user's sound zone and the adjacent sound zone can be enhanced at the same time. Otherwise, only the user's sound zone is enhanced. The adjacent sound zone refers to other sound zones in the same row as the user's sound zone. Figure 1J If the user sits on the left side of row 1, the adjacent sound zone at this time is the right sound zone of row 1.

[0154] Optionally, if the computing power is sufficient, the signal-to-noise ratio detection module can work in parallel with the orientation detection module and the sound zone positioning module to reduce the latency of the entire link and improve the interactive experience.

[0155] Fig. 1I The illustrated process may be executed by a direction determining device.

[0156] The fifth scenario is to enhance the orientation of the human-to-human interaction assistant. Figure 1K , Figure 1K : is a schematic diagram of a human-to-human interaction assistant orientation enhancement process provided by an embodiment of the present application; specifically, it includes the following steps:

[0157] 1) The device receives a voice signal, and the direction detection module performs direction detection to determine the direction angle β of the speaker.

[0158] 2) The speaker localization module is used to determine the speaker's position coordinates b = [b1, b2].

[0159] 3) The interactive object recognition module recognizes the interactive object, such as interactive object B, based on the speaker's position coordinate b, the speaker's orientation angle β and the preset participant position information.

[0160] 4) The speaker recognition module is used to determine the identity of the current speaker, such as speaker A.

[0161] 5) The speech recognition module is used to recognize the current spoken content, such as "Do you have any suggestions?".

[0162] 6) Generate meeting minutes based on the speaker, the interaction object and the speech content, such as "A said to B, do you have any suggestions?"

[0163] Figure 1K The illustrated process may be executed by a direction determining device.

[0164] Exemplarily, the location information of the participants in the above step 3) is system preset information, and the acquisition method includes at least one of the following methods:

[0165] A. For some meeting scenarios with fixed seats, the name and location coordinates of each person in each position can be entered into the system in advance according to the meeting seat distribution map.

[0166] B. For conference scenarios with non-fixed seats, participants can be registered after they are seated. For example, each person says his or her name and seat number, and the system records and generates the location information of the participants.

[0167] C. If the camera is available, the personnel information at each location can be detected in real time during the meeting, and the location information of the participants can be obtained by combining the face recognition system.

[0168] For example, one implementation of the interactive object recognition module in step 3) is: for N participants in the scene excluding the current speaker, respectively calculate the angle difference between each participant and the speaker Select the smallest angle difference The corresponding participant is regarded as the current interaction object B.

[0169] As another example, another implementation of the interactive object recognition module in step 3) is: for N participants in the scene excluding the current speaker, the angle difference between each participant and the speaker is calculated. Determine the smallest angle difference among the above N angle differences Set the candidate object list to an empty list and iterate over all the angle differences like Then the corresponding participant i is added to the candidate list. is the preset angle difference threshold. When the angle difference corresponding to the participant is less than When , the participant is considered to be in the area directly in front of the speaker. Calculate the distance between each participant in the candidate object list and the speaker, and select the participant with the smallest distance as the current interaction object B.

[0170] For example, reference Figure 1L , Figure 1L is a flowchart of calculating an angle difference provided in an embodiment of the present application; for participant i, calculating the angle difference The specific method is:

[0171] Based on the preset participant location information, the location coordinates t of participant i are obtained i =[x i ,y i ], combined with the speaker's position coordinates b = [b1, b2], calculate the azimuth corresponding to participant i where tan -1 is the inverse tangent function.

[0172] According to the azimuth angle α i Calculate the angle difference with the speaker's orientation angle β

[0173] The above speaker recognition module can at least be implemented based on voiceprint technology. Voiceprint registration can be performed after the participants are seated, such as after each person says his or her name and seat number to register, voiceprint registration is performed at the same time. Voiceprint registration can also be completed before the meeting.

[0174] If a camera is available, the speaker localization module, direction detection module and speaker recognition module can fuse audio and visual information for multimodal detection.

[0175] Exemplarily, the microphone array of the conference room is set in the middle of the conference room, and the direction of arrival (DOA) information can be combined to determine the direction of the participants. When speaking, the speaker's conversation partner can be judged based on the direction and recorded in real time in the meeting minutes.

[0176] The sixth scenario is visible and sayable based on the direction of the sound. Visible and sayable is a voice interaction feature, which refers to what interface elements can be seen on the interface. As long as you say its name, the system will operate the element by simulating a click. For example, there is a Bluetooth switch on the interface. As long as the user says "turn on the Bluetooth switch" or "turn off the Bluetooth switch", the system will simulate clicking the hot area of ​​the Bluetooth switch to change the Bluetooth switch. This function is generally used on devices with screens. After the user wakes up, the device enters the wake-up state, which will trigger this function. The above hot areas are the interactive parts of the interface (also known as clickable places).

[0177] For example, reference Figure 1M , Figure 1M This is a schematic diagram of a visible and audible process based on sound direction provided by an embodiment of the present application; specifically, it includes the following steps:

[0178] 1) The device receives a voice signal, and the direction detection module performs direction detection to determine the direction angle β of the speaker.

[0179] 2) Determine whether the orientation angle falls within the third range. The third range indicates that the user is facing the device. For example, the third range is -45° to 45°. If the orientation angle meets this condition, it can be considered that the device is in the area directly in front of the user. If the condition is met, enter the voice recognition module. Otherwise, the voice is rejected and no response is given.

[0180] 3) The voice recognition module extracts keywords from the voice signal and compares them with the keywords corresponding to the touchable elements on the current screen to determine whether the current screen contains the keyword. If the keyword is contained, the command is executed; otherwise, the voice is rejected and no response is given.

[0181] Optionally, if the computing power is sufficient, the speech recognition module can work in parallel with the direction detection module to reduce the latency of the entire link and improve the interactive experience.

[0182] For example, the user is facing the device and says "Open Settings". The device detects that the user is facing the device (the orientation angle is zero) and there is an interface element "Settings" on the current screen. The device will simulate the user clicking "Settings" and enter the settings page. On the contrary, when the user is facing the device, but there is no element indicated by the sound signal on the current display screen, the control device guides the user to enter the second round of interaction through voice or other means to clarify the operation element. For example, the device asks the user by voice or by outputting text on the screen: "The element is not displayed on the screen. Please indicate again or slide the display screen."

[0183] The system architecture of the embodiment of the present application is described below:

[0184] refer to Figure 1N , Figure 1N It is a system architecture diagram of a direction determination method provided in an embodiment of the present application; in the embodiment of the present application, the system architecture of the direction determination method includes a first device, a first feature calculation unit and a direction detection unit, the first device includes m audio acquisition units, m is a positive integer, the first feature calculation unit is used to calculate the first feature according to the m single-channel voice signals of the first device, and the direction detection unit is used to determine the first direction angle between the user and the first device based on the first feature.

[0185] For example, Figure 1N In the embodiment, the first feature calculation unit and the direction detection unit may be arranged in the first device.

[0186] Also illustratively, Figure 1N In the embodiment, the first feature calculation unit and the direction detection unit can be arranged in the main control device.

[0187] Also illustratively, Figure 1N In the embodiment, the first feature calculation unit can be arranged in the first device, and the direction detection unit can be arranged in the main control device. Alternatively, the first feature calculation unit can be arranged in the main control device, and the direction detection unit can be arranged in the first device.

[0188] refer to Fig.1O , Fig.1O It is a system architecture diagram of another orientation determination method provided in an embodiment of the present application; the system architecture of the orientation determination method includes a first device, a second feature calculation unit, a first feature calculation unit and a orientation detection unit, the first device includes m audio acquisition units, m is a positive integer, the second feature calculation unit is used to calculate the second feature according to the m single-channel voice signals of the first device, the first feature calculation unit is used to calculate the first feature according to the second feature, and the orientation detection unit is used to determine the first orientation angle between the user and the first device based on the first feature.

[0189] For example, Fig.1O In the embodiment, the second feature calculation unit, the first feature calculation unit and the direction detection unit can be arranged in the first device.

[0190] Also illustratively, Fig.1O In the embodiment, the second feature calculation unit, the first feature calculation unit and the direction detection unit can be arranged in the main control device.

[0191] Also illustratively, Fig.1O In the embodiment, the second feature calculation unit and the first feature calculation unit can be arranged in the first device; and the direction detection unit can be arranged in the main control device.

[0192] Alternatively, the direction detection unit may be provided in the first device, and the second feature calculation unit and the first feature calculation unit may be provided in the main control device.

[0193] Alternatively, the second feature calculation unit and the direction detection unit may be arranged in the first device; and the first feature calculation unit may be arranged in the main control device.

[0194] Alternatively, the first feature calculation unit may be arranged in the first device, and the second feature calculation unit and the direction detection unit may be arranged in the main control device.

[0195] Alternatively, the first feature calculation unit and the direction detection unit may be arranged in the first device; and the second feature calculation unit may be arranged in the main control device.

[0196] Alternatively, the second feature calculation unit may be arranged in the first device; and the first feature calculation unit and the direction detection unit may be arranged in the main control device.

[0197] refer to Figure 1P , Figure 1P It is a schematic diagram of the system architecture of a focus device determination method provided in an embodiment of the present application; the system architecture of the focus device determination method includes N first devices (such as first device 1, first device 2, ... first device N), N first feature calculation units (such as first feature calculation unit 1, first feature calculation unit 2, ... first feature calculation unit N) and a processing unit, N is greater than or equal to two, each first device includes m audio acquisition units, m is a positive integer, the first feature calculation unit is used to calculate the first feature according to the m single-channel voice signals of each first device, and the processing unit is used to determine the focus device from the N first devices based on the first feature. Exemplarily, the processing unit is also used to control the focus device to enter the wake-up state.

[0198] Exemplarily, the above-mentioned N first feature calculation units can also be combined into one first feature calculation unit, which is used to process the single-channel voice signals of the N first devices respectively to obtain N first features.

[0199] Exemplarily, when the focus device determining means is N first devices, reference Figure 1P In the embodiment, the first feature calculation unit 1 is provided in the first device 1, and is used to calculate the first feature based on the m single-channel speech signals of the first device 1. The first feature calculation unit 2 is provided in the first device 2, and is used to calculate the first feature based on the m single-channel speech signals of the first device 2. Similarly, the first feature calculation unit N is provided in the first device N, and is used to calculate the first feature based on the m single-channel speech signals of the first device N.

[0200] The processing unit is provided in each first device, that is, the first device 1, the first device 2, ... the first device N are each provided with a processing unit. Further, taking the first device 1 as an example, after the first device 1 calculates the first feature corresponding to itself, the first device 1 determines the focus device from the N first devices based on the first features of the N first devices. When the focus device is the first device 1, the first device 1 enters the awake state. Assuming that the focus device is the first device N, the first device 1 controls the first device N to enter the awake state. Similarly, the working process of the first device 2, ... the first device N is the same as that of the first device 1, and will not be repeated.

[0201] Exemplarily, when the focus device determining apparatus is a master device having data processing capability, reference Figure 1P In the embodiment, the first feature calculation unit 1, the first feature calculation unit 2, ... the first feature calculation unit N are arranged in a main control device, and the processing unit is also arranged in the main control device.

[0202] Exemplarily, when the focus device determining apparatus is N first devices and a master device having data processing capability, reference Figure 1P In the embodiment, the first feature calculation unit 1 is arranged in the first device 1, the first feature calculation unit 2 is arranged in the first device 2, and so on, the first feature calculation unit N is arranged in the first device N. The processing unit is arranged in the main control device.

[0203] Exemplarily, when the focus device determining apparatus is N first devices and a master device having data processing capability, reference Figure 1P In the embodiment, the first feature calculation unit 1, the first feature calculation unit 2, ..., and the first feature calculation unit N are arranged in the main control device. The processing unit is arranged in any one of the N first devices, for example, the first device 2.

[0204] Exemplarily, when the focus device determining apparatus is N first devices and a master device having data processing capability, reference Figure 1P In the embodiment, Y of the N first feature calculation units are set in the main control device, where Y is a positive integer, and the remaining NY first feature calculation units are respectively set in the corresponding first devices, and the processing unit is set in the main control device. For example, Y is 3, and it is assumed that the three first feature calculation units corresponding to the first device 1, the first device 2, and the first device 3 (i.e., the first feature calculation unit 1, the first feature calculation unit 2, and the first feature calculation unit 3) are set in the main control device, and the first feature calculation unit 4 is set in the first device 4, and the first feature calculation unit 5 is set in the first device 5, and so on, the first feature calculation unit N is set in the first device N.

[0205] Figure 1PThe specific arrangement of the first feature calculation unit and the processing unit is not limited to the above example.

[0206] refer to Figure 1Q , Figure 1Q It is a schematic diagram of the system architecture of another focus device determination method provided in an embodiment of the present application; the system architecture of the focus device determination method includes N first devices (such as first device 1, first device 2, ... first device N), N second feature calculation units (such as second feature calculation unit 1, second feature calculation unit 2, ... second feature calculation unit N), N first feature calculation units (such as first feature calculation unit 1, first feature calculation unit 2, ... first feature calculation unit N) and a processing unit, N is greater than or equal to two, each first device includes m audio acquisition units, m is a positive integer, the second feature calculation unit is used to calculate the second feature according to the m single-channel voice signals of each first device, the first feature calculation unit is used to calculate the first feature according to the second feature of each first device, and the processing unit is used to determine the focus device from the N first devices based on the first feature. Exemplarily, the processing unit is also used to control the focus device to enter the wake-up state.

[0207] Exemplarily, the N second feature calculation units may be combined into one second feature calculation unit for processing the single-channel voice signals of the N first devices respectively to obtain the N second features. Similarly, the N first feature calculation units may be combined into one first feature calculation unit for processing the N second features respectively to obtain the N first features.

[0208] Exemplarily, when the focus device determining means is N first devices, reference Figure 1Q , the second feature calculation unit 1 is arranged in the first device 1, and is used to calculate the second feature based on the m single-channel speech signals of the first device 1. The second feature calculation unit 2 is arranged in the first device 2, and is used to calculate the second feature based on the m single-channel speech signals of the first device 2. Similarly, the second feature calculation unit N is arranged in the first device N, and is used to calculate the second feature based on the m single-channel speech signals of the first device N. And the first feature calculation unit 1 is arranged in the first device 1, and is used to calculate the first feature based on the second feature of the first device 1. The first feature calculation unit 2 is arranged in the first device 2, and is used to calculate the first feature based on the second feature of the first device 2. Similarly, the first feature calculation unit N is arranged in the first device N, and is used to calculate the first feature based on the second feature of the first device N.

[0209] The processing unit is provided in each first device, that is, the first device 1, the first device 2, ... the first device N are each provided with a processing unit. Further, taking the first device 1 as an example, after the first device 1 calculates the first feature corresponding to itself, the first device 1 determines the focus device from the N first devices based on the first features of the N first devices. When the focus device is the first device 1, the first device 1 enters the awake state. Assuming that the focus device is the first device N, the first device 1 controls the first device N to enter the awake state. Similarly, the working process of the first device 2, ... the first device N is the same as that of the first device 1, and will not be repeated.

[0210] Exemplarily, when the focus device determining apparatus is a master device having data processing capability, reference Figure 1Q In the embodiment, the second feature calculation unit 1, the second feature calculation unit 2, ... the second feature calculation unit N, the first feature calculation unit 1, the first feature calculation unit 2, ... the first feature calculation unit N are arranged in the main control device, and the processing unit is also arranged in the main control device.

[0211] Exemplarily, when the focus device determining apparatus is N first devices and a master device having data processing capability, reference Figure 1Q In the embodiment, the second feature calculation unit 1 and the first feature calculation unit 1 are arranged in the first device 1, the second feature calculation unit 2 and the first feature calculation unit 2 are arranged in the first device 2, and so on, the second feature calculation unit N and the first feature calculation unit N are arranged in the first device N. The processing unit is arranged in the main control device.

[0212] Exemplarily, when the focus device determining apparatus is N first devices and a master device having data processing capability, reference Figure 1Q In the embodiment, the second feature calculation unit 1, the second feature calculation unit 2, ... the second feature calculation unit N, the first feature calculation unit 1, the first feature calculation unit 2, ... the first feature calculation unit N are arranged in the main control device. The processing unit is arranged in any one of the N first devices, for example, the first device 2.

[0213] Exemplarily, when the focus device determining apparatus is N first devices and a master device having data processing capability, reference Figure 1Q In the embodiment, Y second feature calculation units out of N second feature calculation units and Y first feature calculation units out of N first feature calculation units are set in the main control device, Y is a positive integer, the remaining NY second feature calculation units and the remaining NY first feature calculation units are respectively set in the corresponding first devices, and the processing unit is set in the main control device.

[0214] Figure 1QThe specific arrangement of the second feature calculation unit, the first feature calculation unit and the processing unit is not limited to the above example.

[0215] For example, the above Figure 1P and Figure 1Q In the system architecture illustrated in , the system architecture of the application focus device determination method also includes a communication system for realizing communication between N first devices, and / or for realizing communication between N first devices and a master control device. The communication system may be a Bluetooth communication system or a mobile hotspot communication system (such as a WiFi communication system), etc. For example, a first device may obtain a first feature or a second feature of another first device through the communication system, or the master control device may obtain a first feature or a second feature of a first device through the communication system.

[0216] The following is a detailed description of the method for determining the direction:

[0217] refer to Figure 2 , Figure 2 2 is a flow chart of a method for determining a direction provided in an embodiment of the present application. In the embodiment of the present application, the execution subject takes a direction determining device as an example, and the direction determining method 200 includes the following steps:

[0218] 201. A direction determining device obtains a first feature of a first device.

[0219] Specifically, the first feature is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the first device. M is an integer, such as 1, 2, 3, etc. The M audio acquisition units perform audio acquisition for the same sound source, and one audio acquisition unit performs audio acquisition to obtain a single-channel voice signal. The first feature includes features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source.

[0220] 202. The direction determining device determines a first direction angle between a sound source and a first device based on a first feature.

[0221] In an embodiment of the present application, the first feature is calculated based on the single-channel voice signal of the first device, and the direction angle can be determined based on the first feature. There is no need to pre-train a deep learning model. The direction determination method has low cost, strong generalization, and occupies less computing power resources.

[0222] In a possible implementation manner, the above step 201 specifically includes the following steps:

[0223] 211. The direction determination apparatus obtains M single-channel voice signals from a first device.

[0224] Specifically, the first device is in a listening state (ie, standby state), and when a sound source appears, the audio collection unit in the first device collects audio to obtain a single-channel voice signal. In this embodiment, the direction determination device obtains M single-channel voice signals of the first device from the first device.

[0225] Exemplarily, the first device sends M single-channel voice signals to the direction determination apparatus. Also exemplarily, the direction determination apparatus sends a voice signal request to the first device, and the first device sends M single-channel voice signals to the direction determination apparatus in response to the voice signal request.

[0226] 212. The direction determining device obtains a second feature based on the M single-channel voice signals, where the second feature is a feature of the direct sound and / or reflected sound in the sound signal that is sensitive to direction.

[0227] 213. The direction determining device obtains the first feature based on the second feature, where the first feature is obtained according to the high-frequency energy, the low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature.

[0228] In an embodiment of the present application, M single-channel voice signals are first obtained from the first device, and then the second feature is obtained based on the above M single-channel voice signals, and finally the first feature is obtained according to the high-frequency energy, low-frequency energy or the difference between high-frequency energy and low-frequency energy in the second feature. Since the energy is normalized, the embodiment of the present application is not easily affected by factors such as the distance and orientation between the sound source and the first device when determining the direction angle, and the direction angle has high accuracy.

[0229] In a possible implementation manner, the above step 212 specifically includes the following steps:

[0230] A1. The direction determination device performs the following processing on each single-channel speech signal to obtain a first spectrum corresponding to the single-channel speech signal.

[0231] refer to Figure 3A , Figure 3A : is a schematic diagram of a process of obtaining a first spectrum provided in an embodiment of the present application. The above process specifically includes:

[0232] A11, divide the single-channel speech signal into frames and add windows to obtain Q-segment speech signals [x1…x Q ], Q is greater than or equal to two. The window length is L, and the length of each speech signal is L. In the above calculation, the window function can select a rectangular window, a Hamming window, a Hamming window, etc. according to the actual usage scenario, and the frame shift can also be adjusted as appropriate. For example, the frame shift can be set to L / 2.

[0233] A12, perform Fourier transform on the Q segment voice signal to obtain the time spectrum F0. Exemplarily, when the Q segment voice signal is subjected to P-dimensional Fourier transform,

[0234] A13. Perform signal modulation on the time-frequency spectrum F0 to obtain a second frequency spectrum F2.

[0235] Specifically, the first filter group is used for signal modulation. The time spectrum F0 is used as the input of the first filter group, and the second spectrum is obtained after filtering by the first filter group. in is the filter coefficient matrix of the first filter group, which contains E channels in total (i.e., the first filter group includes E filters, E is greater than or equal to two, and the E filters are in parallel). Each channel corresponds to a different center frequency. Exemplarily, in order to facilitate the calculation of the first feature, the first row of G corresponds to the lowest frequency channel, and the last row of G corresponds to the highest frequency channel. Optionally, the first filter group can be a gammatone filter group, a Mel filter group, a wavelet transform filter group, etc.

[0236] A14, filtering the second spectrum F2 to obtain the first spectrum F1. The filtering is to filter out the features of the direct sound and / or reflected sound that are sensitive to the direction in the second spectrum F2. Exemplarily, filtering the second spectrum F2 to obtain the third spectrum F3, and the first spectrum F1 is obtained based on the third spectrum F3.

[0237] Specifically, the second spectrum F2 is used as the input of the second filter bank (comprising K channels in total, K is greater than or equal to 1), and the third spectrum F3 is obtained after filtering.

[0238] Specifically, the second spectrum F2 is used as the input of K channels, and each channel outputs a sub-spectrum k=1, ..., K; combining the above K sub-spectra to obtain a third spectrum

[0239] Exemplarily, the second filter group is a bandpass filter, and each channel corresponds to different parameter settings to obtain direction-sensitive direct sound and / or reflected sound, and the parameters include at least one of the following: center frequency, quality factor, and order. The voice signal received by the first device includes direct sound and reflected sound at different stages. Exemplarily, the output of the low-frequency bandpass filter corresponds to the direct sound and / or early reflected sound in the voice signal, and the output of the high-frequency bandpass filter corresponds to the late reflected sound in the voice signal.

[0240] Calculating the first spectrum F1 based on the third spectrum F3 specifically includes: summing and transposing the time dimension (i.e., the third dimension) of the third spectrum F3 to obtain the first spectrum

[0241] A2. The direction determining device obtains a second feature based on the M first frequency spectra.

[0242] Exemplarily, the M first spectra F1 are averaged to obtain the second feature. Specifically, the second feature T2 is obtained by adding the corresponding elements of the M first spectra F1 and dividing by M, that is, The first row corresponds to the lowest frequency channel of the first filter group, and the Eth row corresponds to the highest frequency channel of the first filter group.

[0243] In step 213, when the second feature is represented in matrix form, the high-frequency energy in the second feature is used as the first feature. Exemplarily, when the elements are processed row by row, the matrix elements of the rows corresponding to the high-frequency output channels in the second feature are summed to obtain the high-frequency energy.

[0244] As another example, when processing the elements column by column, the matrix elements of the columns corresponding to the high-frequency output channels in the second feature are summed and then inverted to obtain the high-frequency energy.

[0245] In step 213, when the second feature is represented in matrix form, the low-frequency energy in the second feature is used as the first feature. Exemplarily, when the elements are processed row by row, the matrix elements of the row corresponding to the low-frequency output channel in the second feature are summed and then inverted to obtain the low-frequency energy.

[0246] As another example, when processing the elements column by column, the matrix elements of the columns corresponding to the low-frequency output channels in the second feature are summed to obtain the low-frequency energy. For example, assuming that each filter in the second filter group is a low-frequency filter, that is, each matrix element in the second feature corresponds to low-frequency energy, then summing each column of the second feature T2 yields in j=1, ..., K; Calculate the first feature based on the matrix P Among them, the element p in the matrix P j The energy of the sound waves reflected in different stages in space, the larger the j, the later the reflection. Therefore, the output p of the low-frequency output channel j The sum corresponds to the energy of the direct sound and early reflection in the signal. If the user is facing the device, this part of energy is larger, and if the user is facing away from the device, this part of energy is smaller. Therefore, T1 can be used to determine the user's direction.

[0247] In step 213, when the second feature is represented in matrix form, the difference between the high-frequency energy and the low-frequency energy in the second feature is used as the first feature. The above difference can be high-frequency energy / low-frequency energy, low-frequency energy / high-frequency energy, |high-frequency energy-low-frequency energy|, |low-frequency energy-high-frequency energy|, etc., / represents division, and || represents absolute value.

[0248] Exemplarily, the ratio of high frequency energy to low frequency energy in the second feature is used as the first feature. When processing the elements row by row, the low frequency energy part and the high frequency energy part of the second feature are summed up to obtain the low frequency energy and high frequency energy in k=1,...,K. Based on the obtained E L and E H Calculate the ratio of high frequency energy / low frequency energy in k=1,...,K;Sum the obtained high-frequency and low-frequency energy ratio R to obtain the first characteristic The above v can be set according to actual conditions without limitation. For example, v can be E / 2.

[0249] Exemplarily, the ratio of low-frequency energy to high-frequency energy in the second feature is used as the first feature. When the elements are processed column by column, the sum of each column of the second feature T2 is obtained. in j = 1, ..., K. Assume that the first k channels in the second filter group are low-frequency channels; the k+1~Kth channels are high-frequency channels. The first feature is calculated based on the matrix P. The energy is normalized by calculating the ratio, eliminating the difference in pickup energy caused by factors such as distance and device type, thus improving robustness.

[0250] In a possible implementation manner, the above step 201 specifically includes the following steps:

[0251] 214. The direction determining device obtains a second feature from the first device, where the second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction.

[0252] Specifically, after the first device obtains M single-channel voice signals, it first processes the M single-channel voice signals to obtain the second feature, and the direction determination device then obtains the second feature from the first device, which can effectively reduce the computing power occupied by this solution. The specific process of the first device first processing the M single-channel voice signals to obtain the second feature can refer to the relevant description of the above step 212, which will not be repeated here.

[0253] 215. The direction determining device obtains the first feature based on the second feature, where the first feature is obtained according to the high-frequency energy, the low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature.

[0254] In a possible implementation manner, the above step 202 specifically includes the following steps:

[0255] The orientation determining device determines the first orientation angle according to the first feature and a preset corresponding relationship, where the preset corresponding relationship is a corresponding relationship between different first features and the first orientation angles.

[0256] Specifically, a correspondence between the first feature of the first device and the first orientation angle is pre-established, and the preset correspondences of different first devices may be the same or different. The orientation determining device determines the first orientation angle corresponding to the first feature by searching the preset correspondence.

[0257] Exemplarily, when the first device is used for the first time, its preset correspondence may be modified. For example, the user may be required to rotate 90 degrees clockwise from facing the first device and wake up three times, and record the first feature and the corresponding orientation angle calculated by the first device each time to refresh the preset correspondence, so that the preset correspondence is adapted to the current use environment and the accuracy of orientation detection is increased.

[0258] In a possible implementation manner, the above step 202 specifically includes the following steps:

[0259] 221. The orientation determining device determines the second orientation angle according to the first feature and a preset corresponding relationship, where the preset corresponding relationship is a corresponding relationship between different first features and the second orientation angles.

[0260] 222. The orientation determining device determines the first orientation angle based on the second orientation angle and the third orientation angle. The third orientation angle is determined based on an image captured by a camera of the first device.

[0261] In the embodiment of the present application, the second orientation angle is determined by audio. In order to improve the accuracy of the orientation angle between the sound source and the first device, the third orientation angle is determined based on the image captured by the camera of the first device, and the first orientation angle is determined by combining the second orientation angle and the third orientation angle.

[0262] The third orientation angle is determined based on the image captured by the camera of the first device, and visual information or eye movement information can be obtained through the image, and the third orientation angle is determined based on the visual information or eye movement information. The third orientation angle can also be obtained based on other methods (non-audio methods).

[0263] Exemplarily, the user's orientation is determined based on the relevant features of the face recorded by the camera. Specifically, based on the face image captured by the camera, a 3D face model is calculated using face attributes, wherein the face attributes include face key points, and the face key points include eyes, nose or lips; and a third orientation angle is calculated based on the face 3D model and the camera position.

[0264] As another example, the image captured by the camera and the eye tracking technology are used to determine the eye direction of the user to assist in determining the third direction angle.

[0265] In a possible implementation, when it is determined that the first device has a camera, the orientation determination method of the embodiment of the present application is not performed, and the orientation between the first device and the user can be determined directly based on the camera.

[0266] Exemplarily, the essence of this embodiment is to determine the user's orientation through audio and decide whether the device enters the awake state. The above orientation determination method also includes the following steps:

[0267] When the first orientation angle falls within the first range and the first command word table includes the first command word, the orientation determination device controls the first device to enter the wake-up state and executes the first operation corresponding to the first command word. The first command word is a command word included in the text corresponding to the sound signal.

[0268] Specifically, when the first command word is "turn on the air conditioner", the first operation is to control the first device to perform cooling.

[0269] When the first orientation angle falls within the second range and the second command word list includes the first command word, the orientation determination device controls the first device to enter the wake-up state and perform the first operation. The number of command words in the first command word list is greater than the number of command words in the second command word list, and the absolute value of any orientation angle in the second range is less than the absolute value of any orientation angle in the first range.

[0270] Specifically, the specific values ​​of the first range and the second range can be set according to actual conditions. The first command word list or the second command word list can be words such as "turn on the air conditioner" and "open the car window". The specific word list can be set according to actual conditions.

[0271] In an embodiment of the present application, two orientation angle ranges, a first range and a second range, and a first command vocabulary table and a second command vocabulary table with different numbers of command words are used to determine whether to wake up the first device and whether to perform a voice interaction response. When the first orientation angle falls into the first range, it can be considered that the user is most likely interacting with the first device, and the first command vocabulary table is used for command word comparison; when the first orientation angle falls into the second range, it can be considered that the user is least likely interacting with the first device, and the second command vocabulary table is used for command word comparison. The difficulty of entering the wake-up state is adjusted according to the orientation angle. This can reduce the risk of false wake-up, ensure the accuracy of voice interaction, and enhance the user's voice interaction experience.

[0272] Exemplarily, the orientation angle judgment and command word comparison can be performed simultaneously, or the two can be performed one after another. For example, after receiving the voice, the first device can first determine whether the first orientation angle falls into the first range. If it does, the command word range is expanded, that is, the first command word table is selected for command word comparison. When the first command word comparison passes, the first device is controlled to enter the wake-up state and execute the first operation corresponding to the first command word. Among them, when the first command word is included in the first command word table, it is considered that the first command word comparison passes. If the first orientation angle does not fall into the first range, it is determined whether the first orientation angle falls into the second range. If the first orientation angle falls into the second range, the command word range is narrowed, and the second command word table is selected for command word comparison. When the first command word comparison passes, the first device is controlled to enter the wake-up state and execute the first operation corresponding to the first command word. Among them, when the first command word is included in the second command word table, it is considered that the first command word comparison passes. This method can reduce the power consumption of the orientation determination device.

[0273] For another example, the first device may perform a command word comparison after receiving the voice. When the first command word is compared with the first command word list, it is determined whether the first orientation angle falls within the first range. If so, the first device is controlled to enter the wake-up state and execute the first operation corresponding to the first command word. When the first command word is compared with the second command word list, it is determined whether the first orientation angle falls within the second range. If so, the first device is controlled to enter the wake-up state and execute the first operation corresponding to the first command word. If the first command word does not appear in the first command word list or the second command word list, the processing is stopped.

[0274] For another example, after receiving the voice, the first device can simultaneously determine the orientation angle and compare the command word. When the first orientation angle falls into the first range and the first command word table includes the first command word, the orientation determination device controls the first device to enter the wake-up state and execute the first operation corresponding to the first command word. When the first orientation angle falls into the second range and the second command word table includes the first command word, the orientation determination device controls the first device to enter the wake-up state and execute the first operation.

[0275] In a possible implementation manner, the above-mentioned orientation determination method further includes the following steps:

[0276] When the first orientation angle is zero, the orientation determination device controls the first device to enter a wake-up state.

[0277] During the interaction process, the user generally faces the device that he wants to wake up, so choosing to face the device directly to wake up is more in line with the user's usage habits. In the embodiment of the present application, when the first orientation angle is zero, it is considered that the user is interacting with the first device, and the first device can be controlled to enter the wake-up state without the user saying the wake-up word, which simplifies the voice interaction process, makes the voice interaction process smoother and more natural, and realizes the wake-up-free interaction experience under the condition of using only audio information, thereby improving the user's voice interaction experience.

[0278] In a possible implementation, after controlling the first device to enter the awake state, the direction determination method further includes the following steps:

[0279] The direction determination device controls the first device to perform a first operation corresponding to a first command word. The first command word is a command word contained in a text corresponding to the sound signal.

[0280] Specifically, after the first device is controlled to enter the awake state, when the first command word appears in a command word table such as a first command word table or a second command word table, the first device is controlled to execute a first operation corresponding to the first command word.

[0281] Exemplarily, the orientation angle judgment and command word comparison can be performed simultaneously, or the two can be performed one after another. For example, after receiving the voice, the first device can first determine whether the first orientation angle is zero. If it is zero, the first device is controlled to enter the wake-up state. Then the first command word is compared with the command word, and the comparison can be performed using a command word table such as a first command word table or a second command word table. When the first command word comparison passes, the first device is controlled to perform the first operation corresponding to the first command word. Among them, when the first command word is included in a command word table such as a first command word table or a second command word table, it is considered that the first command word comparison passes. Since the power consumption of voice recognition (i.e., command word comparison) is relatively large, the power consumption of the orientation determination device can be reduced by first determining the orientation angle and then performing command word comparison.

[0282] For another example, the first device may perform a command word comparison after receiving the voice. When the first command word comparison passes, it determines whether the first orientation angle is zero. If it is zero, the first device is controlled to enter a wake-up state and execute the first operation corresponding to the first command word.

[0283] For another example, after receiving the voice, the first device can simultaneously determine the orientation angle and compare the command word. When the first orientation angle is zero and the command word comparison passes, the orientation determination device controls the first device to enter the wake-up state and execute the first operation corresponding to the first command word.

[0284] In addition, when the text corresponding to the sound signal does not include the first command word, the first device waits for the user to input the next sound signal in the wake-up state, performs voice recognition processing on the next sound signal, and when the text corresponding to the next sound signal includes the first command word, executes the first operation corresponding to the first command word.

[0285] In order to better illustrate the process of waking up the first device, the following takes the simultaneous determination of the orientation angle and the comparison of the command words (ie, voice recognition) as an example, referring to Figure 3B , Figure 3B This is a schematic diagram of a process for waking up a first device provided in an embodiment of the present application.

[0286] For example, the first range is (-45, 45) and the second range is less than -45 or greater than 45. Figure 3B In the process, the first device performs audio collection to obtain M single-channel voice signals, calculates a first feature based on the M single-channel voice signals, and calculates a first orientation angle β in combination with a preset corresponding relationship and the first feature.

[0287] If β=0, it is considered that the user is interacting with the first device, triggering the wake-up-free mode to directly enter the wake-up state; then call the comparison result of the command word, if the command word comparison passes, execute the first operation corresponding to the first command word, otherwise wait for the next voice signal input.

[0288] If -45<β<45, it is considered that the user is most likely interacting with the first device, and the command word list is switched to the first command word list. Detect whether the current voice hits the current command word list. If it hits, the device enters the awake state and executes the command. If not, no response is given.

[0289] If β<-45 or β>45, it is considered that the user is interacting with the first device with a small probability, and the command word list is switched to the second command word list. Check whether the current voice hits the current command word list. If it hits, the device enters the wake-up state and executes the command. If it does not hit, it does not respond.

[0290] The embodiment of the present application can calculate the orientation angle of the user relative to the device based on the voice signal received by the device, and infer the probability of the user interacting with the device based on the orientation angle, thereby adjusting the difficulty of the device entering the wake-up state.

[0291] In a possible implementation, when the first device enters the wake-up state, user feedback may be provided through images, audio, etc., including but not limited to user experience (UX) animations, response words, prompt sounds, etc. Figure 3C When speaking toward the first device, the UX interface shows feedback such as a virtual image looking in the direction of the user, and the wake-up is triggered. Figure 3D,When the user is not speaking directly to the first device, the virtual image looks forward or hides and does not respond to the user.

[0292] In the embodiments of the present application, the numbering of the steps does not limit the execution order between the steps.

[0293] The following is a detailed description of the method for determining the focus device:

[0294] refer to Figure 4 , Figure 4 4 is a flow chart of a focus device determination method provided by an embodiment of the present application; the execution subject takes a focus device determination apparatus as an example, and the focus device determination method 400 includes the following steps:

[0295] 401. A focus device determining apparatus obtains N first features corresponding to N first devices.

[0296] Specifically, the first feature of the first device is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the first device. N first devices perform audio acquisition for the same sound source. The first feature includes features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source. N is greater than or equal to two, and M is an integer.

[0297] 402. A focus device determining apparatus determines a focus device from N first devices based on N first features.

[0298] In an embodiment of the present application, the corresponding first feature is calculated according to the single-channel voice signal of each first device, and the focus device can be determined from the N first devices based on the N first features. This method does not require pre-training of a deep learning model. The focus device determination method has low cost, strong generalization, and occupies less computing resources.

[0299] In a possible implementation manner, the above step 401 specifically includes the following steps:

[0300] 411. The focus device determining apparatus obtains M single-channel voice signals from the first device.

[0301] 412. The focus device determining means determines a second feature of the first device based on the M single-channel voice signals. The second feature is a feature of the direct sound and / or reflected sound in the sound signal that is sensitive to direction.

[0302] 413. The focus device determining means obtains the first feature based on the second feature, where the first feature is obtained according to high-frequency energy, low-frequency energy, or the difference between high-frequency energy and low-frequency energy in the second feature.

[0303] In the embodiment of the present application, M single-channel voice signals are first obtained from the first device, and then the second feature is obtained based on the above M single-channel voice signals, and finally the first feature is obtained based on the high-frequency energy, low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature. Since the energy is normalized, this solution is not easily affected by factors such as the distance and orientation between the sound source and the first device when determining the focal device, and the focal device has high accuracy. The M single-channel voice signals of each first device are processed using steps 411 to 413 to obtain the first feature of each first device.

[0304] In a possible implementation, the above step 412 specifically includes the following steps:

[0305] B1. The focus device determination device performs the following processing on each single-channel speech signal to obtain a first spectrum corresponding to the single-channel speech signal.

[0306] The single-channel voice signal is framed and windowed to obtain Q-segment voice signals, where Q is greater than or equal to 2. The Q-segment voice signal is Fourier transformed to obtain a time-frequency spectrum. The time-frequency spectrum is signal modulated to obtain a second spectrum. The second spectrum is filtered to obtain a first spectrum.

[0307] B2. The focus device determination device obtains a second feature based on the M first spectra.

[0308] For the detailed description of steps 411 to 413, please refer to the above steps 211 to 213, which will not be repeated here.

[0309] In a possible implementation manner, the above step 401 specifically includes the following steps:

[0310] 414. The focus device determining device obtains a second feature from the first device. The second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction.

[0311] 415. The focus device determining means obtains the first feature based on the second feature, where the first feature is obtained according to high-frequency energy, low-frequency energy, or the difference between high-frequency energy and low-frequency energy in the second feature.

[0312] In the embodiment of the present application, the second feature of the direct sound and / or the reflected sound that is sensitive to the direction in the sound signal is first obtained from the first device, and then the first feature is obtained based on the high-frequency energy, low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature. The second feature is first processed in the first device, which can effectively reduce the computing power occupied by the present solution.

[0313] For the detailed description of step 414 to step 415, please refer to the above-mentioned step 214 to step 215, which will not be repeated here.

[0314] In a possible implementation, the above step 402 specifically includes the following steps:

[0315] The focus device determining means determines, as a focus device, a first device corresponding to at least one first feature having a top feature value among the N first features.

[0316] In an embodiment of the present application, the first device corresponding to the first feature with the highest feature value among the N first features can be determined as the focus device, or the first devices corresponding to the x first features with the highest feature values ​​among the N first features can be determined as the focus devices, where x is greater than or equal to two, for example, x is 2, 3, 5, etc.

[0317] In a possible implementation manner, the focus device determination method 400 further includes the following steps:

[0318] The focus device determining means controls the focus device to enter a wake-up state.

[0319] In an embodiment of the present application, after the focus device is determined, the focus device can be controlled to enter the wake-up state without the user saying a wake-up word, thereby simplifying the voice interaction process and improving the user's voice interaction experience.

[0320] In a possible implementation manner, after controlling the focus device to enter the awake state, the focus device determination method further includes the following steps:

[0321] The focus device determining means controls the focus device to execute a first operation corresponding to a first command word. The first command word is a command word contained in a text corresponding to the sound signal.

[0322] In an embodiment of the present application, after the focus device is controlled to enter a wake-up state without the user saying a wake-up word, the focus device is controlled to execute a first operation corresponding to a command word contained in a text corresponding to a sound signal, in response to a voice interaction command of the user.

[0323] Specifically, the corresponding text is determined according to the sound signal, and then the text is compared with the command word list of the focus device to determine whether the command word list includes the words in the text, that is, the command word. If the command word is included, the focus device is controlled to perform the operation corresponding to the command word. For example, if the first command word is "turn on the air conditioner", the first operation is to turn on the air conditioner for cooling. In addition, the command word lists corresponding to the N first devices can be the same or different, and this is not limited.

[0324] Exemplarily, the focus device determination and command word comparison can be performed simultaneously, or the two can be performed successively. For example, the first feature can be determined first, so that the focus device determination device determines the focus device according to the first feature. Then, the command word comparison is performed on the first command word, and when the first command word comparison passes, the focus device is controlled to perform the first operation corresponding to the first command word. Since the power consumption of speech recognition (i.e., command word comparison) is relatively large, the power consumption of the focus device determination device can be reduced by first determining the focus device and then performing command word comparison.

[0325] For another example, a command word comparison is first performed based on the voice and the command word list of the first device. When the first command word comparison passes, the focus device is determined. At this time, the focus device is determined from the first device that passes the first command word comparison.

[0326] For another example, the focus device judgment and command word comparison can be performed simultaneously, that is, the command word comparison is performed according to the voice and the command word list of the first device, and the focus device is determined from the N first devices according to the first feature. When the first command word comparison passes, the focus device determination device controls the focus device to enter the wake-up state and execute the first operation corresponding to the first command word.

[0327] In addition, when the text corresponding to the sound signal does not include the first command word, the focus device waits for the user to input the next sound signal in the awake state, performs voice recognition processing on the next sound signal, and when the text corresponding to the next sound signal includes the first command word, executes the first operation corresponding to the first command word.

[0328] In the embodiments of the present application, the numbering of the steps does not limit the execution order between the steps.

[0329] In a possible implementation, when the focus device enters the wake-up state, user feedback may be provided through images, audio, etc., including but not limited to UX animations, response words, prompt sounds, etc. For example, when there are multiple first devices, the first device (i.e., the focus device) that the user is facing lights up the screen or displays a virtual image, triggering the wake-up-free state; other first devices turn off the screen or hide the virtual image, and do not respond.

[0330] Refer to Table 1 below. In Table 1, the first device takes the central control screen of a liquid crystal display (LCD) as an example. The facing device refers to the device that the user is facing, that is, the device that is expected to be awakened. The non-facing device is a device that is not directly in front of the user, that is, the device that is not expected to be awakened. The last column in Table 1 shows the accuracy of the focus device determination method of the present application in different scenarios and different device states. It can be seen that the focus device determination method proposed in the embodiment of the present application does not require pre-training and has strong generalization. There is no need to predict the speaker's position, distance and other information. Robust to distance and noise. It is not limited by the number of device channels, has low computing power occupancy, and has a small communication burden, so it can achieve long-distance running on the end side. Long-distance running on the end side means that the computing power of the terminal device is usually very limited, and the power consumption requirements are very high. Only algorithms with low computational complexity can perform long-distance running on the end side, that is, the algorithm corresponding to the focus device determination method of the embodiment of the present application is run on the terminal device for a long time.

[0331] Table 1 Device wake-up indicator table

[0332]

[0333] The method of the embodiment of the present application is described in detail above, and the device provided by the embodiment of the present application is introduced below.

[0334] Figure 5 A schematic diagram of a possible device structure provided for an embodiment of the present application. In an embodiment of the present application, the device may be an electronic device, or a module (such as a chip) used in an electronic device.

[0335] Figure 5 The device 500 shown may be a device for determining an orientation angle, and is used to implement the functions of the above-mentioned embodiment of the method for determining an orientation angle, and thus can also achieve the beneficial effects possessed by the above-mentioned embodiment of the method for determining an orientation angle. Figure 5 As shown, the orientation angle determination device includes an acquisition module 501 and a determination module 502. The orientation angle determination device is used to implement the above Figure 2 Alternatively, the orientation angle determination device may include a method for implementing the above Figure 2 A module for any function or operation in the embodiment of the method for determining the orientation angle shown in the figure may be implemented in whole or in part by software, hardware, firmware or any combination thereof.

[0336] When the heading angle determining device is used to achieve Figure 2When the function in the method embodiment shown is used, the acquisition module 501 is used to obtain the first feature of the first device. The above-mentioned first feature is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the first device. M is an integer. The M audio acquisition units perform audio acquisition for the same sound source. The first feature includes the characteristics of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source. The determination module 502 is used to determine the first orientation angle between the sound source and the first device based on the first feature.

[0337] In an embodiment of the present application, the direction determination device calculates the first feature based on the single-channel voice signal of the first device, and can determine the direction angle based on the first feature. There is no need to pre-train a deep learning model. The direction determination method has low cost, strong generalization, and occupies less computing power resources.

[0338] In one possible implementation, reference Figure 5 The above-mentioned direction determination device also includes a processing module 503.

[0339] The processing module 503 is used to control the first device to enter the wake-up state and execute the first operation corresponding to the first command word when the first orientation angle falls into the first range and the first command word table includes the first command word. The first command word is the command word included in the text corresponding to the sound signal.

[0340] The processing module 503 is further configured to control the first device to enter a wake-up state and perform a first operation when the first orientation angle falls within the second range and the second command word list includes the first command word. The number of command words in the first command word list is greater than the number of command words in the second command word list, and the absolute value of any orientation angle in the second range is less than the absolute value of any orientation angle in the first range.

[0341] In a possible implementation manner, the processing module 503 is further configured to control the first device to enter a wake-up state when the first orientation angle is zero.

[0342] In a possible implementation, after controlling the first device to enter the wake-up state, the processing module 503 is further used to control the first device to execute a first operation corresponding to a first command word, where the first command word is a command word contained in the text corresponding to the sound signal.

[0343] In a possible implementation, when the first device enters the wake-up state, user feedback can be given through images, audio, etc., including but not limited to user experience (UX) animations, response words, prompt sounds, etc. Correspondingly, the above-mentioned direction determination device also includes a display module, which is used to give user feedback through images when the first device enters the wake-up state. Optionally, the above-mentioned direction determination device also includes an output module, which is used to give user feedback through audio when the first device enters the wake-up state, and the output module can be a speaker.

[0344] Figure 5 The device 500 shown may also be a focus device determination device, which is used to implement the functions of the above focus device determination method embodiment, and thus can also achieve the beneficial effects possessed by the above focus device determination method embodiment. Figure 5 The focus device determination device includes an acquisition module 501 and a determination module 502. The focus device determination device is used to implement the above Figure 4 Alternatively, the focus device determination apparatus may include a device for implementing the above Figure 4 A module for any function or operation in the focus device determination method embodiment shown in the figure may be implemented in whole or in part by software, hardware, firmware or any combination thereof.

[0345] When the focus device determines the device used to achieve Figure 4 When the function in the method embodiment shown is used, the acquisition module 501 is used to obtain N first features corresponding to N first devices. The first feature of the above-mentioned first device is obtained based on M single-channel voice signals corresponding to M audio acquisition units in the first device. The N first devices perform audio acquisition for the same sound source. The first feature includes the features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source. N is greater than or equal to two, and M is an integer.

[0346] The determination module 502 is configured to determine a focus device from N first devices based on the N first features.

[0347] In an embodiment of the present application, the focus device determination device calculates the corresponding first feature based on the single-channel voice signal of each first device, and can determine the focus device from N first devices based on the N first features. This method does not require pre-training of a deep learning model. The focus device determination method has low cost, strong generalization, and occupies less computing resources.

[0348] In a possible implementation manner, the focus device determination apparatus further includes a processing module 503 .

[0349] The processing module 503 is used to control the focus device to enter the awake state.

[0350] In a possible implementation, after the focus device is controlled to enter the awake state, the processing module 503 is further used to control the focus device to execute a first operation corresponding to a first command word, where the first command word is a command word contained in the text corresponding to the sound signal.

[0351] In a possible implementation, when the focus device enters the wake-up state, user feedback can be given through images, audio, etc., including but not limited to UX animations, response words, prompt sounds, etc. Correspondingly, the focus device determination device also includes a display module, which is used to provide user feedback through images when the focus device enters the wake-up state. Optionally, the focus device determination device also includes an output module, which is used to provide user feedback through audio when the focus device enters the wake-up state, and the output module can be a speaker.

[0352] The present application also provides an electronic device, referring to Figure 6 , Figure 6 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 600 includes a memory 601, a processor 602, a communication interface 604 and a bus 603. The memory 601, the processor 602 and the communication interface 604 are connected to each other through the bus 603.

[0353] Optionally, the electronic device 600 further includes a display (not shown), which is connected to the memory 601, the processor 602, and the communication interface 604 via the bus 603. The display is used to output images. Optionally, the electronic device further includes an output module (not shown), which is connected to the memory 601, the processor 602, and the communication interface 604 via the bus 603, and the output module is used to output audio. The output module can be a speaker.

[0354] The memory 601 may be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 601 may store a program. When the program stored in the memory 601 is executed by the processor 602, the processor 602 and the communication interface 604 are used to execute the various steps of the method for determining the heading angle or the method for determining the focus device of any embodiment of the present application.

[0355] Processor 602 can adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the functions required to be performed by the units in the orientation angle determination device or the focus device determination device of any embodiment of the present application, or to execute the orientation angle determination method or the focus device determination method of any embodiment of the present application.

[0356] The processor 602 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the method for determining the orientation angle or the method for determining the focus device of any embodiment of the present application may be completed by an integrated logic circuit of hardware or software instructions in the processor 602. The above-mentioned processor 602 may also be a general-purpose processor, a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method for determining the orientation angle or the method for determining the focus device in combination with any embodiment of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 601, and the processor 602 reads the information in the memory 601, and combines its hardware to complete the functions required to be performed by the units included in the orientation angle determination device or the focus device determination device of any embodiment of the present application, or executes the orientation angle determination method or the focus device determination method of any embodiment of the present application.

[0357] The communication interface 604 uses a transceiver such as, but not limited to, a transceiver to implement communication between the electronic device 600 and other devices or a communication network. For example, the first feature can be acquired through the communication interface 604.

[0358] The bus 603 may include a path for transmitting information between various components of the electronic device 600 (e.g., the memory 601, the processor 602, and the communication interface 604). In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0359] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0360] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0361] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0362] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for determining a direction, characterized in that: include: Obtaining a first characteristic of a first device; The first feature is obtained according to M single-channel voice signals corresponding to M audio collection units in the first device, where M is an integer, and the M audio collection units collect audio for the same sound source, and the first feature includes features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source; A first orientation angle between the sound source and the first device is determined based on the first feature.

2. The method according to claim 1, characterized in that The obtaining the first characteristic of the first device includes: Acquire the M single-channel voice signals from the first device; Obtaining a second feature based on the M single-channel speech signals, where the second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction; The first feature is obtained based on the second feature, and the first feature is obtained according to the high-frequency energy, the low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature.

3. The method according to claim 2, characterized in that The obtaining a second feature based on the M single-channel speech signals includes: Perform the following processing on each of the single-channel speech signals to obtain a first spectrum corresponding to the single-channel speech signal: Performing frame windowing on the single-channel voice signal to obtain Q-segment voice signals, where Q is greater than or equal to two; Performing Fourier transform on the Q segment sub-voice signal to obtain a time-frequency spectrum; Performing signal modulation on the time-frequency spectrum to obtain a second frequency spectrum; Performing filtering processing on the second spectrum to obtain the first spectrum; The second feature is obtained based on M first spectra.

4. The method according to claim 1, characterized in that: The obtaining the first characteristic of the first device includes: Acquire a second feature from the first device, where the second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction; The first feature is obtained based on the second feature, and the first feature is obtained according to the high-frequency energy, the low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature.

5. The method according to any one of claims 1 to 4, characterized in that: The determining a first orientation angle between the sound source and the first device based on the first feature includes: Determine a second orientation angle according to the first feature and a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between different first features and second orientation angles; The first orientation angle is determined based on the second orientation angle and a third orientation angle, and the third orientation angle is determined based on an image captured by a camera of the first device.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: When the first orientation angle falls within the first range and the first command word table includes a first command word, controlling the first device to enter a wake-up state and executing a first operation corresponding to the first command word; the first command word is a command word included in the text corresponding to the sound signal; When the first orientation angle falls into the second range and the second command word list includes the first command word, the first device is controlled to enter a wake-up state and perform the first operation; the number of command words in the first command word list is greater than the number of command words in the second command word list, and the absolute value of any orientation angle in the second range is less than the absolute value of any orientation angle in the first range.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: When the first orientation angle is zero, the first device is controlled to enter a wake-up state.

8. The method according to claim 7, characterized in that After controlling the first device to enter the awake state, the method further includes: The first device is controlled to execute a first operation corresponding to a first command word, where the first command word is a command word contained in a text corresponding to the sound signal.

9. A method for determining a focus device, characterized in that: include: Obtaining N first features corresponding to N first devices, where the first features of the first devices are obtained according to M single-channel voice signals corresponding to M audio acquisition units in the first devices, the N first devices perform audio acquisition for the same sound source, and the first features include features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source; N is greater than or equal to two, and M is an integer; A focus device is determined from the N first devices based on the N first features.

10. The method according to claim 9, characterized in that The determining a focus device from the N first devices based on the N first features includes: A first device corresponding to at least one first feature with a higher feature value among the N first features is determined as the focus device.

11. The method according to claim 9 or 10, characterized in that: The obtaining of N first features corresponding to the N first devices includes: Acquire the M single-channel voice signals from the first device; Determine a second feature of the first device based on the M single-channel voice signals, where the second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction; The first feature is obtained based on the second feature, and the first feature is obtained according to the high-frequency energy, the low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature.

12. The method according to claim 11, characterized in that The determining the second feature of the first device based on the M single-channel voice signals includes: Perform the following processing on each of the single-channel speech signals to obtain a first spectrum corresponding to the single-channel speech signal: Performing frame windowing on the single-channel voice signal to obtain Q-segment voice signals, where Q is greater than or equal to two; Performing Fourier transform on the Q segment sub-voice signal to obtain a time-frequency spectrum; Performing signal modulation on the time-frequency spectrum to obtain a second frequency spectrum; Performing filtering processing on the second spectrum to obtain the first spectrum; The second feature is obtained based on M first spectra.

13. The method according to claim 9 or 10, characterized in that: The obtaining of N first features corresponding to the N first devices includes: Acquire a second feature from the first device, where the second feature is a feature of direct sound and / or reflected sound in the sound signal that is sensitive to direction; The first feature is obtained based on the second feature, and the first feature is obtained according to the high-frequency energy, the low-frequency energy, or the difference between the high-frequency energy and the low-frequency energy in the second feature.

14. The method according to any one of claims 9 to 13, characterized in that: The method further comprises: Control the focus device to enter a wake-up state.

15. The method according to claim 14, characterized in that After controlling the focus device to enter the awake state, the method further includes: The focus device is controlled to execute a first operation corresponding to a first command word, where the first command word is a command word contained in a text corresponding to the sound signal.

16. A device for determining a direction, characterized in that: include: An acquisition module, configured to acquire a first feature of a first device; The first feature is obtained according to M single-channel voice signals corresponding to M audio collection units in the first device, where M is an integer, and the M audio collection units collect audio for the same sound source, and the first feature includes features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source; A determination module is used to determine a first orientation angle between the sound source and the first device based on the first feature.

17. The device according to claim 16, characterized in that The device also includes: a processing module, configured to control the first device to enter a wake-up state and execute a first operation corresponding to the first command word when the first orientation angle falls within a first range and the first command word table includes a first command word; the first command word is a command word included in the text corresponding to the sound signal; The processing module is also used to control the first device to enter a wake-up state and perform the first operation when the first orientation angle falls into the second range and the second command vocabulary includes the first command word; the number of command words in the first command vocabulary is greater than the number of command words in the second command vocabulary, and the absolute value of any orientation angle in the second range is less than the absolute value of any orientation angle in the first range.

18. The device according to claim 16, characterized in that The device also includes: The processing module is used to control the first device to enter a wake-up state when the first orientation angle is zero.

19. The device according to claim 18, characterized in that After controlling the first device to enter the wake-up state, the processing module is further used to control the first device to execute a first operation corresponding to a first command word, where the first command word is a command word contained in the text corresponding to the sound signal.

20. A focus device determination device, characterized in that: include: an acquisition module, configured to acquire N first features corresponding to N first devices, wherein the first features of the first devices are obtained according to M single-channel voice signals corresponding to M audio acquisition units in the first devices, the N first devices perform audio acquisition for the same sound source, and the first features include features of direct sound and / or reflected sound that are sensitive to direction in the sound signal generated by the sound source; N is greater than or equal to two, and M is an integer; A determination module is used to determine a focus device from the N first devices based on the N first features.

21. The device according to claim 20, characterized in that The device also includes: The processing module is used to control the focus device to enter the awake state.

22. The device according to claim 21, characterized in that After controlling the focus device to enter the awake state, the processing module is further used to control the focus device to execute a first operation corresponding to a first command word, where the first command word is a command word contained in the text corresponding to the sound signal.

23. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the method according to any one of claims 1 to 15.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 15.

Images