Sound source direction determination method, device, electronic device and storage medium

Abstract

The present application relates to a method, device, electronic device and storage medium for determining the direction of a sound source, and belongs to the field of audio processing technology. The method comprises: based on M pickup directions, M voice signals are picked up, each voice signal corresponds to a pickup direction, and the M voice signals are used to wake up the terminal, where M is an integer greater than 1; based on the M voice signals, M wake-up parameters and initial sound source directions are determined, and the M wake-up parameters are used to indicate the contribution of the M voice signals to waking up the terminal; based on the M wake-up parameters and the M pickup directions, the initial sound source direction is adjusted to obtain the target sound source direction. This method can improve the accuracy of determining the target sound source direction.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a method, device, electronic device, and storage medium for determining a sound source direction. Background Art

[0002] Currently, terminals have voice recognition capabilities that can identify voice control commands in user voice signals and then execute the corresponding operations. To save power, the terminal enters a dormant state before recognizing voice control commands. Only after receiving a wake-up command does it wake up and collect voice signals, and then perform voice signal recognition. To improve the clarity of the collected voice signals, the terminal can use the wake-up command voice signal to locate the direction of the sound source, and then collect voice signals based on this sound source direction.

[0003] In related technologies, when a user wakes up the terminal through a voice signal, the terminal picks up voice signals from multiple directions. The voice signals from multiple directions are used to wake up the terminal, and based on the voice signals from multiple directions, the sound source is located. Based on the pickup direction obtained by positioning, the voice signal in that direction is beamformed, thereby recognizing the voice signal after beamforming.

[0004] However, in the above method, when the ambient noise is large, the sound pickup direction obtained by positioning is inaccurate. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, electronic device, and storage medium for determining the direction of a sound source, which can improve the accuracy of determining the direction of a sound source. The technical solution is as follows:

[0006] According to one aspect of an embodiment of the present application, a method for determining a sound source direction is provided, the method comprising:

[0007] Based on M pickup directions, M voice signals are picked up, each voice signal corresponds to a pickup direction, and the M voice signals are used to wake up the terminal, where M is an integer greater than 1;

[0008] Determining, based on the M voice signals, M wake-up parameters and an initial sound source direction, the M wake-up parameters being used to indicate a contribution degree of the M voice signals to waking up the terminal;

[0009] Based on the M wake-up parameters and the M sound pickup directions, the initial sound source direction is adjusted to obtain a target sound source direction.

[0010] In a possible implementation, adjusting the initial sound source direction based on the M wake-up parameters and the M sound pickup directions to obtain a target sound source direction includes:

[0011] Based on the M wake-up parameters and the M sound pickup directions, selecting N sound pickup directions from the M sound pickup directions, where the N wake-up parameters of the N sound pickup directions are the first N wake-up parameters of the M wake-up parameters arranged from largest to smallest, and N is an integer less than M and greater than 1;

[0012] Based on the N wake-up parameters and the N sound pickup directions, the initial sound source direction is adjusted to determine the target sound source direction.

[0013] In another possible implementation, adjusting the initial sound source direction based on the N wake-up parameters and the N sound pickup directions to determine the target sound source direction includes:

[0014] Based on the N sound pickup directions, determining a positional relationship between the N sound pickup directions;

[0015] If the relationship between the initial sound source direction and the position meets a preset condition, determining an adjustment angle based on the N wake-up parameters and the N sound pickup directions; adjusting the initial sound source direction based on the adjustment angle to obtain the target sound source direction;

[0016] If the initial sound source direction and the position relationship do not satisfy a preset condition, the target sound source direction is determined to be the sound pickup direction corresponding to the maximum wake-up parameter.

[0017] In another possible implementation, if N is greater than 2 and less than M, the angle between any two adjacent pickup directions in the M pickup directions is a preset angle; if the initial sound source direction and the position relationship meet a preset condition, determining the adjustment angle based on the N wake-up parameters and the N pickup directions includes:

[0018] The positional relationship is that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not a boundary pickup direction of the first pickup range, the first pickup range is a pickup range composed of the N pickup directions, and the boundary pickup direction is two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle. If the initial sound source direction is within the first pickup range, determine a first weight and a first angle of the pickup direction with the smallest angle to the initial sound source direction;

[0019] The adjustment angle is determined based on the first weight and the first angle.

[0020] In another possible implementation, if N is greater than 2 and less than M, the angle between any two adjacent pickup directions in the M pickup directions is a preset angle; if the initial sound source direction and the position relationship meet a preset condition, determining the adjustment angle based on the N wake-up parameters and the N pickup directions includes:

[0021] The positional relationship is that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is one of the boundary pickup directions of the first pickup range, the first pickup range is a pickup range composed of the N pickup directions, and the boundary pickup directions are two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle; if the initial sound source direction is within the second pickup range, the second pickup range is a pickup range composed of other pickup directions except the pickup direction corresponding to the maximum wake-up parameter, and second weights of the other pickup directions are determined;

[0022] The adjustment angle is determined based on the second weight.

[0023] In another possible implementation, if N is equal to 2; if the relationship between the initial sound source direction and the position satisfies a preset condition, determining the adjustment angle based on the N wake-up parameters and the N sound pickup directions includes:

[0024] The positional relationship is that the N pickup directions are adjacent, and the pickup direction corresponding to the maximum wake-up parameter is any one of the N pickup directions. If the initial sound source direction is within a first pickup range, the first pickup range is a pickup range composed of the N pickup directions, and a third weight and a second angle are determined for the pickup direction having the smallest angle with the initial sound source direction.

[0025] The adjustment angle is determined based on the third weight and the second angle.

[0026] In another possible implementation, if N is equal to M, the method further includes:

[0027] If the initial sound source direction is within the sound pickup range, and the first sound pickup range is a sound pickup range consisting of the N sound pickup directions, determining a fourth weight and a third angle of the sound pickup direction having the smallest angle with the initial sound source direction;

[0028] The adjustment angle is determined based on the fourth weight and the third angle.

[0029] In another possible implementation, the angle between any two adjacent sound pickup directions in the M sound pickup directions is a preset angle; if the initial sound source direction and the position relationship meet a preset condition, determining the adjustment angle based on the N wake-up parameters and the N sound pickup directions includes:

[0030] The positional relationship is that there are at least a first pair of two adjacent pickup directions and a second pair of two adjacent pickup directions among the N pickup directions, the angle between the first pair of two adjacent pickup directions is the preset angle, the angle between the second pair of two adjacent pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is any pickup direction of the first pair of adjacent pickup directions; if the initial sound source direction is within a third pickup range, the third pickup range is a pickup range composed of the first pair of two adjacent pickup directions, and a fifth weight and a fourth angle are determined between adjacent pickup directions of the pickup direction corresponding to the maximum wake-up parameter;

[0031] The adjustment angle is determined based on the fifth weight and the fourth angle.

[0032] In another possible implementation, the angle between any two adjacent sound pickup directions in the M sound pickup directions is a preset angle; and the method further includes:

[0033] The positional relationship is that among the N sound pickup directions, there are at least a first pair of two adjacent sound pickup directions and a second pair of two adjacent sound pickup directions, the angle between the first pair of two adjacent sound pickup directions is the preset angle, the angle between the second pair of two adjacent sound pickup directions is greater than the preset angle, and the sound pickup direction corresponding to the maximum wake-up parameter is not a sound pickup direction in the first pair of adjacent sound pickup directions; if the initial sound source direction is within a fourth sound pickup range, the fourth sound pickup range is the sound pickup range formed by the first pair of two adjacent sound pickup directions, and the sum of the wake-up parameters of the adjacent sound pickup directions is determined;

[0034] If the sum of the wake-up parameters is not less than the product of the maximum wake-up parameter and a preset coefficient, the target sound source direction is determined to be the initial sound source direction.

[0035] In another possible implementation, the angle between any two adjacent sound pickup directions in the M sound pickup directions is a preset angle; and the method further includes:

[0036] If the positional relationship is that there is at least one pair of adjacent two pickup directions among the N pickup directions, the angle between them is the preset angle, and the step of determining the adjustment angle based on the N wake-up parameters and the N pickup directions if the initial sound source direction and the positional relationship meet the preset conditions is executed.

[0037] In another possible implementation, the angle between any two adjacent sound pickup directions in the M sound pickup directions is a preset angle; and the method further includes:

[0038] If the positional relationship is such that no two adjacent sound pickup directions in the N sound pickup directions have an angle equal to the preset angle, the target sound source direction is determined to be the initial sound source direction.

[0039] In another possible implementation, the method further includes:

[0040] If the sound pickup direction with the smallest angle with the initial sound source direction is not the sound pickup direction corresponding to the largest wake-up parameter, the step of adjusting the initial sound source direction based on the M wake-up parameters to obtain the target sound source direction is performed.

[0041] In another possible implementation, the method further includes:

[0042] If the sound pickup direction having the smallest angle with the initial sound source direction is the sound pickup direction corresponding to the maximum wake-up parameter, the target sound source direction is determined to be the initial sound source direction.

[0043] According to one aspect of an embodiment of the present application, a device for determining a sound source direction is provided, the device comprising:

[0044] A pickup module, configured to pick up M voice signals based on M pickup directions, each voice signal corresponding to a pickup direction, and the M voice signals are used to wake up the terminal, where M is an integer greater than 1;

[0045] A first determination module is configured to determine M wake-up parameters and an initial sound source direction based on the M voice signals, wherein the M wake-up parameters are used to indicate the contribution of the M voice signals to waking up the terminal;

[0046] An adjustment module is configured to adjust the initial sound source direction based on the M wake-up parameters and the M sound pickup directions to obtain a target sound source direction.

[0047] In a possible implementation, the adjustment module includes:

[0048] a selection unit, configured to select N sound pickup directions from the M sound pickup directions based on the M wake-up parameters and the M sound pickup directions, wherein the N wake-up parameters of the N sound pickup directions are first N wake-up parameters of the M wake-up parameters arranged from largest to smallest, and N is an integer less than M and greater than 1;

[0049] An adjustment unit is configured to adjust the initial sound source direction based on the N wake-up parameters and the N sound pickup directions to determine the target sound source direction.

[0050] In another possible implementation, the adjustment unit includes:

[0051] A first determining subunit, configured to determine a positional relationship between the N sound pickup directions based on the N sound pickup directions;

[0052] a second determining subunit, configured to determine an adjustment angle based on the N wake-up parameters and the N sound pickup directions if the relationship between the initial sound source direction and the position satisfies a preset condition; and adjust the initial sound source direction based on the adjustment angle to obtain the target sound source direction;

[0053] The third determining subunit is configured to determine, if the initial sound source direction and the position relationship do not satisfy a preset condition, that the target sound source direction is the sound pickup direction corresponding to the maximum wake-up parameter.

[0054] In another possible implementation, the second determining subunit is used for the positional relationship that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between the two is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not the boundary pickup direction of the first pickup range, the first pickup range is the pickup range composed of the N pickup directions, and the boundary pickup direction is the two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle. If the initial sound source direction is within the first pickup range, determine the first weight and the first angle of the pickup direction with the smallest angle with the initial sound source direction; and determine the adjustment angle based on the first weight and the first angle.

[0055] In another possible implementation, the second determining subunit is used for determining the positional relationship such that there is only a pair of adjacent two pickup directions among the N pickup directions, the angle between which is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is one of the boundary pickup directions of the first pickup range, the first pickup range is the pickup range composed of the N pickup directions, and the boundary pickup direction is the two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle; if the initial sound source direction is within the second pickup range, the second pickup range is the pickup range composed of other pickup directions except the pickup direction corresponding to the maximum wake-up parameter, determining the second weight of the other pickup directions; and determining the adjustment angle based on the second weight.

[0056] In another possible implementation, the N is equal to 2, and the second determining subunit is used for the positional relationship that the N pickup directions are adjacent, and the pickup direction corresponding to the maximum wake-up parameter is any one of the N pickup directions. If the initial sound source direction is within the first pickup range, the first pickup range is the pickup range composed of the N pickup directions, and the third weight and second angle of the pickup direction with the smallest angle with the initial sound source direction are determined; based on the third weight and the second angle, the adjustment angle is determined.

[0057] In another possible implementation, the apparatus further includes:

[0058] The second determination module is configured to determine, if the initial sound source direction is within a first sound pickup range, a fourth weight and a third angle of the sound pickup direction having the smallest angle with the initial sound source direction; and determine the adjustment angle based on the fourth weight and the third angle.

[0059] In another possible implementation, the second determining subunit is used for determining that the positional relationship is that there are at least a first pair of adjacent two pickup directions and a second pair of adjacent two pickup directions among the N pickup directions, the angle between the first pair of adjacent two pickup directions is the preset angle, the angle between the second pair of adjacent two pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is any one of the first pair of adjacent pickup directions; if the initial sound source direction is within a third pickup range, the third pickup range is the pickup range composed of the first pair of adjacent two pickup directions, determining a fifth weight and a fourth angle between adjacent pickup directions of the pickup direction corresponding to the maximum wake-up parameter; and determining the adjustment angle based on the fifth weight and the fourth angle.

[0060] In another possible implementation, the apparatus further includes:

[0061] A third determination module is used for the positional relationship that there are at least a first pair of adjacent two pickup directions and a second pair of adjacent two pickup directions among the N pickup directions, the angle between the first pair of adjacent two pickup directions is the preset angle, the angle between the second pair of adjacent two pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not a pickup direction in the first pair of adjacent pickup directions; if the initial sound source direction is within a fourth pickup range, the fourth pickup range is the pickup range composed of the first pair of adjacent two pickup directions, the sum of the wake-up parameters of the adjacent pickup directions is determined; if the sum of the wake-up parameters is not less than the product of the maximum wake-up parameter and the preset coefficient, the target sound source direction is determined to be the initial sound source direction.

[0062] In another possible implementation, the apparatus further includes:

[0063] The fourth determination module is used to determine the adjustment angle based on the N wake-up parameters and the N pickup directions if the position relationship is that there is at least one pair of adjacent two pickup directions in the N pickup directions and the angle between the two directions is the preset angle, and if the initial sound source direction and the position relationship meet the preset conditions.

[0064] In another possible implementation, the apparatus further includes:

[0065] A fifth determining module is configured to determine that the target sound source direction is the initial sound source direction if the positional relationship is such that the angle between no two adjacent sound pickup directions in the N sound pickup directions is the preset angle.

[0066] In another possible implementation, the apparatus further includes:

[0067] The sixth determination module is configured to adjust the initial sound source direction based on the M wake-up parameters and the M sound pickup directions to obtain a target sound source direction if the sound pickup direction having the smallest angle with the initial sound source direction is not the sound pickup direction corresponding to the maximum wake-up parameter.

[0068] In another possible implementation, the apparatus further includes:

[0069] The seventh determination module is configured to determine that the target sound source direction is the initial sound source direction if the sound pickup direction having the smallest angle with the initial sound source direction is the sound pickup direction corresponding to the maximum wake-up parameter.

[0070] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement a method for determining the direction of a sound source as described in any possible implementation manner.

[0071] According to one aspect of an embodiment of the present application, a storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement a method for determining a sound source direction in any possible implementation manner as described above.

[0072] According to one aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program code, wherein the computer program code is stored in a computer-readable storage medium, a processor of an electronic device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the electronic device performs a method for determining a sound source direction in any one of the possible implementations described above.

[0073] In an embodiment of the present application, voice signals from multiple directions are picked up, and the positioning direction of the sound source is adjusted by determining the wake-up parameters corresponding to the voice signals. Since the wake-up parameters can indicate the contribution degree of the corresponding voice signal to waking up the terminal, adjusting the positioning direction of the sound source by using the wake-up parameters can improve the accuracy of obtaining the target sound source direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0075] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0076] Figure 2 is a flow chart of a method for determining a sound source direction provided by an exemplary embodiment of the present application;

[0077] Figure 3 is a flow chart of a method for determining a sound source direction provided by an exemplary embodiment of the present application;

[0078] Figure 4 1 is a schematic diagram of M sound pickup directions in a case provided by an exemplary embodiment of the present application;

[0079] Figure 5 is a flow chart of a method for determining a sound source direction provided by an exemplary embodiment of the present application;

[0080] Figure 6 1 is a schematic structural diagram of a device for determining a sound source direction provided by an exemplary embodiment of the present application;

[0081] Figure 7 This is a schematic diagram of the structure of a terminal provided by an exemplary embodiment of the present application;

[0082] Figure 8 It is a structural diagram of a server provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0083] To make the objectives, technical solutions, and advantages of this application more clear, the following will further describe the embodiments of this application in detail with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this invention.

[0084] It is understood that the terms "first," "second," and the like used herein may be used to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are used solely to distinguish one concept from another. For example, a first pickup range may be referred to as a second pickup range, and similarly, a second pickup range may be referred to as a first pickup range, without departing from the scope of the present application.

[0085] The terms "at least one", "a plurality", "each", and "any" used in this application include one, two, or more than two, a plurality includes two or more than two, each refers to each of the corresponding plurality, and any refers to any one of the plurality. For example, the plurality of wake-up parameters include three wake-up parameters, and each refers to each of the three wake-up parameters. Any refers to any one of the three wake-up parameters, which can be the first, the second, or the third.

[0086] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application, such as Figure 1 As shown, the implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 are connected via a wireless or wired network.

[0087] Optionally, the terminal 101 is any type of terminal such as a smart phone, a tablet computer, a smart wearable device, or a smart home device, and the smart home device is a smart speaker, a smart TV, a smart refrigerator, a smart air conditioner, a smart robot, a smart lamp, a smart lock, etc. The server 102 is a server, or a server cluster consisting of multiple servers, or a cloud computing service center.

[0088] The terminal 101 is installed with an application provided by the server 102, and the terminal 101 can use the application to implement functions such as data transmission and message exchange. Optionally, the application is an application in the operating system of the terminal 101, or an application provided by a third party. For example, the application is a voice assistant, and the voice assistant application has a positioning function. Of course, the voice assistant application can also have other functions, such as recognition function, voice command execution function, etc.

[0089] In some embodiments, when a user uses voice control to perform a target operation on terminal 101, the user speaks a voice signal, and terminal 101 locates the direction of the user's voice signal. Based on this direction, it picks up the user's voice signal and sends the picked-up voice signal to server 102. Server 102 receives the voice signal, recognizes it, and transmits the recognition result to terminal 101, which then performs the target operation corresponding to the recognition result. In other embodiments, server 102 may also locate the direction of the user's voice signal.

[0090] The sound source direction determination method provided in the embodiment of the present application can be applied to any scenario where a voice control terminal performs a target operation.

[0091] The first scenario is when the terminal is a smart home device, and is applied to the scenario of controlling the smart home device through voice signals.

[0092] For example, if the smart home device is a smart TV, and the user wants to change the channel or adjust the volume of the smart TV, the user can control the smart TV through voice signals. The smart TV locates the direction of the voice signal, picks up the voice signal according to the direction, recognizes the voice signal, and then performs the corresponding operation.

[0093] The second scenario, when the terminal is a mobile phone, is applied to the scenario of controlling the mobile phone through voice signals.

[0094] For example, when a user is driving and it is inconvenient for the user to operate the mobile phone, but the user wants to use the mobile phone for navigation, the user can control the mobile phone through voice signals. The mobile phone locates the direction of the voice signal, picks up the voice signal according to the direction, recognizes the voice signal, and then opens the navigation software for navigation.

[0095] It should be noted that the embodiments of the present application are only described by taking a mobile phone or a smart home device as an example to determine the direction of the sound source, and do not limit the scenarios of the sound source direction determination method of the present application.

[0096] Figure 2 This is a flow chart of a method for determining the direction of a sound source provided by an embodiment of the present application. This embodiment of the present application is executed by a terminal, and the method includes the following steps:

[0097] Step 201: The terminal picks up M voice signals based on M sound pickup directions, each voice signal corresponds to a sound pickup direction, and the M voice signals are used to wake up the terminal, where M is an integer greater than 1.

[0098] Step 202: The terminal determines M wake-up parameters and an initial sound source direction based on the M voice signals. The M wake-up parameters are used to represent the contribution of the M voice signals to waking up the terminal.

[0099] Step 203: The terminal adjusts the initial sound source direction based on the M wake-up parameters and the M sound pickup directions to obtain the target sound source direction.

[0100] In an embodiment of the present application, the terminal picks up voice signals from multiple directions and adjusts the positioning direction of the sound source by determining the wake-up parameters corresponding to the voice signals. Since the wake-up parameters can indicate the contribution degree of the corresponding voice signal to waking up the terminal, adjusting the positioning direction of the sound source by using the wake-up parameters can improve the accuracy of determining the direction of the target sound source.

[0101] Figure 3 This is a flow chart of a method for determining the direction to a sound source provided by an embodiment of the present application. This embodiment of the present application is executed by a terminal, and the method includes the following steps:

[0102] Step 301: The terminal picks up M voice signals based on M sound pickup directions.

[0103] In which, each voice signal corresponds to a pickup direction, M voice signals are used to wake up the terminal, and M is an integer greater than 1; for example, M is 2, 3, 4, 5 or 6. There is an angle between any two adjacent pickup directions in the M pickup directions, and the angles can be the same or different. For example, M is 4, and the four pickup directions include a 90-degree pickup direction, a 180-degree pickup direction, a 270-degree pickup direction, and a 360-degree pickup direction. In this case, the angle between any two adjacent pickup directions is the same, all 90 degrees. For another example, M is 6, and the six pickup directions include a 45-degree pickup direction, a 90-degree pickup direction, a 135-degree pickup direction, a 180-degree pickup direction, a 270-degree pickup direction, and a 360-degree pickup direction. In this case, the angle between any two adjacent pickup directions is different, some are 45 degrees, and some are 90 degrees. In the embodiment of the present application, the angle between any two adjacent pickup directions is not specifically limited here.

[0104] In a possible implementation, the M pickup directions may be pre-set by the user at the terminal, that is, the terminal pre-sets the M pickup directions. Accordingly, the step of the terminal determining the M pickup directions is as follows: the terminal directly determines the M pickup directions based on the pre-set pickup directions.

[0105] For example, if the user pre-sets the terminal to pick up voice signals in the direction of 90-degree pickup, 180-degree pickup, 270-degree pickup, and 360-degree pickup, the terminal directly uses these four pickup directions as the M pickup directions. For example, if the user pre-sets the terminal to pick up voice signals in the direction of 45-degree pickup, 90-degree pickup, 135-degree pickup, 180-degree pickup, 270-degree pickup, and 360-degree (0-degree) pickup, the terminal directly uses these six pickup directions as the M pickup directions, which are not specifically limited here.

[0106] In the embodiment of the present application, the terminal directly determines M pickup directions based on the pre-set pickup directions, which is simple to operate and has low running consumption.

[0107] In another possible implementation, the terminal can determine M pickup directions based on the historical number of pickups for each pickup direction. Accordingly, the terminal determines the M pickup directions by obtaining the historical number of pickups and determining the M pickup directions based on the historical number of pickups. The historical number of pickups is positively correlated with the number of pickup directions. Specifically, a greater number of historical pickups corresponds to a greater number of pickup directions, i.e., a greater M corresponds to a smaller number of historical pickups, and a smaller number of pickup directions corresponds to a smaller M corresponds to a smaller number of historical pickups.

[0108] For example, the number of pickup times within the pickup range of 0 degrees to 180 degrees is relatively large, and the number of pickup times within the pickup range of 180 degrees to 360 degrees is relatively small. The terminal determines multiple pickup directions within the pickup range of 0 degrees to 180 degrees, and determines fewer pickup directions within the pickup range of 180 degrees to 360 degrees. The M pickup directions that the terminal can determine are 45-degree pickup direction, 90-degree pickup direction, 135-degree pickup direction, 180-degree pickup direction, 270-degree pickup direction, and 360-degree pickup direction, etc., which are not specifically limited here.

[0109] In an embodiment of the present application, the terminal sets M pickup directions based on the historical number of pickup times, which is in line with the working rules of the terminal, that is, it can meet the user's usage habits. Within the pickup range where the terminal picks up the user's voice signal more times, more directions are set for pickup to avoid signal omission. Within the pickup range where the terminal picks up the user's voice signal less times, fewer directions are set for pickup to reduce operating consumption.

[0110] In another possible implementation, the terminal can determine M pickup directions based on its own type. Accordingly, the step for the terminal to determine M pickup directions is: the terminal obtains the type of the terminal, and based on the type, determines M pickup directions. The type of the terminal can be a mobile phone, a smart TV, or a smart refrigerator, etc., which is not specifically limited here. For example, the type of the terminal is a mobile phone, and the user can be located in any direction of the mobile phone. The terminal needs to pick up voice signals in the surrounding 360-degree direction. Therefore, the M pickup directions determined by the terminal can be a 90-degree pickup direction, a 180-degree pickup direction, a 270-degree pickup direction, and a 360-degree pickup direction. These four directions can cover the 360-degree direction of the terminal. For example, the terminal is a smart TV. At this time, the user is usually in front of, to the left of, or to the right of the terminal. The terminal needs to pick up voice signals in the range of these three directions. Therefore, the M pickup directions determined by the terminal can be a 0-degree pickup direction, a 45-degree pickup direction, a 90-degree pickup direction, a 135-degree pickup direction, and a 180-degree pickup direction. These five pickup directions can cover the front, left, and right of the terminal, and there is no need to cover 360 degrees of the terminal.

[0111] In an embodiment of the present application, the terminal determines the pickup direction based on the type of the terminal, which can be more in line with the terminal itself. In a range where there is no voice signal, the pickup direction is not set, which can reduce operating consumption.

[0112] In the examples of the following steps, the angle between any two sound pickup directions in the M sound pickup directions is taken as the preset angle for explanation. For example, see Figure 4, M=8, the 8 pickup directions are pickup direction x0, pickup direction x1, pickup direction x2, pickup direction x3, pickup direction x4, pickup direction x5, pickup direction x6 and pickup direction x7, and the angle between any two pickup directions among these 8 pickup directions is 45 degrees.

[0113] Step 302: The terminal determines M wake-up parameters and an initial sound source direction based on the M voice signals.

[0114] The M wake-up parameters represent the contribution of the M voice signals to waking up the terminal. The terminal inputs the M voice signals into the wake-up model, which then outputs the corresponding wake-up parameter based on the contribution of each voice signal to waking up the terminal. Each voice signal corresponds to one wake-up parameter. If multiple voice signals can wake up the terminal, the size of the wake-up parameter is positively correlated with the quality of the corresponding voice signal. That is, the better the voice signal quality, the larger the corresponding wake-up parameter, and the worse the voice signal quality, the smaller the corresponding wake-up parameter.

[0115] Based on M voice signals, the terminal obtains the initial sound source direction through the DOA (Direction Of Arrival, direction of arrival positioning technology) algorithm. However, the initial sound source direction obtained by the traditional DOA algorithm is not necessarily accurate. The terminal can verify the direction through the pickup direction corresponding to the largest wake-up parameter among the M wake-up parameters. Accordingly, the step of verifying the initial sound source direction of the terminal is: the terminal obtains the pickup direction corresponding to the largest wake-up parameter, and verifies the initial sound source direction based on the positional relationship between the pickup direction corresponding to the largest wake-up parameter and the initial sound source direction. It can be divided into the following two cases, including:

[0116] In the first case, if the pickup direction with the smallest angle with the initial sound source direction is not the pickup direction corresponding to the largest wake-up parameter, the step of adjusting the initial sound source direction based on M wake-up parameters and M pickup directions to obtain the target sound source direction is performed. Since the largest wake-up parameter indicates that the voice signal corresponding to the wake-up parameter has the greatest contribution to waking up the terminal, but the obtained initial sound source direction differs significantly from the pickup direction corresponding to the largest wake-up parameter, it indicates that the initial sound source direction obtained by the traditional algorithm is inaccurate and needs to be adjusted to obtain a more accurate direction of the sound source.

[0117] In the second case, if the pickup direction with the smallest angle with the initial sound source direction is the pickup direction corresponding to the largest wake-up parameter, the target sound source direction is determined to be the initial sound source direction, and then the step of identifying the target voice signal based on the target sound source direction is performed. Since the largest wake-up parameter indicates that the voice signal corresponding to the wake-up parameter has the greatest contribution to waking up the terminal, the difference between the obtained initial sound source direction and the pickup direction corresponding to the largest wake-up parameter is not much, indicating that the initial sound source direction obtained by the traditional algorithm is relatively accurate and can be used as the direction of the sound source. In the subsequent process, the voice signal is collected and recognized based on this initial sound source direction.

[0118] In an embodiment of the present application, the terminal determines whether the initial sound source direction needs to be adjusted by determining the positional relationship between the initial sound source direction and the pickup direction corresponding to the maximum wake-up parameter. In the first case mentioned above, the initial sound source direction is relatively inaccurate and needs to be adjusted to facilitate the subsequent acquisition of the accurate target sound source direction. In the second case mentioned above, the initial sound source direction is relatively accurate and does not need to be adjusted, thereby reducing operating consumption.

[0119] It should be noted that, when the terminal is not awakened, the terminal executes step 301 and continues to pick up the voice signal.

[0120] Step 303: The terminal selects N sound pickup directions from the M sound pickup directions based on the M wake-up parameters and the M sound pickup directions.

[0121] Among them, the N wake-up parameters of the N pickup directions are the first N wake-up parameters of the M wake-up parameters arranged from large to small, and N is an integer less than M and greater than 1. For example, M=4, N=3, that is, the terminal selects the pickup directions corresponding to the first three wake-up parameters of the four wake-up parameters arranged from large to small from the four pickup directions. If the wake-up parameter of the 90-degree pickup direction is 4, the wake-up parameter of the 180-degree pickup direction is 6, the wake-up parameter of the 270-degree pickup direction is 5, and the wake-up parameter of the 360-degree pickup direction is 2, then the terminal selects the three pickup directions of 90-degree pickup direction, 180-degree pickup direction, and 270-degree pickup direction based on these four wake-up parameters.

[0122] For example, M=6, N=3, that is, the terminal selects the pickup directions corresponding to the first three wake-up parameters of the six wake-up parameters arranged from large to small from the six pickup directions. If the wake-up parameter of the pickup direction is 45 degrees, the wake-up parameter of the pickup direction is 4, the wake-up parameter of the pickup direction is 90 degrees, the wake-up parameter of the pickup direction is 135 degrees, the wake-up parameter of the pickup direction is 5, the wake-up parameter of the pickup direction is 180 degrees, the wake-up parameter of the pickup direction is 6, the wake-up parameter of the pickup direction is 270 degrees, and the wake-up parameter of the pickup direction is 360 degrees. The terminal selects the three pickup directions of 135 degrees, 180 degrees, and 270 degrees based on these six wake-up parameters.

[0123] For example, M=8, N=3 or M=8, N=4 or M=7, N=4, etc. In the embodiments of the present application, there is no specific limitation on the numerical values ​​of M and N.

[0124] Step 304: The terminal determines the positional relationship of the N sound pickup directions based on the N sound pickup directions.

[0125] Among them, the positional relationship is the positional relationship between N pickup directions. For example, continuing the above example, the terminal selects three pickup directions of 90 degrees, 180 degrees and 270 degrees as N pickup directions. It can be seen that these three pickup directions are adjacent. Among them, the wake-up parameter corresponding to the 180-degree pickup direction is 6. Compared with the wake-up parameters corresponding to the other two directions, the wake-up parameter corresponding to the 180-degree pickup direction is the largest, that is, the pickup direction corresponding to the largest wake-up parameter is located at the center of the N pickup directions.

[0126] Based on the adjacent situations between the N pickup directions and the position of the pickup direction corresponding to the maximum wake-up parameter among these N pickup directions, the position relationship can be divided into the following situations, including:

[0127] In the first case, if N is greater than 2 and less than M, the positional relationship is that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not the boundary pickup direction of the first pickup range. The first pickup range is the pickup range composed of N pickup directions, and the boundary pickup direction is the two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle.

[0128] For example, see Figure 4Assuming N=3, the N pickup directions are pickup direction x1, pickup direction x2, and pickup direction x3. The wake-up parameters corresponding to these three pickup directions are s1, s2, and s3, respectively, and s2 is the largest. It can be seen that the angle between the pickup direction x1 and the pickup direction x2 is the preset angle, the angle between the pickup direction x2 and the pickup direction x3 is the preset angle, and the angle between the pickup direction x1 and the pickup direction x3 is greater than the preset angle. That is, among these three pickup directions, only one pair of adjacent two pickup directions has an angle greater than the preset angle. At this time, the pickup direction x1 and the pickup direction x3 are boundary pickup directions. The pickup direction corresponding to the largest wake-up parameter is the pickup direction x2, not the boundary pickup direction. It can be seen that these three pickup directions meet the positional relationship in this case.

[0129] In the second case, if N is greater than 2 and less than M, the positional relationship is that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is one of the boundary pickup directions of the first pickup range. The first pickup range is a pickup range composed of N pickup directions, and the boundary pickup direction is two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle.

[0130] For example, see Figure 4 , assuming N = 3, the N pickup directions are pickup direction x1, pickup direction x2, and pickup direction x3, the wake-up parameters corresponding to these three pickup directions are s1, s2, and s3, respectively, and s1 is the largest. It can be seen that the angle between the pickup direction x1 and the pickup direction x2 is the preset angle, the angle between the pickup direction x2 and the pickup direction x3 is the preset angle, and the angle between the pickup direction x1 and the pickup direction x3 is greater than the preset angle, that is, among these three pickup directions, there is only a pair of adjacent two pickup directions whose angle is greater than the preset angle. At this time, the pickup direction x1 and the pickup direction x3 are boundary pickup directions, and the pickup direction corresponding to the largest wake-up parameter is the pickup direction x1, which is the boundary pickup direction. It can be seen that these three pickup directions meet the positional relationship in this case.

[0131] In the third case, if N is equal to 2, the position relationship is that the N pickup directions are adjacent, and the pickup direction corresponding to the maximum wake-up parameter is any one of the N pickup directions. Figure 4 , the N pickup directions are pickup direction x1 and pickup direction x2, the wake-up parameters corresponding to these two pickup directions are s1 and s2 respectively, and s1>s2. It can be seen that these two pickup directions meet the position relationship in this situation.

[0132] In the fourth case, if N=M, it indicates that all pickup directions are selected. At this time, there is no need to determine the positional relationship of the N pickup directions. It is only necessary to adjust the initial sound source direction by the pickup direction closest to the initial sound source direction.

[0133] In the fifth case, the positional relationship is that there are at least a first pair of two adjacent pickup directions and a second pair of two adjacent pickup directions among the N pickup directions, the angle between the first pair of two adjacent pickup directions is a preset angle, the angle between the second pair of two adjacent pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is any one of the first pair of adjacent pickup directions.

[0134] For example, see Figure 4 , the N sound pickup directions may be the sound pickup direction x1, the sound pickup direction x2, and the sound pickup direction x5. It can be seen that, among the N sound pickup directions, there is a pair of adjacent two sound pickup directions whose included angle is the preset angle, that is, the included angle between the sound pickup direction x1 and the sound pickup direction x2 is the preset angle, and the sound pickup direction x1 and the sound pickup direction x2 constitute a first pair of adjacent two sound pickup directions. There is an angle between two adjacent sound pickup directions greater than the preset angle, that is, the included angle between the adjacent sound pickup direction x1 and the sound pickup direction x5 is greater than the preset angle, and the sound pickup direction x1 and the sound pickup direction x5 constitute a second pair of adjacent two sound pickup directions. The included angle between the adjacent sound pickup direction x2 and the sound pickup direction x5 is greater than the preset angle, and the sound pickup direction x2 and the sound pickup direction x5 constitute a second pair of adjacent two sound pickup directions. These three sound pickup directions meet the positional relationship in this case. At this time, the sound pickup direction corresponding to the maximum wake-up parameter is the sound pickup direction x1 or the sound pickup direction x2, which is any one of the first pair of adjacent sound pickup directions.

[0135] In the sixth case, the position relationship is that there are at least a first pair of two adjacent pickup directions and a second pair of two adjacent pickup directions in the N pickup directions, the angle between the first pair of two adjacent pickup directions is a preset angle, the angle between the second pair of two adjacent pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not a pickup direction in the first pair of adjacent pickup directions. For example, continue to refer to Figure 4 The N sound pickup directions can be pickup direction x1, pickup direction x2, and pickup direction x5. These three pickup directions satisfy the positional relationship in this case, which is the same as the fifth case above and will not be repeated here. In this case, the pickup direction corresponding to the maximum wake-up parameter is pickup direction x5, which is not a pickup direction in the first pair of adjacent pickup directions.

[0136] In the seventh case, the position relationship is that there are no two adjacent sound pickup directions in the N sound pickup directions, and the angle between them is the preset angle. Figure 4 , the N sound pickup directions may be the sound pickup direction x1, the sound pickup direction x5 and the sound pickup direction x7. It can be seen that the angle between any two adjacent sound pickup directions among the three sound pickup directions is not the preset angle, which satisfies the positional relationship in this case.

[0137] The positional relationship may also include other situations, which are not specifically limited here.

[0138] It should be noted that the method for determining the target sound source direction is different depending on the positional relationship between the N pickup directions. If the positional relationship is such that the angle between no two adjacent pickup directions among the N pickup directions is the preset angle, the terminal determines the target sound source direction as the initial sound source direction and then performs the step of identifying the target voice signal based on the target sound source direction. In this case, it indicates that the N pickup directions corresponding to the maximum wake-up parameter are not adjacent. The terminal cannot determine the approximate range of the sound source based on the wake-up parameter and thus cannot adjust the initial sound source direction based on the wake-up parameter. In this case, the terminal directly determines the target sound source direction as the initial sound source direction and then performs the step of identifying the target voice signal based on the target sound source direction. If the positional relationship is such that the angle between at least one pair of adjacent pickup directions among the N pickup directions is the preset angle, the terminal performs the step of determining the adjustment angle based on the N wake-up parameters and the N pickup directions if the initial sound source direction and the positional relationship meet the preset conditions, that is, the terminal executes step 305.

[0139] Step 305: If the relationship between the initial sound source direction and position meets the preset conditions, the terminal determines an adjustment angle based on the N wake-up parameters and the N sound pickup directions.

[0140] Among them, the preset conditions include the preset conditions of the initial sound source direction and the preset conditions of the position relationship. The preset conditions are the prerequisites for the terminal to determine the adjustment angle and then adjust the initial sound source direction. When the preset conditions are different, the terminal determines the adjustment angle in different ways, that is, the obtained adjustment angle is different, which can be divided into the following situations, including:

[0141] For ease of understanding, for each case, a is taken as the initial sound source direction, and for cases where the specific value of N is not indicated, N=3. The three pickup directions are a1, a2, and a3, and their corresponding wake-up parameters are s1, s2, and s3, respectively. s1 is the largest, and is taken as an example for explanation.

[0142] In the first case, the positional relationship is that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not the boundary pickup direction of the first pickup range. The first pickup range is the pickup range composed of N pickup directions, and the boundary pickup direction is the two adjacent pickup directions in the first pickup range with an angle greater than the preset angle. If the initial sound source direction is within the pickup direction range, the terminal determines the first weight and the first angle of the pickup direction with the smallest angle with the initial sound source direction, and determines the adjustment angle based on the first weight and the first angle.

[0143] For example, the positional relationship is that among the sound pickup directions a1, a2, and a3, there is only a pair of adjacent sound pickup directions with an angle greater than the preset angle, and the sound pickup direction a1 is located between the sound pickup direction a2 and the sound pickup direction a3, that is, the sound pickup direction a1 is not the boundary sound pickup direction of the first sound pickup range. At this time, the first sound pickup range is min(a2, a3)-θ / 2 to max(a2, a3)+θ / 2, then the initial sound source direction is located between min(a2, a3)-θ / 2 and max(a2, a3)+θ / 2, where θ is the angle between the sound pickup directions. When the relationship between the initial sound source direction and the position meets the above conditions, the first weight and the first angle of the sound pickup direction with the smallest angle with the initial sound source direction are determined; if the sound pickup direction with the smallest angle with the initial sound source direction is a2, the first weight is s2 / s1, and the first angle is a2-a; based on the first weight and the first angle, the adjustment angle is determined to be (a2-a)*s2 / s1; if the sound pickup direction with the smallest angle with the initial sound source direction is a3, the first weight is s3 / s1, and the first angle is a3-a; based on the first weight and the first angle, the adjustment angle is determined to be (a3-a)*s3 / s1.

[0144] In the second case, the positional relationship is that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is one of the boundary pickup directions of the first pickup range. The first pickup range is the pickup range composed of N pickup directions, and the boundary pickup direction is the two adjacent pickup directions in the first pickup range with an angle greater than the preset angle. If the initial sound source direction is within the second pickup range, the second pickup range is the pickup range composed of other pickup directions except the pickup direction corresponding to the maximum wake-up parameter. The terminal determines the second weight of the other pickup directions, and determines the adjustment angle based on the second weight.

[0145] For example, if the positional relationship is such that only one pair of adjacent pickup directions among pickup directions a1, a2, and a3 has an angle greater than a preset angle, and pickup direction a1 is one of the boundary pickup directions of the first pickup range, when a1 is the largest, the second pickup range is a1-θ / 2 and max(a2, a3)+θ / 2, then the initial sound source direction is between min(a2, a3)-θ / 2 and a1+θ / 2; when a1 is the smallest, the second pickup range is a1-θ / 2 and max(a2, a3)+θ / 2, then the initial sound source direction is between a1-θ / 2 and max(a2, a3)+θ / 2. If the initial sound source direction and positional relationship meet the above conditions, the terminal determines the second weight of the other pickup directions to be (s2+s3) / (2*s1), and based on this second weight, determines the adjustment angle to be (s2+s3) / (2*s1).

[0146] In the third case, if N=2, the positional relationship is that N pickup directions are adjacent, and the pickup direction corresponding to the maximum wake-up parameter is any one of the N pickup directions. If the initial sound source direction is within the first pickup range, the first pickup range is the pickup range composed of the N pickup directions. The terminal determines the third weight and second angle of the pickup direction with the smallest angle with the initial sound source direction; based on the third weight and the second angle, determines the adjustment angle.

[0147] For example, the positional relationship is that the two pickup directions a1 and a2 are adjacent, the pickup direction corresponding to the maximum wake-up parameter is the pickup direction a1, and the first pickup range is the pickup range composed of the pickup directions a1 and a2. The terminal determines the third weight and the second angle of the pickup direction with the smallest angle with the initial sound source direction. If the pickup direction with the smallest angle with the initial sound source direction is a2, the third weight is s2 / s1, and the second angle is a2-a. Based on the third weight and the second angle, the adjustment angle is determined to be (a2-a)*s2 / s1.

[0148] In the fourth case, if N = M, all sound pickup directions are selected. In this case, there's no need to determine the positional relationship among the N sound pickup directions. The initial sound source direction can be adjusted based on the pickup direction closest to the initial sound source direction. If the initial sound source direction is within the first sound pickup range, which consists of N pickup directions, the fourth weight and third angle are determined for the pickup direction with the smallest angle with the initial sound source direction. The adjustment angle is determined based on the fourth weight and the third angle.

[0149] For example, the first pickup range is the pickup range composed of these N pickup directions, the pickup direction corresponding to the maximum wake-up parameter is the pickup direction a1, the initial sound source direction is within this first pickup range, and the terminal determines the fourth weight and third angle of the pickup direction with the smallest angle with the initial sound source direction. If the pickup direction with the smallest angle with the initial sound source direction is a3, the fourth weight is s3 / s1, and the third angle is a3-a. Based on the fourth weight and the third angle, the adjustment angle is determined to be (a3-a)*s3 / s1.

[0150] In the fifth case, the positional relationship is that there are at least a first pair of two adjacent pickup directions and a second pair of two adjacent pickup directions among the N pickup directions, the angle between the first pair of adjacent pickup directions is a preset angle, the angle between the second pair of adjacent pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is any pickup direction in the first pair of adjacent pickup directions. If the initial sound source direction is within the third pickup range, the third pickup range is the pickup range composed of the first pair of adjacent pickup directions. The terminal determines the second angle and the third weight between the adjacent pickup directions of the pickup direction corresponding to the maximum wake-up parameter; based on the third weight and the second angle, the adjustment angle is determined.

[0151] For example, the angle between a1 and a2 is a preset angle, and the angle between a1 and a3 and the angle between a2 and a3 are both greater than the preset angle. The initial sound source direction is located in the pickup range formed by a1 and a2. At this time, the second angle between the adjacent pickup directions corresponding to the maximum wake-up parameter is determined to be a2-a, and the third weight is s2 / s1. Based on the third weight and the second angle, the adjustment angle is determined to be (a2-a)*s2 / s1; or, a1 is adjacent to a3, and neither is adjacent to a2. The initial sound source direction is located in the pickup range formed by a1 and a3. At this time, the second angle between the adjacent pickup directions corresponding to the maximum wake-up parameter is determined to be a3-a, and the third weight is s2 / s1. Based on the third weight and the second angle, the adjustment angle is determined to be (a3-a)*s2 / s1.

[0152] In the sixth case, the positional relationship is that there are at least a first pair of two adjacent pickup directions and a second pair of two adjacent pickup directions in the N pickup directions, the angle between the first pair of two adjacent pickup directions is a preset angle, the angle between the second pair of two adjacent pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not a pickup direction in the first pair of adjacent pickup directions. If the initial sound source direction is within the fourth pickup range, the fourth pickup range is the pickup range composed of the first pair of adjacent pickup directions. The terminal determines the sum of the wake-up parameters of the adjacent pickup directions. If the sum of the wake-up parameters is not less than the product of the maximum wake-up parameter and the preset coefficient, the target sound source direction is determined to be the initial sound source direction, and then step 307 is executed.

[0153] For example, the angle between a2 and a3 is a preset angle, and the angle between a1 and a2 and the angle between a1 and a3 are both greater than the preset angle, then the fourth pickup range is min(a2, a3)-θ / 2 to max(a2, a3)+θ / 2, the initial sound source direction is between min(a2, a3)-θ / 2 and max(a2, a3)+θ / 2, and the sum of the wake-up parameters of adjacent pickup directions is determined to be s2+s3. If the sum of the wake-up parameters is not less than the product of the maximum wake-up parameter and the preset coefficient, the target sound source direction is determined to be the initial sound source direction. For example, the preset coefficient is 1.5, that is, if the sum of the wake-up parameters s2+s3>=1.5*s1, the terminal determines that the target sound source direction is a.

[0154] In an embodiment of the present application, the terminal uses different methods to determine the adjustment angle based on the different preset conditions satisfied by the initial sound source direction and position relationship, which is conducive to improving the accuracy of the adjustment angle. Therefore, when adjusting the initial sound source direction by adjusting the angle, the accuracy of obtaining the target sound source direction can be improved, and then based on the pickup direction, the quality of the target voice signal obtained is better, which can improve the accuracy of recognizing the target voice signal.

[0155] One thing that needs to be explained is that if the initial sound source direction and position relationship do not meet the preset conditions, the terminal determines the target sound source direction as the pickup direction corresponding to the maximum wake-up parameter. This situation indicates that the N pickup directions corresponding to the maximum wake-up parameters are adjacent, and the terminal can judge the approximate range of the sound source based on the wake-up parameters. If the initial sound source direction does not meet the preset conditions, that is, the initial sound source direction differs greatly from the N pickup directions, it means that the initial sound source direction is inaccurate. At this time, the terminal directly determines the target sound source direction as the pickup direction corresponding to the maximum wake-up parameter.

[0156] Another point that needs to be explained is that the pickup range in the above example does not include the pickup range between a1-θ / 2 and a1+θ / 2. If the initial sound source direction is between a1-θ / 2 and a1+θ / 2, it means that the pickup direction with the smallest angle with the initial sound source direction is the pickup direction corresponding to the largest wake-up parameter. Since the largest wake-up parameter represents the greatest contribution of the voice signal corresponding to the wake-up parameter to awakening the terminal, the obtained initial sound source direction is not much different from the pickup direction corresponding to the largest wake-up parameter, indicating that the initial sound source direction is more accurate. At this time, the terminal directly determines the target sound source direction as the initial sound source direction.

[0157] Step 306: The terminal adjusts the initial sound source direction based on the adjustment angle to obtain the target sound source direction.

[0158] The terminal adds an adjustment angle to the initial sound source direction to adjust the initial sound source direction, thereby obtaining the target sound source direction. For example, following the example in the above steps, in the first case, when the adjustment angle is (a2-a)*s2 / s1, the target sound source direction is a_final=a+(a2-a)*s2 / s1, and when the adjustment angle is (a3-a)*s3 / s1, the target sound source direction is a_final=a+(a3-a)*s3 / s1; in the second case, when the adjustment angle is (s2+s3) / (2*s1), the target sound source direction is a_final=a+(s2+s3) / (2*s1); in the third case, when the adjustment angle is (a2-a)*s2 / s1, the target sound source direction is a_final=a+(a2-a)*s2 / s1; in the fourth case, when the adjustment angle is (a3-a)*s3 / s1, the target sound source direction is a_final=a+(a3-a)*s3 / s1; in the fifth case, when the adjustment angle is (a2-a)*s2 / s1, the target sound source direction is a_final=a+(a2-a)*s2 / s1, and when the adjustment angle is (a3-a)*s3 / s1, the target sound source direction is a_final=a+(a3-a)*s3 / s1; in the sixth case, when the initial sound source direction and position relationship meet the preset conditions, the terminal directly determines the target sound source direction as the initial sound source direction, that is, a_final=a, and the adjustment angle at this time is 0.

[0159] It should be noted that after determining the target sound source direction, the terminal can also identify the voice signal in the target sound source direction. That is, the terminal identifies the target voice signal based on the target sound source direction, where the target voice signal is the voice signal collected in the target sound source direction. The terminal needs to first collect the voice signal in the target sound source direction based on the target sound source direction, and then identify the collected voice signal. Accordingly, the process of the terminal identifying the target voice signal based on the target sound source direction can be achieved through the following two steps, including:

[0160] (1) The terminal collects the target voice signal based on the direction of the target sound source.

[0161] Because there are many voice signals in the surrounding environment (for example, including the user's voice signal and environmental noise), the terminal only needs to identify the control instructions in the user's voice signal and then execute the operation corresponding to the control instruction. Therefore, before recognizing the voice signal, the terminal only needs to collect the user's voice signal. To avoid interference from other environmental noise, the terminal needs to suppress noise in other directions when collecting the target voice signal based on the direction of the target sound source and enhance the target voice signal in the direction of the target sound source. For example, the terminal can use beamforming to enhance the voice signal in that direction, thereby improving the effect of voice recognition.

[0162] (2) The terminal recognizes the target speech signal.

[0163] The terminal performs voice recognition on the target voice signal and identifies the voice control instruction from the target voice signal.

[0164] In one possible implementation, the terminal identifies the voice control instructions in the target voice signal through the server; the process of the terminal identifying the voice signal can be: the terminal sends the target voice signal to the server, the server receives the target voice signal, identifies the voice control instructions from the target voice signal, sends the voice control instructions to the terminal, and the terminal receives the voice control instructions.

[0165] Wherein, the server stores a plurality of voice control instructions. Accordingly, the step of the server identifying the voice control instruction from the target voice signal may be: the server may determine the voice control instruction of the target voice signal from the plurality of voice control instructions.

[0166] In an embodiment of the present application, the terminal sends the target voice signal to the server, and the server recognizes the target voice signal. Since the server can store more voice control instructions than the terminal, the server recognizes the target voice signal, which can improve the recognition accuracy.

[0167] In another possible implementation, the terminal stores multiple voice control instructions, and the terminal can directly determine the voice control instruction for the target voice signal from the multiple voice control instructions. Accordingly, the process of the terminal recognizing the voice signal can be: the terminal performs voice recognition on the target voice signal, and determines the voice control instruction that matches the target voice signal from the multiple voice control instructions stored locally. In the embodiment of the present application, the terminal recognizes the target voice signal locally, which is simple to operate and does not need to be sent to the server, which can improve the efficiency of voice recognition.

[0168] In an embodiment of the present application, the terminal picks up voice signals from multiple directions and adjusts the positioning direction of the sound source by determining the wake-up parameters corresponding to the voice signals. Since the wake-up parameters can indicate the contribution degree of the corresponding voice signal to waking up the terminal, adjusting the positioning direction of the sound source by using the wake-up parameters can improve the accuracy of determining the direction of the target sound source, thereby improving the quality of the target voice signal obtained based on the target sound source direction, and further improving the accuracy of identifying the target voice signal.

[0169] Figure 5 This is a flow chart of a method for determining the direction of a sound source provided by an embodiment of the present application. This embodiment of the present application is executed by a server, and the method includes the following steps:

[0170] Step 501: The terminal picks up M voice signals based on M sound pickup directions.

[0171] Step 501 is the same as step 301 and will not be described again here.

[0172] Step 502: The terminal sends M voice signals to the server.

[0173] Step 503: The server receives M voice signals.

[0174] Step 504: The server determines M wake-up parameters and initial sound source directions based on the M voice signals.

[0175] Step 505: The server selects N sound pickup directions from the M sound pickup directions based on the M wake-up parameters.

[0176] Step 506: The server determines the positional relationship of the N sound pickup directions based on the N sound pickup directions.

[0177] Step 507: If the relationship between the initial sound source direction and position meets the preset conditions, the server determines the adjustment angle based on the N wake-up parameters and the N sound pickup directions.

[0178] Step 508: The server adjusts the initial sound source direction based on the adjustment angle to obtain the target sound source direction.

[0179] Steps 504-508 are the same as steps 302-306 and will not be repeated here.

[0180] Step 509: The server sends the target sound source direction to the terminal.

[0181] Step 510: The terminal receives the direction of the target sound source.

[0182] In an embodiment of the present application, the server picks up voice signals from multiple directions and adjusts the positioning direction of the sound source by determining the wake-up parameters corresponding to the voice signals. Since the wake-up parameters can indicate the contribution degree of the corresponding voice signal to waking up the terminal, adjusting the positioning direction of the sound source by using the wake-up parameters can improve the accuracy of determining the direction of the target sound source.

[0183] Figure 6 This is a schematic diagram of the structure of a device for determining the direction of a sound source provided by an embodiment of the present application, see Figure 6 , the device comprises:

[0184] A pickup module 601 is configured to pick up M voice signals based on M pickup directions, where each voice signal corresponds to one pickup direction, and the M voice signals are used to wake up the terminal, where M is an integer greater than 1.

[0185] A first determining module 602 is configured to determine M wake-up parameters and an initial sound source direction based on the M voice signals, where the M wake-up parameters represent the contribution of the M voice signals to waking up the terminal;

[0186] The adjustment module 603 is configured to adjust the initial sound source direction based on the M wake-up parameters and the M sound pickup directions to obtain a target sound source direction.

[0187] In a possible implementation, the adjustment module includes:

[0188] A selection unit is configured to select N sound pickup directions from the M sound pickup directions based on the M wake-up parameters and the M sound pickup directions, wherein the N wake-up parameters of the N sound pickup directions are the first N wake-up parameters of the M wake-up parameters arranged from largest to smallest, and N is an integer less than M and greater than 1;

[0189] The adjustment unit is used to adjust the initial sound source direction based on N wake-up parameters and N sound pickup directions to determine the target sound source direction.

[0190] In another possible implementation, the adjustment unit includes:

[0191] A first determining subunit is configured to determine a positional relationship among the N sound pickup directions based on the N sound pickup directions;

[0192] The second determining subunit is configured to determine an adjustment angle based on the N wake-up parameters and the N sound pickup directions if the relationship between the initial sound source direction and the position meets a preset condition; and adjust the initial sound source direction based on the adjustment angle to obtain a target sound source direction;

[0193] The third determining subunit is configured to determine the target sound source direction as the sound pickup direction corresponding to the maximum wake-up parameter if the initial sound source direction and position relationship do not meet the preset conditions.

[0194] In another possible implementation, the second determination subunit is used to determine that the positional relationship is such that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not the boundary pickup direction of the first pickup range. The first pickup range is a pickup range composed of N pickup directions, and the boundary pickup direction is two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle. If the initial sound source direction is within the first pickup range, determine the first weight and the first angle of the pickup direction with the smallest angle with the initial sound source direction; and determine the adjustment angle based on the first weight and the first angle.

[0195] In another possible implementation, the second determination subunit is used to determine the positional relationship of only one pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is one of the boundary pickup directions of the first pickup range. The first pickup range is the pickup range composed of N pickup directions, and the boundary pickup direction is the two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle. If the initial sound source direction is within the second pickup range, the second pickup range is the pickup range composed of other pickup directions except the pickup direction corresponding to the maximum wake-up parameter, and the second weight of the other pickup directions is determined; based on the second weight, the adjustment angle is determined.

[0196] In another possible implementation, the second determination subunit is used to determine the positional relationship of N pickup directions that are adjacent, and the pickup direction corresponding to the maximum wake-up parameter is any one of the N pickup directions. If the initial sound source direction is within the first pickup range, the first pickup range is a pickup range composed of N pickup directions, and the third weight and second angle of the pickup direction with the smallest angle with the initial sound source direction are determined; based on the third weight and the second angle, the adjustment angle is determined.

[0197] In another possible implementation, the apparatus further includes:

[0198] The second determination module is configured to determine, if the initial sound source direction is within the first sound pickup range, a fourth weight and a third angle of the sound pickup direction having the smallest angle with the initial sound source direction; and determine an adjustment angle based on the fourth weight and the third angle.

[0199] In another possible implementation, the second determination subunit is used to determine that there are at least a first pair of adjacent two pickup directions and a second pair of adjacent two pickup directions in the N pickup directions, the angle between the first pair of adjacent two pickup directions is a preset angle, the angle between the second pair of adjacent two pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is any pickup direction of the first pair of adjacent pickup directions. If the initial sound source direction is within the third pickup range, the third pickup range is the pickup range composed of the first pair of adjacent two pickup directions, and the fifth weight and the fourth angle between the adjacent pickup directions of the pickup direction corresponding to the maximum wake-up parameter are determined; based on the fifth weight and the fourth angle, the adjustment angle is determined.

[0200] In another possible implementation, the apparatus further includes:

[0201] The third determination module is used to determine that, among the N pickup directions, there are at least a first pair of adjacent two pickup directions and a second pair of adjacent two pickup directions, the angle between the first pair of adjacent two pickup directions is a preset angle, the angle between the second pair of adjacent two pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not a pickup direction in the first pair of adjacent pickup directions; if the initial sound source direction is within a fourth pickup range, the fourth pickup range is the pickup range composed of the first pair of adjacent two pickup directions, and the sum of the wake-up parameters of the adjacent pickup directions is determined; if the sum of the wake-up parameters is not less than the product of the maximum wake-up parameter and the preset coefficient, the target sound source direction is determined to be the initial sound source direction.

[0202] In another possible implementation, the apparatus further includes:

[0203] The fourth determination module is used to determine the adjustment angle based on N wake-up parameters and N pickup directions if the position relationship is that the angle between at least one pair of adjacent two pickup directions in the N pickup directions is a preset angle, and if the initial sound source direction and the position relationship meet the preset conditions.

[0204] In another possible implementation, the apparatus further includes:

[0205] The fifth determining module is configured to determine the target sound source direction as the initial sound source direction if the positional relationship is such that the angle between no two adjacent sound pickup directions in the N sound pickup directions is the preset angle.

[0206] In another possible implementation, the apparatus further includes:

[0207] The sixth determination module is configured to adjust the initial sound source direction based on the M wake-up parameters and the M sound pickup directions to obtain the target sound source direction if the sound pickup direction with the smallest angle with the initial sound source direction is not the sound pickup direction corresponding to the largest wake-up parameter.

[0208] In another possible implementation, the apparatus further includes:

[0209] The seventh determination module is configured to determine the target sound source direction as the initial sound source direction if the sound pickup direction having the smallest angle with the initial sound source direction is the sound pickup direction corresponding to the maximum wake-up parameter.

[0210] In an embodiment of the present application, voice signals from multiple directions are picked up, and the positioning direction of the sound source is adjusted by determining the wake-up parameters corresponding to the voice signals. Since the wake-up parameters can indicate the contribution degree of the corresponding voice signal to waking up the terminal, adjusting the positioning direction of the sound source by using the wake-up parameters can improve the accuracy of determining the direction of the target sound source.

[0211] Optionally, the electronic device is provided as a terminal. Figure 7 The following is a block diagram of a terminal 700 according to an exemplary embodiment of the present application. Terminal 700 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. Terminal 700 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other similar names.

[0212] Typically, the terminal 700 includes a processor 701 and a memory 702 .

[0213] The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0214] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one instruction, which is executed by the processor 701 to implement the sound source direction determination method provided in the method embodiment of the present application.

[0215] In some embodiments, terminal 700 may optionally include a peripheral device interface 703 and at least one peripheral device. Processor 701, memory 702, and peripheral device interface 703 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 703 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 704, a touch screen display 705, a camera 706 assembly 707, an audio circuit 707, a positioning assembly 708, and a power supply 709.

[0216] The peripheral device interface 703 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 701 and the memory 702. In some embodiments, the processor 701, the memory 702, and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral device interface 703 can be implemented on separate chips or circuit boards, which is not limited in this embodiment of the present application.

[0217] The radio frequency circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 704 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The radio frequency circuit 704 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 704 may also include circuits related to NFC (Near Field Communication), which is not limited in this embodiment of the present application.

[0218] Display screen 705 is used to display a user interface (UI). This UI can include graphics, text, icons, videos, or any combination thereof. If display screen 705 is a touchscreen display, it can also capture touch signals on or above the surface of display screen 705. These touch signals can be input as control signals to processor 701 for processing. Display screen 705 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be a single display screen 705, located on the front panel of terminal 700. In other embodiments, there can be at least two display screens 705, located on different surfaces of terminal 700 or in a foldable design. In still other embodiments, display screen 705 can be a flexible display, located on a curved or foldable surface of terminal 700. Display screen 705 can also be configured as a non-rectangular, irregular shape, also known as a special-shaped screen. Display screen 705 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0219] The camera assembly 706 is used to capture images or videos. Optionally, the camera assembly 706 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0220] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals to be input into the processor 701 for processing, or input into the radio frequency circuit 704 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there can be multiple microphones, which are respectively arranged in different parts of the terminal 700. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signals into sound waves audible to humans, but also convert the electrical signals into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0221] Positioning component 708 is used to locate the current geographic location of terminal 700 to implement navigation or LBS (Location Based Service). Positioning component 708 can be based on the US GPS (Global Positioning System), China's BeiDou system, Russia's Greiner system, or the European Union's Galileo system.

[0222] Power supply 709 is used to power various components in terminal 700. Power supply 709 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 709 includes a rechargeable battery, the rechargeable battery can be wired or wirelessly rechargeable. A wired rechargeable battery is charged via a wired line, while a wireless rechargeable battery is charged via a wireless coil. The rechargeable battery can also support fast charging technology.

[0223] In some embodiments, the terminal 700 further includes one or more sensors 710 , including but not limited to: an acceleration sensor 711 , a gyroscope sensor 712 , a pressure sensor 713 , a fingerprint sensor 714 , an optical sensor 715 , and a proximity sensor 716 .

[0224] The accelerometer 711 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the terminal 700. For example, the accelerometer 711 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 701 can control the touch screen display 705 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 711. The accelerometer 711 can also be used to collect game or user motion data.

[0225] The gyroscope sensor 712 can detect the orientation and rotation angle of the terminal 700. It can work with the accelerometer 711 to collect the user's 3D movements on the terminal 700. Based on the data collected by the gyroscope sensor 712, the processor 701 can implement the following functions: motion sensing (for example, changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0226] The pressure sensor 713 can be set on the side frame of the terminal 700 and / or the lower layer of the touch screen display 705. When the pressure sensor 713 is set on the side frame of the terminal 700, it can detect the user's grip signal of the terminal 700, and the processor 701 performs left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 713. When the pressure sensor 713 is set on the lower layer of the touch screen display 705, the processor 701 controls the operable controls on the UI interface based on the user's pressure operation on the touch screen display 705. The operable controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0227] The fingerprint sensor 714 is used to collect the user's fingerprint. The processor 701 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 714, or the fingerprint sensor 714 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 701 authorizes the user to perform relevant sensitive operations, such as unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 714 can be set on the front, back, or side of the terminal 700. When a physical button or manufacturer logo is set on the terminal 700, the fingerprint sensor 714 can be integrated with the physical button or manufacturer logo.

[0228] The optical sensor 715 is used to detect ambient light intensity. In one embodiment, the processor 701 can control the display brightness of the touchscreen display 705 based on the ambient light intensity detected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the touchscreen display 705 is increased; when the ambient light intensity is low, the display brightness of the touchscreen display 705 is decreased. In another embodiment, the processor 701 can also dynamically adjust the shooting parameters of the camera assembly 706 based on the ambient light intensity detected by the optical sensor 715.

[0229] Proximity sensor 716, also known as a distance sensor, is typically located on the front panel of terminal 700. Proximity sensor 716 is used to detect the distance between the user and the front of terminal 700. In one embodiment, when proximity sensor 716 detects that the distance between the user and the front of terminal 700 is gradually decreasing, processor 701 controls touchscreen display 705 to switch from the screen-on state to the screen-off state. When proximity sensor 716 detects that the distance between the user and the front of terminal 700 is gradually increasing, processor 701 controls touchscreen display 705 to switch from the screen-off state to the screen-on state.

[0230] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the terminal 700, and the terminal 700 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0231] Optionally, the electronic device is provided as a server. Figure 8 8 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 800 may vary significantly due to different configurations or performances, and may include one or more processors (Central Processing Units, CPUs) 801 and one or more memories 802, wherein the memories 802 store at least one computer program, which is loaded and executed by the processor 801 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0232] In an exemplary embodiment, a computer-readable storage medium is also provided. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a terminal to implement the sound source direction determination method in the above-described embodiment. The computer-readable storage medium can be a memory. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0233] In an exemplary embodiment, a computer program product is further provided. The computer program product includes computer program code. When the computer program code is executed by a processor, the computer implements the sound source direction determination method in the above embodiment.

[0234] In an exemplary embodiment, the computer program involved in the embodiments of the present application can be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. Multiple computer devices distributed at multiple locations and interconnected through a communication network can constitute a blockchain system.

[0235] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0236] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for determining the direction of a sound source, characterized in that: The method comprises: Based on M pickup directions, M voice signals are picked up, each voice signal corresponds to a pickup direction, and the M voice signals are used to wake up the terminal, where M is an integer greater than 2; Determining, based on the M voice signals, M wake-up parameters and an initial sound source direction, the M wake-up parameters being used to indicate a contribution degree of the M voice signals to waking up the terminal; Based on the M wake-up parameters and the M sound pickup directions, selecting N sound pickup directions from the M sound pickup directions, where the N wake-up parameters of the N sound pickup directions are the first N wake-up parameters of the M wake-up parameters arranged from largest to smallest, and N is an integer less than or equal to M and greater than 1; Based on the N sound pickup directions, determining a positional relationship between the N sound pickup directions; The angle between any two adjacent pickup directions among the M pickup directions is a preset angle; if the positional relationship is that there is at least one pair of adjacent two pickup directions among the N pickup directions, the angle between them is the preset angle, then if the initial sound source direction and the positional relationship meet the preset conditions, based on the N wake-up parameters and the N pickup directions, an adjustment angle is determined; based on the adjustment angle, the initial sound source direction is adjusted to obtain the target sound source direction.

2. The method according to claim 1, characterized in that The method further comprises: If the initial sound source direction and the position relationship do not satisfy a preset condition, the target sound source direction is determined to be the sound pickup direction corresponding to the maximum wake-up parameter.

3. The method according to claim 2, characterized in that If N is greater than 2 and less than M; if the relationship between the initial sound source direction and the position satisfies a preset condition, determining an adjustment angle based on the N wake-up parameters and the N sound pickup directions, including: The positional relationship is that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is not a boundary pickup direction of the first pickup range, the first pickup range is a pickup range composed of the N pickup directions, and the boundary pickup direction is two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle. If the initial sound source direction is within the first pickup range, determine a first weight and a first angle of the pickup direction with the smallest angle to the initial sound source direction; The adjustment angle is determined based on the first weight and the first angle.

4. The method according to claim 2, characterized in that If N is greater than 2 and less than M; if the relationship between the initial sound source direction and the position satisfies a preset condition, determining an adjustment angle based on the N wake-up parameters and the N sound pickup directions, including: The positional relationship is that there is only a pair of adjacent two pickup directions among the N pickup directions, and the angle between them is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is one of the boundary pickup directions of the first pickup range, the first pickup range is a pickup range composed of the N pickup directions, and the boundary pickup directions are two adjacent pickup directions in the first pickup range whose angle is greater than the preset angle; if the initial sound source direction is within the second pickup range, the second pickup range is a pickup range composed of other pickup directions except the pickup direction corresponding to the maximum wake-up parameter, and second weights of the other pickup directions are determined; The adjustment angle is determined based on the second weight.

5. The method according to claim 2, characterized in that: If N is equal to 2; if the relationship between the initial sound source direction and the position meets a preset condition, determining the adjustment angle based on the N wake-up parameters and the N sound pickup directions includes: The positional relationship is that the N pickup directions are adjacent, and the pickup direction corresponding to the maximum wake-up parameter is any one of the N pickup directions. If the initial sound source direction is within a first pickup range, the first pickup range is a pickup range composed of the N pickup directions, and a third weight and a second angle are determined for the pickup direction having the smallest angle with the initial sound source direction. The adjustment angle is determined based on the third weight and the second angle.

6. The method according to claim 2, characterized in that: If the N is equal to the M, the method further includes: If the initial sound source direction is within the sound pickup range, and the first sound pickup range is a sound pickup range consisting of the N sound pickup directions, determining a fourth weight and a third angle of the sound pickup direction having the smallest angle with the initial sound source direction; The adjustment angle is determined based on the fourth weight and the third angle.

7. The method according to claim 2, characterized in that: If the relationship between the initial sound source direction and the position satisfies a preset condition, determining an adjustment angle based on the N wake-up parameters and the N sound pickup directions includes: The positional relationship is that there are at least a first pair of two adjacent pickup directions and a second pair of two adjacent pickup directions among the N pickup directions, the angle between the first pair of two adjacent pickup directions is the preset angle, the angle between the second pair of two adjacent pickup directions is greater than the preset angle, and the pickup direction corresponding to the maximum wake-up parameter is any pickup direction of the first pair of two adjacent pickup directions; if the initial sound source direction is within a third pickup range, the third pickup range is a pickup range formed by the first pair of two adjacent pickup directions, and a fifth weight and a fourth angle are determined between adjacent pickup directions of the pickup direction corresponding to the maximum wake-up parameter; The adjustment angle is determined based on the fifth weight and the fourth angle.

8. The method according to claim 2, characterized in that: The method further comprises: The positional relationship is that there are at least a first pair of two adjacent sound pickup directions and a second pair of two adjacent sound pickup directions among the N sound pickup directions, the angle between the first pair of two adjacent sound pickup directions is the preset angle, the angle between the second pair of two adjacent sound pickup directions is greater than the preset angle, and the sound pickup direction corresponding to the maximum wake-up parameter is not a pick-up direction in the first pair of two adjacent sound pickup directions; if the initial sound source direction is within a fourth sound pickup range, the fourth sound pickup range is the pick-up range formed by the first pair of two adjacent sound pickup directions, and the sum of the wake-up parameters of the adjacent sound pickup directions is determined; If the sum of the wake-up parameters is not less than the product of the maximum wake-up parameter and a preset coefficient, the target sound source direction is determined to be the initial sound source direction.

9. The method according to claim 2, characterized in that: The method further comprises: If the positional relationship is such that no two adjacent sound pickup directions in the N sound pickup directions have an angle equal to the preset angle, the target sound source direction is determined to be the initial sound source direction.

10. The method according to claim 1, characterized in that: The method further comprises: If the pickup direction having the smallest angle with the initial sound source direction is not the pickup direction corresponding to the maximum wake-up parameter, the step of adjusting the initial sound source direction based on the M wake-up parameters and the M pickup directions to obtain a target sound source direction is performed.

11. The method according to claim 1, characterized in that: The method further comprises: If the sound pickup direction having the smallest angle with the initial sound source direction is the sound pickup direction corresponding to the maximum wake-up parameter, the target sound source direction is determined to be the initial sound source direction.

12. A device for determining the direction of a sound source, characterized in that: The device comprises: A pickup module, configured to pick up M voice signals based on M pickup directions, each voice signal corresponding to a pickup direction, and the M voice signals are used to wake up the terminal, where M is an integer greater than 2; A first determination module is configured to determine M wake-up parameters and an initial sound source direction based on the M voice signals, wherein the M wake-up parameters are used to indicate the contribution of the M voice signals to waking up the terminal; An adjustment module is configured to select N pickup directions from the M pickup directions based on the M wake-up parameters and the M pickup directions, where the N wake-up parameters of the N pickup directions are the first N wake-up parameters of the M wake-up parameters arranged from large to small, and N is an integer less than or equal to M and greater than 1; determine the positional relationship of the N pickup directions based on the N pickup directions; the angle between any two adjacent pickup directions in the M pickup directions is a preset angle; if the positional relationship is that there is at least one pair of adjacent two pickup directions in the N pickup directions with the angle being the preset angle, then if the initial sound source direction and the positional relationship meet a preset condition, determine an adjustment angle based on the N wake-up parameters and the N pickup directions; and adjust the initial sound source direction based on the adjustment angle to obtain a target sound source direction.

13. An electronic device, characterized in that: The electronic device includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the sound source direction determination method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that At least one program code is stored in the storage medium, and the at least one program code is loaded and executed by the processor to implement the sound source direction determination method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product includes a computer program code, which is stored in a computer-readable storage medium. The processor of the electronic device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the electronic device performs the sound source direction determination method according to any one of claims 1 to 11.

Images