Precise pickup voice positioning method and device, terminal and storage medium

Abstract

The invention discloses a precise pickup voice positioning method and device, a terminal and a storage medium. The method comprises the following steps: when a user enters a receiving range of a millimeter wave receiver, calculating first user position information according to a detection signal received by the millimeter wave receiver; when a wake-up instruction of the user is obtained, calculating second user position information according to the sound signal received by the microphone array; and calculating target user position information according to the first user position information and the second user position information. The millimeter wave receiver can provide high distance resolution and angle resolution, and a user can be accurately positioned. The microphone array may utilize acoustic characteristics of sound to locate a user in a horizontal direction and / or a vertical direction. According to the invention, the position information of the user is determined through the millimeter wave receiver and the microphone array, then the two pieces of position information are combined to obtain the accurate position information of the user, and accurate pickup positioning of the smart home is realized through a multi-sensor fusion mode.

Description

Technical Field

[0001] The present invention relates to the technical field of voice processing, and particularly to a voice positioning method, device, terminal and storage medium for accurate sound pickup. Background Art

[0002] With the rapid popularization and development of the Internet of Things, as the main way for humans to obtain information, sound has gradually become an important means of communication and human-computer interaction. Therefore, many interactive applications based on sound perception methods have been developed in various industries, such as smart homes.

[0003] At present, most smart homes mainly use microphones as sensors and perform pure audio perception through microphones. According to the number of microphones, it can be further divided into single microphone sound perception methods and multi-microphone array sound perception methods.

[0004] However, relying solely on a single microphone sensor for sound perception is difficult to achieve an accurate sound pickup effect.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a voice positioning method, device, terminal and storage medium for accurate sound pickup in view of the above-mentioned defects of the existing technology, aiming to solve the problem that in the existing technology, smart homes rely solely on a single microphone sensor for sound perception and it is difficult to achieve an accurate sound pickup effect.

[0007] The technical solution adopted by the present invention to solve the problem is as follows:

[0008] In a first aspect, an embodiment of the present invention provides a voice positioning method for accurate sound pickup, and the method includes:

[0009] When a user enters the reception range of a millimeter wave receiver, calculate first user position information according to the detection signal received by the millimeter wave receiver;

[0010] When a wake-up instruction of the user is obtained, calculate second user position information according to the sound signal received by the microphone array;

[0011] Calculate target user position information according to the first user position information and the second user position information.

[0012] In an implementation manner, calculating first user position information according to the detection signal received by the millimeter wave receiver includes:

[0013] Calculate time delay data according to the detection signal received by the millimeter wave receiver;

[0014] Calculate the distance between the user and the millimeter-wave receiver based on the latency data to obtain user distance information;

[0015] Calculate the azimuth angle and elevation angle of the user based on the phase difference between the receiving antennas of the millimeter-wave receiver to obtain user angle information;

[0016] Calculate the first user position information based on the user distance information and the user angle information.

[0017] In one implementation, calculating the second user position information based on the sound signal received by the microphone array includes:

[0018] Calculate the sound source distance information and time difference information based on the sound signal received by the microphone array; wherein, the time difference information is used to reflect the time difference of the sound reaching different microphones in the microphone array;

[0019] Calculate the second user position information based on the sound source distance information and the time difference information.

[0020] In one implementation, calculating the second user position information based on the sound source distance information and the time difference information includes:

[0021] Obtain the structure information of the microphone array;

[0022] Calculate the angle of arrival based on the time difference information and the structure information of the microphone array, wherein the angle of arrival is used to reflect the azimuth angle and elevation angle of the sound source;

[0023] Calculate the second user position information based on the angle of arrival and the sound source distance information.

[0024] In one implementation, calculating the target user position information based on the first user position information and the second user position information includes:

[0025] Calculate the position distance between the first user position information and the second user position information, and determine whether the position distance is greater than a preset distance error value;

[0026] If the position distance is greater than the distance error value, recalculate the first user position information and the second user position information;

[0027] If the position distance is less than or equal to the distance error value, fuse the first user position information and the second user position information to obtain the target user position information.

[0028] In one implementation, the method further includes:

[0029] According to the time difference information, perform weighted summation on the sound signals received by each microphone in the microphone array to obtain a directional beam;

[0030] Determine the target sound source direction according to the directional beam, and enhance the sound signal corresponding to the target sound source direction.

[0031] In one implementation, the method further includes:

[0032] Perform noise reduction on the sound signals in the remaining directions except the target sound source direction.

[0033] In a second aspect, an embodiment of the present invention further provides a voice positioning device for accurate sound pickup, and the device includes:

[0034] A first positioning module, configured to calculate first user position information according to the detection signal received by the millimeter wave receiver when the user enters the reception range of the millimeter wave receiver;

[0035] A second positioning module, configured to calculate second user position information according to the sound signal received by the microphone array when a wake-up instruction of the user is obtained;

[0036] A fusion positioning module, configured to calculate target user position information according to the first user position information and the second user position information.

[0037] In a third aspect, an embodiment of the present invention further provides a terminal, and the terminal includes a memory and more than one processor; the memory stores more than one program; the program includes instructions for executing the voice positioning method for accurate sound pickup as described in any one of the above; the processor is configured to execute the program.

[0038] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which multiple instructions are stored, and the instructions are suitable for being loaded and executed by a processor to implement the steps of the voice positioning method for accurate sound pickup as described in any one of the above.

[0039] Advantages of the present invention: When the user enters the reception range of the millimeter-wave receiver in the embodiments of the present invention, the first user position information is calculated according to the detection signal received by the millimeter-wave receiver; when a wake-up instruction of the user is obtained, the second user position information is calculated according to the sound signal received by the microphone array; according to the first user position information and the second user position information, the target user position information is calculated. The millimeter-wave receiver can provide high range resolution and angle resolution, and can position the user more accurately. The microphone array can use the acoustic characteristics of sound to position the user in the horizontal direction and / or the vertical direction. The present invention first determines the position information of the user through the millimeter-wave receiver and the microphone array respectively, and then combines the two position information to finally obtain the accurate position information of the user. Precise voice pickup of the smart home is realized through the multi-sensor fusion method. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0041] Figure 1 It is a flowchart of the voice positioning method for precise voice pickup provided by the embodiments of the present invention.

[0042] Figure 2 It is a scenario diagram of the voice positioning method for precise voice pickup provided by the embodiments of the present invention.

[0043] Figure 3 It is a schematic diagram of the functional modules of the voice positioning method device for precise voice pickup provided by the embodiments of the present invention.

[0044] Figure 4 It is a schematic block diagram of the terminal provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The present invention discloses a voice positioning method, device, terminal and storage medium for precise voice pickup. To make the purpose, technical solutions and effects of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present invention means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more of the associated listed items.

[0047] Those skilled in the art can understand that, unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention pertains. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0048] With the rapid popularization and development of the Internet of Things, sound, as the main way for humans to obtain information, has gradually become an important means of communication and human-computer interaction. Therefore, many interactive applications based on sound perception methods have been developed in various industries, such as smart homes.

[0049] At present, most smart homes mainly use microphones as sensors and perform pure audio perception through microphones. According to the number of microphones, it can be further divided into single microphone sound perception methods and multi-microphone array sound perception methods.

[0050] However, relying solely on a single microphone sensor for sound perception is difficult to achieve accurate sound pickup effects.

[0051] In view of the above defects of the prior art, the present invention provides a voice positioning method for accurate sound pickup. The method includes: when a user enters the reception range of a millimeter-wave receiver, calculating first user position information according to a detection signal received by the millimeter-wave receiver; when a wake-up instruction of the user is obtained, calculating second user position information according to a sound signal received by a microphone array; calculating target user position information according to the first user position information and the second user position information. The millimeter-wave receiver can provide high range resolution and angle resolution, and can locate the user relatively accurately. The microphone array can use the acoustic characteristics of sound to locate the user in the horizontal direction and / or the vertical direction. The present invention first determines the position information of the user through the millimeter-wave receiver and the microphone array respectively, and then combines the two position information to finally obtain the accurate position information of the user. Accurate sound pickup of the smart home is realized through the multi-sensor fusion method.

[0052] As Figure 1 shown, the method includes:

[0053] Step S100, when a user enters the reception range of a millimeter-wave receiver, calculating first user position information according to a detection signal received by the millimeter-wave receiver.

[0054] Specifically, when the user is within the reception range of the millimeter-wave receiver, the detection signal received by the millimeter-wave receiver may include information such as reflection and attenuation of millimeter-wave signals. The millimeter-wave band has a relatively wide bandwidth and can provide high range resolution and angle resolution. Therefore, by analyzing and calculating the detection signal, the user can be located. In this embodiment, the user position obtained based on the detection signal received by the millimeter-wave receiver is defined as the first user position information.

[0055] In one implementation, calculating first user position information according to the detection signal received by the millimeter-wave receiver includes:

[0056] Calculating delay data according to the detection signal received by the millimeter-wave receiver;

[0057] Calculating the distance between the user and the millimeter-wave receiver according to the delay data to obtain user distance information;

[0058] Calculating the azimuth angle and elevation angle of the user according to the phase difference between the receiving antennas of the millimeter-wave receiver to obtain user angle information;

[0059] Calculating first user position information according to the user distance information and the user angle information.

[0060] Specifically, the detection signal received by the millimeter wave includes the signal reflected by the target (i.e., the user). By comparing the time difference between the transmitted signal and the received signal, the time delay data is calculated. The time delay data reflects the time experienced by the signal from transmission to reception, and can be used as the basic data for calculating the distance between the user and the millimeter wave receiver, thereby obtaining the user distance information. The phase difference is caused by the distance and / or angle difference between the target and different antennas of the millimeter wave receiver, and can be used as the basic data for calculating the angle deviation of the user relative to the due front of the millimeter wave receiver, thereby obtaining the user angle information. Finally, by integrating the user distance information and the user angle information, the position information of the user in space can be determined using geometric calculation methods, that is, the first user position information is obtained.

[0061] In one implementation, when the detection signal received by the millimeter wave receiver includes multiple reflected signals, the position information of multiple targets can be detected simultaneously, that is, this embodiment is also applicable to multi-user scenarios.

[0062] Step S200: When a wake-up instruction of the user is obtained, calculate the second user position information according to the sound signal received by the microphone array.

[0063] Generally speaking, in this embodiment, when a person enters the millimeter wave reception range, the position where the person is located is calculated by the millimeter wave receiver; when the person issues a wake-up instruction, the position information of the person is obtained by receiving the strength information of the human voice through the microphone array, and the two are combined to lock the specific position range where the person is located (as Figure 2 shown). Specifically, when a wake-up instruction of the user is obtained, it indicates that the current user has a specific interaction requirement. The microphone array can receive the sound signal of the user. Due to the characteristics of sound propagation, for example, the time difference and phase difference of the sound arriving at each microphone of the microphone array from different directions have different manifestations. Therefore, by processing and calculating these sound signals and using the acoustic characteristics, the user can be located in the horizontal direction and / or the vertical direction. The user position obtained based on the sound signal received by the microphone array in this embodiment is defined as the second user position information.

[0064] In one implementation, calculating the second user position information according to the sound signal received by the microphone array includes:

[0065] Calculate the sound source distance information and the time difference information according to the sound signal received by the microphone array; wherein, the time difference information is used to reflect the time difference of the sound arriving at different microphones of the microphone array;

[0066] Calculate the second user position information according to the sound source distance information and the time difference information.

[0067] Specifically, for the sound signal received by the microphone array, the distance between the sound source and the microphone array can be calculated by using the sound wave propagation speed and the time when the sound reaches the microphone array, so as to obtain the sound source distance information, which is used as the basic data for calculating the sound source position. At the same time, since there are differences in the time when the user's voice reaches different microphones in the microphone array, by calculating these time differences, the propagation time delay of the sound between different microphones can be obtained, so as to obtain the time difference information, which is used as the basic data for calculating the sound source position. Finally, by combining the sound source distance information and the time difference information, the specific position of the sound source in space can be determined by using geometric calculation methods, that is, the second user position information is obtained.

[0068] In one implementation, calculating the second user position information according to the sound source distance information and the time difference information includes:

[0069] Obtain the structure information of the microphone array;

[0070] Calculate the arrival angle according to the time difference information and the structure information of the microphone array, where the arrival angle is used to reflect the azimuth angle and elevation angle of the sound source;

[0071] Calculate the second user position information according to the arrival angle and the sound source distance information.

[0072] Specifically, the structure information of the microphone array can be obtained through the device parameter file. The structure information includes but is not limited to the number of microphones and the relative position relationship between the microphones, such as the arrangement method, spacing, etc. When the user issues a wake-up command, the user is the sound source. Since the distances between the microphones in the microphone array and the sound source are different, the time when the sound reaches each microphone will be different. By obtaining the relative position information of each microphone through the structure information of the microphone array and combining the time difference information, the direction of the sound source can be roughly judged, that is, the arrival angle of the sound is calculated. The arrival angle includes the azimuth angle and elevation angle of the sound source. The azimuth angle reflects the direction of the sound source on the horizontal plane, and the elevation angle reflects the direction of the sound source on the vertical plane. Taking a two-microphone linear array as an example, by calculating the time difference when the sound reaches the two microphones and combining the spacing between them, the azimuth angle of the sound source relative to the microphone array can be calculated. The arrival angle reflects the direction of the sound source, and the position of the sound source can be accurately determined only by combining the sound source distance information. The sound source distance information can be obtained by means of signal strength ranging. For example, the sound intensity decays with distance, and the sound source distance information can be deduced according to the intensity of the received sound signal. By combining the arrival angle and the sound source distance information, the second user position information can be obtained through geometric calculation methods.

[0073] In one implementation, the method for calculating the angle of arrival specifically includes: calculating the distance difference of sound propagation based on the speed of sound and the time difference information, and calculating the angle of arrival based on the distance difference of sound propagation and the structural information of the microphone array (for example, using trigonometric function relationships).

[0074] Step S300: Calculate the target user location information according to the first user location information and the second user location information.

[0075] Generally speaking, this embodiment combines millimeter wave and sound source localization functions, which can more accurately determine whether the human voice is within the effective range, thereby ensuring the effectiveness of sound collection, strengthening the sound collection intensity at this angle, picking up sound more accurately, and ensuring effectiveness and accuracy. Specifically, this embodiment fuses the first user location information obtained by the millimeter wave receiver and the second user location information obtained by the microphone array to obtain a more accurate and reliable location information. This embodiment defines the location information obtained after fusion as the target user location information. Compared with the positioning method of a single sensor, this embodiment combines the positioning results of multiple sensors, enabling each sensor to make up for each other's deficiencies, thereby improving the overall positioning accuracy. In addition, when one of the positioning methods fails or is severely interfered, the other positioning method can still provide certain positioning information to ensure that the entire system does not completely lose the ability to judge the user's location.

[0076] In one implementation, calculating the target user location information according to the first user location information and the second user location information includes:

[0077] Calculating the location distance between the first user location information and the second user location information, and determining whether the location distance is greater than a preset distance error value;

[0078] If the location distance is greater than the distance error value, recalculate the first user location information and the second user location information;

[0079] If the location distance is less than or equal to the distance error value, fuse the first user location information and the second user location information to obtain the target user location information.

[0080] Specifically, the first user location information and the second user location information are obtained based on the millimeter-wave receiver positioning method and the microphone array positioning method respectively. The preset distance error value is a threshold set according to the application scenario, and this value reflects the acceptable position error range. For example, in an indoor small-range positioning scenario, the distance error value is set to a small value; in an outdoor large-range positioning scenario, the distance error value is set to a large value. Calculate the position distance between the first user location information and the second user location information. If the position distance between the two is greater than the set distance error value, it indicates that the difference between the user location information calculated by the two positioning methods is large and the positioning is inaccurate. Therefore, in order to ensure the accuracy of the subsequent calculation results, it is necessary to recalculate the first user location information through the millimeter-wave receiver positioning method and recalculate the second user location information through the microphone array positioning method. If the position distance between the two is less than or equal to the set distance error value, it indicates that the user location information calculated by the two positioning methods is consistent within the acceptable error range, then the user location information calculated by the two positioning methods is fused to lock the specific position range where the user is located to obtain the target user location information.

[0081] Kalman filtering is a dynamic data fusion algorithm, suitable for processing sensor data with noise, and can update the position estimate in real time. In one implementation, fusing the first user location information and the second user location information to obtain the target user location information includes:

[0082] Initial stage: Establish a state model and define the state vector, including position and velocity.

[0083] Prediction stage: Predict the state at the current moment according to the state and motion model at the previous moment.

[0084] Update stage: Use the user location information obtained by positioning the millimeter-wave receiver and the microphone array respectively (i.e., the first user location information and the second user location information) as the observed values, combine with the predicted values, and update the state estimate through the Kalman gain to obtain the updated state data.

[0085] Output stage: Determine the fused position through the updated state data to obtain the target user location information.

[0086] Particle filtering is a non-linear filtering algorithm based on the Monte Carlo method, suitable for dealing with non-Gaussian noise and multi-modal distribution problems. In another implementation, fusing the first user location information and the second user location information to obtain the target user location information includes:

[0087] Initialize the particle swarm: Generate a set of random particles, and each particle represents a possible position state.

[0088] Prediction stage: Update the positions of the particles according to the motion model.

[0089] Weight update: Calculate the weight of each particle according to the user position information (i.e., the first user position information and the second user position information) obtained by positioning with the millimeter-wave receiver and the microphone array respectively.

[0090] Resampling: Resample the particles according to the weights, and retain the particles with high weights.

[0091] Output the fused position: Determine the fused position according to the weighted average of the particles to obtain the target user position information.

[0092] In one implementation, the method further includes:

[0093] According to the time difference information, perform weighted summation on the sound signals received by each microphone in the microphone array to obtain a directional beam;

[0094] Determine the target sound source direction according to the directional beam, and enhance the sound signal corresponding to the target sound source direction.

[0095] Specifically, when the sound source emits sound, the sound will reach different microphones in the microphone array simultaneously. However, due to the different distances between each microphone and the sound source, there will be a difference in the arrival time of the sound. For example, the microphone closer to the sound source will receive the sound signal first. According to the sound propagation speed and the time difference information, the distance between each microphone and the sound source can be calculated, and then the weighting coefficient of the sound signal received by each microphone can be calculated according to the distance between each microphone and the sound source. Then, the sound signals received by each microphone are synthesized according to the weight coefficients, so that the sound signals in the wake-up angle direction are enhanced.

[0096] In one implementation, the method further includes:

[0097] Perform noise reduction on the sound signals in the directions other than the target sound source direction.

[0098] Reduce the noise in other angles, so as to achieve a more accurate sound pickup effect.

[0099] Specifically, in order to further reduce the interfering sound, in this embodiment, while enhancing the sound signal in the target sound source direction, the sound signals in the other directions are also subjected to noise reduction. Especially in a multi-person scenario, this sound optimization method can more effectively reduce the interfering sound.

[0100] Based on the above embodiments, the present invention further provides a voice positioning device for accurate sound pickup, as Figure 3 described, the device includes:

[0101] The first positioning module 01 is used to calculate the first user position information according to the detection signal received by the millimeter-wave receiver when the user enters the reception range of the millimeter-wave receiver;

[0102] The second positioning module 02 is used to calculate the second user position information according to the sound signal received by the microphone array when a wake-up instruction of the user is obtained;

[0103] The fusion positioning module 03 is used to calculate the target user position information according to the first user position information and the second user position information.

[0104] The advantages of the present invention are as follows:

[0105] 1. Improve positioning accuracy: A single positioning method may have errors. For example, the millimeter-wave receiver may be affected in some complex environments, and the microphone array may be affected by environmental noise and multipath effects. The present invention can calculate the angular range of the sound source through the combination of the microphone array and sound source positioning. The millimeter-wave receiver accurately determines the position of the sound source according to the position where the sound source is located. In the actual application scenario, when the user wakes up, the sound source positioning of the microphone is combined with the millimeter-wave function to accurately locate the angle where the sound source is located. By combining the positioning results of the two sensors, the deficiencies of each sensor can be compensated for each other, thereby improving the overall positioning accuracy.

[0106] 2. Enhance the robustness of positioning: When one of the positioning methods of the millimeter-wave receiver and the microphone array fails or is severely interfered, the other positioning method can still provide certain positioning information, ensuring that the entire system will not completely lose the ability to judge the user's position.

[0107] 3. Adapt to various scenario requirements: In different application scenarios, the combination of the two positioning methods of the millimeter-wave receiver and the microphone array can better meet the requirements. For example, in the smart home scenario, the positioning method of the millimeter-wave receiver can be used to initially determine the approximate position of the user in the room. When the user issues a wake-up instruction, the positioning method of the microphone array can further accurately determine the specific position of the user when interacting with the device, so as to better adjust the pointing of the device or provide other personalized services.

[0108] 4. Improve the user experience: More accurate position information allows users to enjoy more intelligent and convenient services. For example, in the intelligent security scenario, accurate positioning can enable the monitoring system to timely discover the position of users with abnormal behaviors and give an alarm in a timely manner; in the intelligent business scenario, merchants can provide accurate product recommendations or promotional activities according to the user's position.

[0109] Based on the above embodiments, the present invention also provides a terminal, and its principle block diagram can be as Figure 4As shown. The terminal includes a processor, a memory, a network interface, and a display screen connected via a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a voice localization method for accurate voice pickup. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0110] Those skilled in the art can understand that Figure 4 the block diagram of the principle shown is only a block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0111] In one implementation, more than one program is stored in the memory of the terminal, and is configured to be executed by more than one processor. The more than one program includes instructions for performing a voice localization method for accurate voice pickup.

[0112] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or an external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0113] In summary, the present invention discloses a voice localization method, device, terminal, and storage medium for precise sound pickup. The method includes: when a user enters the reception range of a millimeter-wave receiver, calculating first user position information based on the detection signal received by the millimeter-wave receiver; when a wake-up instruction of the user is obtained, calculating second user position information based on the sound signal received by a microphone array; and calculating target user position information based on the first user position information and the second user position information. The millimeter-wave receiver can provide high range resolution and angle resolution, and can locate the user relatively accurately. The microphone array can use the acoustic characteristics of sound to locate the user in the horizontal direction and / or the vertical direction. The present invention first determines the user's position information through the millimeter-wave receiver and the microphone array respectively, and then combines the two position information to finally obtain the precise position information of the user. Precise sound pickup of a smart home is achieved through a multi-sensor fusion method.

[0114] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for accurately picking up voice localization, characterized in that: The method comprises: When the user enters the receiving range of the millimeter wave receiver, calculating the first user location information according to the detection signal received by the millimeter wave receiver; When a wake-up instruction from the user is obtained, calculating the second user position information according to the sound signal received by the microphone array; Calculate target user location information based on the first user location information and the second user location information.

2. The method for accurate voice positioning according to claim 1, characterized in that: Calculating first user location information according to the detection signal received by the millimeter wave receiver includes: Calculating delay data according to the detection signal received by the millimeter wave receiver; Calculate the distance between the user and the millimeter wave receiver according to the delay data to obtain user distance information; Calculate the azimuth and elevation angles of the user according to the phase difference between the receiving antennas of the millimeter wave receiver to obtain the user angle information; First user position information is calculated according to the user distance information and the user angle information.

3. The method for accurate voice positioning according to claim 1, characterized in that: Calculating the second user's location information according to the sound signal received by the microphone array includes: Calculating the sound source distance information and the time difference information according to the sound signal received by the microphone array; wherein the time difference information is used to reflect the time difference when the sound reaches different microphones in the microphone array; The second user position information is calculated according to the sound source distance information and the time difference information.

4. The method for accurate voice positioning according to claim 3, characterized in that: Calculating the second user position information according to the sound source distance information and the time difference information includes: Acquiring structural information of the microphone array; Calculating an angle of arrival according to the time difference information and the structural information of the microphone array, wherein the angle of arrival is used to reflect the azimuth and elevation angle of the sound source; The second user position information is calculated according to the arrival angle and the sound source distance information.

5. The method for accurate voice positioning according to claim 1, characterized in that: Calculating target user location information according to the first user location information and the second user location information includes: Calculating the location distance between the first user location information and the second user location information, and determining whether the location distance is greater than a preset distance error value; If the location distance is greater than the distance error value, recalculating the first user location information and the second user location information; If the location distance is less than or equal to the distance error value, the first user location information and the second user location information are fused to obtain the target user location information.

6. The method for accurate voice positioning according to claim 4, characterized in that: The method further comprises: According to the time difference information, weighted summation is performed on the sound signals received by each microphone in the microphone array to obtain a directional beam; The target sound source direction is determined according to the directional beam, and the sound signal corresponding to the target sound source direction is enhanced.

7. The method for accurate voice positioning according to claim 6, characterized in that: The method further comprises: Noise reduction is performed on sound signals in directions other than the target sound source direction.

8. A voice positioning device for accurate sound pickup, characterized in that: The device comprises: A first positioning module, configured to calculate first user location information according to a detection signal received by the millimeter wave receiver when the user enters a receiving range of the millimeter wave receiver; A second positioning module is used to calculate the second user position information according to the sound signal received by the microphone array when the user's wake-up instruction is obtained; The fusion positioning module is used to calculate the target user location information based on the first user location information and the second user location information.

9. A terminal, characterized in that: The terminal includes a memory and one or more processors; the memory stores one or more programs; the program contains instructions for executing the voice localization method for precise sound pickup as described in any one of claims 1-7; and the processor is used to execute the program.

10. A computer-readable storage medium having a plurality of instructions stored thereon, characterized in that: The instructions are suitable for being loaded and executed by a processor to implement the steps of the voice localization method for accurate sound pickup as described in any one of claims 1-7.

Images