Sound source localization method, device, and storage medium

The camera determines the credibility of the sound source position and dynamically updates the credibility interval, which solves the problem of wrong positioning of traditional devices under sound reflection, and improves the accuracy and user experience of sound source positioning.

WO2024244528A1PCT designated stage expired Publication Date: 2024-12-05HUAWEI TECH CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/075741
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-30
Filing Date
2024-02-04
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

After receiving the user wake-up command, traditional devices may cause errors in positioning the sound source due to reflections of sound, causing the back of the device to face the user, reducing the user experience.

Method used

The camera determines the credibility of the sound source position, and only turns to the sound source position when the sound source position is trustworthy, and turns to the commonly used position of the user in case of untrustworthy conditions or uses vertical direction detection to improve positioning accuracy and user experience .

Benefits of technology

It improves the accuracy and user experience of sound source positioning, avoids erroneous turn caused by sound reflection, and enhances the device's detection success rate of users.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024075741_05122024_PF_FP_ABST
    Figure CN2024075741_05122024_PF_FP_ABST
Patent Text Reader

Abstract

A sound source localization method, a device, and a storage medium. The method comprises: in response to detecting a wake-up instruction of a user, determining, by means of a camera, whether a sound source location is trustworthy (101), the sound source location being a vocalization location of the user calculated by a device; and if it is determined that the sound source location is trustworthy, horizontally turning to the sound source location (104). The method is conducive to improving the accuracy of sound source localization.
Need to check novelty before this filing date? Find Prior Art

Description

Sound source localization method, device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on May 30, 2023, with application number 202310628610.5 and application name “Sound Source Localization Method, Device and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of intelligent robots, and in particular to a sound source localization method, device, and storage medium. Background Art

[0003] When a traditional device (e.g., a robot) receives a wake-up command from a user, it may mislocalize the sound source due to sound reflections if there are obstacles (e.g., walls) around it. For example, if a robot is placed against a wall and the user wakes it up, the robot will face the user directly if it correctly locates the sound source. However, due to sound reflections, the robot may mislocalize the sound source, causing it to face the user with its back, thus reducing the user experience.

[0004] Summary of the Invention

[0005] The present application provides a sound source localization method, device, and storage medium, which help to improve the accuracy of sound source localization.

[0006] A first aspect provides a sound source localization method, which is applied to a device, wherein the device includes a camera, including:

[0007] In response to detecting a user's wake-up command, determining, by the camera, whether a sound source position is credible, the sound source position being a position of the user's voice calculated by the device;

[0008] If the sound source position is determined to be reliable, turn horizontally to the sound source position.

[0009] In this application, by judging the credibility of the sound source position, steering to the sound source position only when the sound source position is credible can improve the accuracy of sound source positioning and avoid steering errors that lead to a decrease in user experience.

[0010] In one possible implementation, determining whether the sound source position is credible by using the camera includes:

[0011] Determine whether the camera is in a credible interval, where the credible interval is used to represent an angle interval within which the device can successfully detect the user after turning to the sound source position.

[0012] In one possible implementation, the credible interval is determined by statistically analyzing detection results, where the detection results are the number of times the user is successfully detected after turning within a corresponding angle interval.

[0013] In this application, the confidence interval is dynamically updated through the statistics of the detection results, which can continuously improve the accuracy of locating the sound source position.

[0014] In one possible implementation, the method further includes: if it is determined that the sound source position is unreliable, redirecting to a position commonly used by the user.

[0015] In this application, when the location of the sound source is unreliable, turning to the user's frequently used location can increase the success rate of detecting the user after turning and improve the user experience.

[0016] In one possible implementation, the user's frequently used location is an angle at which the user frequently appears; or, the user's frequently used location is an angle range at which the user frequently appears.

[0017] In one possible implementation, after turning to the sound source location, the method further includes: if it is detected that there is no one at the sound source location, using the camera to turn in a vertical direction.

[0018] In this application, through vertical detection, missed detection due to user's own reasons (for example, short height) can be avoided.

[0019] In one possible implementation, after turning to the sound source location, the method further includes: if it is detected that there is no one at the sound source location, determining whether there was anyone before the wake-up instruction was detected.

[0020] In one possible implementation manner, the method further includes: if it is determined that there is a person before the wake-up instruction is detected, turning horizontally to the position of the user detected before the wake-up instruction is detected.

[0021] In one possible implementation, the method further includes: if it is determined that there is no one before the wake-up instruction is detected, turning to the user's frequently used location.

[0022] A second aspect provides a sound source localization device, comprising one or more functional modules, wherein the one or more functional modules are used to implement the sound source localization method according to the first aspect.

[0023] A third aspect provides a device, comprising: a processor and a memory, wherein the memory is used to store a program; and the processor is used to run the program to implement the sound source localization method as described in the first aspect.

[0024] A fourth aspect provides a readable storage medium, which stores a program. When the program is run on a device, the device implements the sound source localization method as described in the first aspect.

[0025] The fifth aspect provides a program, which, when executed on a device, enables the device to execute the sound source localization method as described in the first aspect.

[0026] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged with the processor, or may be stored in whole or in part on a memory not packaged with the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 is a flow chart of an embodiment of a sound source localization method provided by the present application;

[0028] FIG2 is a schematic structural diagram of a sound source localization device provided in an embodiment of the present application;

[0029] FIG3 is a schematic diagram of the structure of the device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In the embodiments of this application, unless otherwise specified, the character " / " indicates that the associated objects are in an "or" relationship. For example, A / B can represent A or B. "And / or" describes the relationship between the associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exists simultaneously, or B exists alone.

[0031] It should be pointed out that the words "first", "second", etc. involved in the embodiments of this application are only used for distinguishing description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor can they be understood as indicating or implying order.

[0032] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. In addition, "at least one of the following" or similar expressions refers to any combination of these items, which may include any combination of single items or plural items. For example, at least one of A, B, or C can represent: A, B, C, A and B, A and C, B and C, or A, B and C. Among them, each of A, B, and C can be an element itself, or a set containing one or more elements.

[0033] In the embodiments of this application, the terms "exemplary," "in some embodiments," and "in another embodiment" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" in this application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner.

[0034] In the embodiments of this application, the terms "of," "corresponding," and "relevant" may sometimes be used interchangeably. It should be noted that, when the distinction between them is not emphasized, the meanings they convey are the same. In the embodiments of this application, the terms "communication" and "transmission" may sometimes be used interchangeably. It should be noted that, when the distinction between them is not emphasized, the meanings they convey are the same. For example, "transmission" may include "sending" and / or "receiving," and may be either a noun or a verb.

[0035] In the embodiments of this application, "equal to" can be used in conjunction with "greater than" and is applicable to the technical solution adopted when "greater than" is used, and can also be used in conjunction with "less than" and is applicable to the technical solution adopted when "less than" is used. It should be noted that when "equal to" is used in conjunction with "greater than", it cannot be used in conjunction with "less than"; and when "equal to" is used in conjunction with "less than", it cannot be used in conjunction with "greater than".

[0036] After a device (e.g., a robot) receives a wake-up command from the user, if there are obstacles (e.g., walls) around it, the device may mislocalize the sound source due to sound reflection. For example, if a robot is placed against a wall and the user wakes it up, the robot will face the user directly if it correctly locates the sound source. However, due to sound reflection, the robot may mislocalize the sound source, causing it to face the user with its back, reducing the user experience.

[0037] Based on the above problems, an embodiment of the present application proposes a sound source localization method, which is applied to a device. The device may include a microphone, a camera, and a servo. The microphone is used to collect the user's voice to identify the user's instructions. For example, the instructions may include wake-up instructions or operation instructions. The wake-up instructions are used to wake up the device, and the operation instructions are used to control the awakened device. The camera is used to perform face detection to identify whether there is a user. The servo is used to steer the device so that the camera can face the user for face tracking.

[0038] The sound source localization method provided in the embodiment of the present application is now described with reference to FIG1 .

[0039] FIG1 is a flow chart of an embodiment of a sound source localization method provided by the present application, which specifically includes the following steps:

[0040] In step 101 , in response to detecting a user's wake-up command, the device determines through a camera whether the location of the sound source is credible.

[0041] Specifically, the sound source position is the position where the user makes a sound. The sound source position can be calculated based on an existing sound source localization algorithm. The specific algorithm is not limited here and will not be described in detail in the embodiments of the present application.

[0042] The device can monitor the user's voice to identify the user's instructions. In response to detecting the user's wake-up instruction, the device can determine whether the location of the sound source is credible.

[0043] It is understood that the device can be rotated via a servo, causing the camera on the device to rotate accordingly. When the camera is in different angle ranges, the device's accuracy in locating the sound source varies. In other words, the device's camera can correspond to different confidence levels in the sound source location at different angle ranges. Based on this confidence level, the reliability of the sound source location identified by the device in the current angle range can be determined.

[0044] Exemplarily, the device can pre-divide the rotatable angle into multiple angle intervals. For example, if the rotatable angle is 360 degrees and the range of each angle interval is 20 degrees, it can be divided into 18 angle intervals, for example, 0 to 20 degrees, 20 to 40 degrees, ..., 340 to 360 degrees. It is understood that the above values ​​are only exemplary and do not constitute a limitation of the embodiments of the present application. In some embodiments, the angle intervals can also be divided into other degrees to obtain multiple angle intervals; or the range of the angle interval can be expressed in other ways, for example, -180 to -160 degrees, -160 to -140 degrees, ..., 160-180 degrees.

[0045] Next, the detection results within each angle interval can be counted to determine whether the sound source location is reliable. The detection result is the number of times the user is successfully detected after turning within the corresponding angle interval. The detection result can also be understood as the detection result of each detection process, and the detection result can include successful detection or failed detection. A successful detection can be used to indicate that the device successfully detected the user after turning, and a failed detection can be used to indicate that the device did not detect the user after turning.

[0046] For any angle interval, by counting the detection results, if the number of successful detections within the angle interval is greater than or equal to the first threshold, the angle interval can be considered a credible interval. The credible interval is used to represent the angle interval in which the device can successfully detect the user when turning to the sound source. Or,

[0047] For any angle interval, by counting the detection results, if the number of failed detections within the angle interval is greater than or equal to the second threshold, the angle interval can be considered as an unreliable interval. The unreliable interval is used to represent the angle interval in which the device turns to the sound source position but fails to successfully detect the user.

[0048] It is understood that the above method of determining whether the number of times is a credible interval is only an exemplary description and does not constitute a limitation of the embodiments of the present application. In some embodiments, other methods can also be used, for example, the number of successful recognitions or the proportion of the number of failed recognitions to the total number of times can be counted.

[0049] When the device determines that it is in the credible interval, it can be considered that the calculated sound source position is accurate, that is, the sound source position is credible, and step 104 can be executed. Alternatively,

[0050] When the device determines that it is in the untrustworthy interval, it can be considered that the calculated sound source position is inaccurate, that is, the sound source position is untrustworthy, and step 102 can be executed.

[0051] In some optional embodiments, this step 102 is an optional step, that is, when the device is determined to be in an untrustworthy interval, it turns horizontally to the user's usual position.

[0052] Step 102: The device detects whether there is anyone at the current location.

[0053] Specifically, if the device determines that the location of the sound source is unreliable, it can detect through the camera at the current location to determine whether there is anyone at the current location.

[0054] If it is determined that there is someone at the current location, step 112 may be executed. Alternatively,

[0055] If it is determined that there is no one at the current location, step 103 may be executed.

[0056] Step 103: The device determines whether there is a frequently used location of the user.

[0057] Specifically, the frequently used location can be determined by the user's location. The user's location can be the user's location each time the user uses the device. That is, each time the user wakes up the device, the device can track the user through the camera to obtain the user's location. The user's location can be represented by the deflection angle of the device.

[0058] For example, when the device is tracking a user, it can calculate the user's location within a first preset duration and according to a preset period. For example, if the first preset duration is 5 minutes and the preset period is 1 second, the device can record the user's location every 1 second for 5 minutes and can generate statistics based on the user's location.

[0059] After obtaining the statistical results, the user's frequently used locations can be determined based on the statistical results. A user's frequently used locations can be considered locations where the user is likely to appear. The statistical results can be multiple locations of the user obtained by the device during user tracking. The user's location is the user's deflection angle relative to the device's original angle.

[0060] For example, when the device is initially placed, the original angle of the device can be set to 0 degrees, or to other angles, which are not specifically limited in the embodiments of the present application. When the device is rotated a certain angle based on the initial angle and finds the user, the rotation angle relative to the initial angle can be used as the deflection angle.

[0061] It is understandable that the device can also be turned at any angle to find the user. In this case, the rotation angle relative to the initial angle is also used as the deflection angle.

[0062] In some optional embodiments, since the device usually detects the face of the user in the process of tracking the user to determine the user's position. However, the result of the face detection will be disturbed. In order to avoid the rotation of the device being too sensitive, the deflection angle of the device and the deflection angle of the user's face can be used as the user's position to reduce the deviation caused by the steering jitter of the device. Among them, the deflection angle of the user's face is the deflection angle of the user's face relative to the direction of the device. The deflection angle of the user's face can be obtained by performing face recognition on the image captured by the user by the device, or it can be obtained by other means. The embodiments of the present application do not specifically limit this.

[0063] In some optional embodiments, in the process of the device tracking the user, in order to reduce the impact of face jumps, in addition to the above-mentioned method of representing the user's position by angle, the user's position can also be represented by angle intervals.

[0064] The method of determining the user's frequently used locations based on the statistical results may include the following two methods:

[0065] Method 1: Count the frequency of users appearing at any angle.

[0066] After the device obtains the positions of users at different angles within the first preset time period, it can count the frequencies of users appearing at corresponding angles.

[0067] If the frequency of the user appearing at any angle is greater than or equal to the third threshold, the angle corresponding to the angle greater than or equal to the third threshold can be regarded as the user's common location. Or,

[0068] If the frequency of the user appearing at any angle is less than a third threshold, the angle corresponding to the angle less than the third threshold may not be considered as the user's frequently used position.

[0069] Method 2: Count the frequency of users appearing in any angle interval.

[0070] After the device obtains the positions of users at different angles within the first preset time period, it can count the frequencies of users appearing within the corresponding angle intervals.

[0071] It is understood that since the camera of the device has a preset field of view, for example, the preset field of view can be 60 degrees, or other angles, and the embodiments of the present application are not particularly limited to this. The field of view of the camera can cover the above angle range. Taking the preset field of view of 60 degrees as an example, the frequency of users appearing in the angle range less than or equal to 60 degrees can be counted.

[0072] If the frequency of the user appearing in any angle interval is greater than or equal to the fourth threshold, the angle interval corresponding to the fourth threshold can be regarded as the user's frequently used location.

[0073] If the frequency of the user appearing in any angle interval is less than a fourth threshold, the angle interval corresponding to the angle interval less than the fourth threshold may not be used as the user's frequently used location.

[0074] It can be understood that compared with method one, method two does not require multiple rotations. The electron only needs one rotation to detect the angles of multiple high-frequency users within the field of view, which can improve detection efficiency and enhance the user experience.

[0075] If the device determines that the user has a common location, step 105 is executed.

[0076] In some optional embodiments, after the device determines that it has the user's frequently used locations, it may also count a number of failures. This number of failures indicates the number of times the device failed to find the user after redirecting to the user's frequently used locations within a second preset time period. If the number of failures is greater than or equal to a fifth threshold, step 104 may be executed; alternatively, if the number of historical failures is less than the fifth threshold, redirection to the user's frequently used locations may be performed. This approach can avoid frequent failed redirections, thereby improving the user experience.

[0077] If the device determines that it does not have the user's usual location, step 104 may be executed.

[0078] Step 104: The device turns to the location of the sound source.

[0079] Specifically, the steering gear can be used to horizontally turn to the sound source position so that the camera can capture the user at the sound source position to achieve tracking of the user.

[0080] When the device turns to the location of the sound source, step 106 may be executed.

[0081] Step 105: The device turns to the user's usual location.

[0082] Specifically, the device may be horizontally steered to the user's usual position through the servo. After the device is horizontally steered to the user's usual position, step 106 may be executed.

[0083] Step 106: The device detects whether there is anyone at the location after the turn.

[0084] Specifically, the position after steering in step 106 may be the sound source position or the user's usual position.

[0085] When the device turns to the user's usual location, the camera can be used to detect whether there is anyone at the user's usual location, so as to determine whether the camera is facing the user after the device turns to the user's usual location. Or,

[0086] When the device turns to the sound source, the camera can be used to detect whether there is a person at the sound source, so as to determine whether the camera is facing the user after the device turns to the sound source.

[0087] If it is determined that there is someone at the location after turning, execute step 112. Or,

[0088] If it is determined that there is no one at the location after turning, step 107 is executed.

[0089] In some optional embodiments, as described above, the device's camera typically rotates horizontally and has a limited vertical field of view (FOV), for example, typically 40 degrees. Different users have different heights, and the device's camera may not be able to detect shorter users. Therefore, if the device's camera fails to detect a user after rotating horizontally, the camera can be rotated vertically to detect shorter users. If no user is detected through vertical detection, step 107 is executed; alternatively, if a user is detected through vertical detection, the detected user is tracked.

[0090] In some optional embodiments, to avoid errors caused by short detection times, a third preset time period may be set. If no person is detected within the third preset time period, step 107 is executed. Alternatively, if a person is detected within the third preset time period, the detected user is tracked.

[0091] In step 107 , the device determines whether there is a person before detecting the wake-up command.

[0092] Specifically, before the device is awakened, the camera of the device may be in a normally-on state so as to detect surrounding users.

[0093] It is understood that when the device detects a person through the camera, the position of the user detected can be stored. If the user is detected multiple times, the position of the user detected last time before the device wakes up can be stored.

[0094] In some optional embodiments, the user's location detected at any time before the device is awakened may also be stored, and this embodiment of the present application does not specifically limit this.

[0095] If it is determined that there is someone before the wake-up command is detected, step 109 is executed. Or,

[0096] If it is determined that no one was present before the wake-up command was detected, step 108 is executed.

[0097] Step 108: The device determines whether there is a frequently used location of the user.

[0098] Specifically, the method of determining whether there is a user's frequently used location may refer to the relevant description in the above embodiment, which will not be repeated here.

[0099] If it is determined that there is no user's frequently used location, step 109 is executed. Or,

[0100] If it is determined that the user's frequently used location exists, step 110 is executed.

[0101] Step 109 : The device turns to the user's location detected before the wake-up command is detected.

[0102] Specifically, for turning to the position of the user detected before the wake-up instruction is detected, reference may be made to the relevant description in the above embodiment, which will not be repeated here.

[0103] When the device is turned horizontally to the position of the user detected before the wake-up instruction is detected, step 112 is executed.

[0104] Step 110: The device is turned to the user's usual location.

[0105] The user's frequently used locations may be specifically described in the above embodiments and will not be described in detail here.

[0106] In step 111, the device determines whether there is anyone at the location after turning.

[0107] Specifically, the position after turning in step 111 may include a common position of the user or a position of the user detected before the wake-up instruction is detected.

[0108] When the device turns to the user's usual location or the user's location detected before the wake-up command is detected, it can be determined whether there is anyone at the location after the device turns.

[0109] If it is determined that there is someone at the location after turning, execute step 112. Or,

[0110] If it is determined that there is no one at the location after the turn, step 113 is executed.

[0111] Step 112: The device tracks the detected user.

[0112] It is understandable that when the device successfully tracks the detected user, it indicates that the wake-up task is successful, and the device can continue to remain in the wake-up state and wait for the user's instructions.

[0113] Step 113: The device ends the wake-up task.

[0114] For example, the device may re-enter the non-awakened state and wait to be awakened again.

[0115] In some embodiments, the device may also remain in the awake state and wait for user instructions, which is not specifically limited in the embodiments of the present application.

[0116] FIG2 is a schematic structural diagram of an embodiment of a sound source localization device of the present application. As shown in FIG2 , the above-mentioned sound source localization device 20 is applied to a device, and the device includes a camera. The sound source localization device 20 may include: a judgment module 21 and a steering module 22; wherein,

[0117] a determination module 21 for determining, in response to detecting a user's wake-up command, whether a sound source position is credible through the camera, the sound source position being a position of the user's voice calculated by the device;

[0118] The steering module 22 is configured to, if it is determined that the sound source position is credible, horizontally steer to the sound source position.

[0119] In one possible implementation, the judgment module 21 is further configured to determine whether the camera is within a credible interval, where the credible interval represents an angle interval within which the device can successfully detect the user after turning to the location of the sound source.

[0120] In one possible implementation, the credible interval is determined by statistically analyzing detection results, where the detection results are the number of times the user is successfully detected after turning within a corresponding angle interval.

[0121] In one possible implementation, the steering module 22 is further configured to, if it is determined that the sound source position is unreliable, steer the camera to a position commonly used by the user.

[0122] In one possible implementation, the user's frequently used position is an angle at which the user appears frequently; or,

[0123] The user's frequently used position is an angle range where the user appears frequently.

[0124] In one possible implementation, the steering module 22 is further configured to use the camera to steer in a vertical direction if it is detected that there is no one at the sound source location.

[0125] In one possible implementation, the judgment module 21 is further configured to determine whether there was a person before the wake-up instruction was detected if no person was detected at the sound source location.

[0126] In one possible implementation, the steering module 22 is further configured to, if it is determined that there was a person before the wake-up instruction was detected, horizontally turn to the position of the user detected before the wake-up instruction was detected.

[0127] In one possible implementation, the steering module 22 is further configured to, if it determines that no one was present before the wake-up command was detected, steer to a location commonly used by the user. The sound source localization device 20 provided in the embodiment of FIG2 can be used to implement the technical solution of the method embodiment of this application. The implementation principles and technical effects thereof can be further referenced to the relevant description of the method embodiment.

[0128] It should be understood that the division of the various modules of the sound source localization device 20 shown in FIG. 2 is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a single physical entity, or physically separated. Furthermore, these modules can be implemented entirely in the form of software invoked through processing elements, or entirely in the form of hardware. Alternatively, some modules can be implemented in the form of software invoked through processing elements, while others can be implemented in the form of hardware. For example, the detection module can be a separate processing element, or it can be integrated into a chip in the device. The implementation of other modules is similar. Furthermore, these modules can be fully or partially integrated together, or implemented independently. During implementation, each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element, or by software instructions.

[0129] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, these modules may be integrated together to implement a system-on-a-chip (SOC).

[0130] The following further describes an exemplary device provided in the embodiments of the present application in conjunction with Figure 3. Figure 3 shows a schematic structural diagram of a device 300.

[0131] The above-mentioned device 300 may include: at least one processor; and at least one memory communicatively connected to the above-mentioned processor, wherein: the above-mentioned memory stores program instructions that can be executed by the above-mentioned processor, and the processor calls the above-mentioned program instructions to execute the method provided in the embodiment shown in this application.

[0132] Figure 3 shows a block diagram of an exemplary device 300 suitable for implementing the embodiments of the present application. The device 300 shown in Figure 3 is only an example and should not limit the functions and scope of use of the embodiments of the present application.

[0133] As shown in Figure 3, device 300 is implemented as a device. Components of device 300 may include, but are not limited to, one or more processors 310, memory 320, a communication bus 340 connecting various system components (including memory 320 and processor 310), and a communication interface 330.

[0134] Communication bus 340 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of such architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnection (PCI) bus.

[0135] Device 300 typically includes a variety of system-readable media. These media can be any available media that can be accessed by the device, including volatile and non-volatile media, removable and non-removable media.

[0136] The memory 320 may include system-readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile system storage media. Although not shown in Figure 3, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a Compact Disc Read Only Memory (CD-ROM), a Digital Video Disc Read Only Memory (DVD-ROM) or other optical media) may be provided. In these cases, each drive can be connected to the communication bus 340 via one or more data medium interfaces. The memory 320 may include at least one program product having a set (e.g., at least one) program modules that are configured to perform the functions of the various embodiments of the present application.

[0137] A program / utility having a set (at least one) of program modules may be stored in memory 320. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. The program modules generally perform the functions and / or methods of the embodiments described herein.

[0138] Device 300 can also communicate with one or more external devices (e.g., a keyboard, pointing device, display, etc.), one or more devices that enable a user to interact with the device, and / or any device that enables the device to communicate with one or more other devices (e.g., a network card, a modem, etc.). Such communication can occur via communication interface 330. Furthermore, device 300 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter (not shown in FIG. 3 ). The network adapter can communicate with other modules of the device via communication bus 340. It should be understood that, although not shown in FIG. 3 , other hardware and / or software modules can be used in conjunction with device 300, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0139] The processor 310 executes various functional applications and data processing by running the programs stored in the memory 320, such as implementing the method provided in the embodiment of the present application.

[0140] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative illustration and does not constitute a structural limitation on the device 300. In other embodiments of the present application, the device 300 may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0141] In the above embodiments, the processor involved may include, for example, a CPU, a DSP, a microcontroller, or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and an image signal processor (hereinafter referred to as: ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASICs, or one or more integrated circuits for controlling the execution of the program of the technical solution of this application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0142] An embodiment of the present application also provides a readable storage medium, which stores a computer program. When the computer program is run on a device, the device executes the method provided by the embodiment shown in the present application.

[0143] The embodiments of the present application also provide a program, which, when executed on a device, enables the device to execute the method provided in the embodiments shown in the present application.

[0144] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single or multiple.

[0145] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, software, and a combination of electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0146] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0147] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), magnetic disk or optical disk, and other media that can store program code.

[0148] The above description is merely a specific embodiment of the present application. Any person skilled in the art may easily conceive of variations or substitutions within the technical scope disclosed in this application, and such variations or substitutions shall be within the scope of protection of this application. The scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. A sound source localization method, characterized in that: Applied to a device, the device includes a camera, and the method includes: In response to detecting a wake-up command from a user, determining, by the camera, whether a sound source position is credible, the sound source position being a position of a user's voice calculated by the device; If the sound source position is determined to be credible, turn horizontally to the sound source position.

2. The method according to claim 1, characterized in that The determining whether the sound source position is credible by the camera includes: Determine whether the camera is in a credible interval, where the credible interval is used to represent an angle interval in which the device can successfully detect the user after turning to the sound source position.

3. The method according to claim 2, characterized in that The credible interval is determined by statistically analyzing detection results, where the detection results are the number of times a user is successfully detected after turning within a corresponding angle interval.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: If the location of the sound source is determined to be unreliable, redirect to the user's usual location.

5. The method according to claim 4, characterized in that The user's frequently used position is the angle at which the user appears most frequently; or, The user's frequently used position is an angle interval where the user appears frequently.

6. The method according to any one of claims 1 to 5, characterized in that: After the turning to the sound source position, the method further comprises: If it is detected that there is no one at the location of the sound source, the camera is used to turn in the vertical direction.

7. The method according to any one of claims 1 to 6, characterized in that: After the turning to the sound source position, the method further comprises: If it is detected that there is no person at the sound source location, it is determined whether there was a person before the wake-up instruction was detected.

8. The method according to claim 7, characterized in that The method further comprises: If it is determined that there is a person before the wake-up command is detected, turn horizontally to the position of the user detected before the wake-up command is detected.

9. The method according to claim 7, characterized in that: The method further comprises: If it is determined that there is no one before the wake-up command is detected, turn to the user's usual location.

10. A device, characterized in that: include: A processor and a memory, wherein the memory is used to store a program; and the processor is used to run the program to implement the sound source localization method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Natural human-computer voice interaction method and system

    CN107230476A

  • Sound source positioning method of audio equipment and audio equipment

    CN110691196A

  • Sound source localization method, device, intelligent speaker and storage medium

    CN110794368A

  • Tracking camera shooting method and device and terminal equipment

    CN110830708A

  • Sweeper control method and device

    CN110881909A