Sound source positioning method and device and storage medium

Abstract

The invention provides a sound source positioning method and device and a storage medium, and the method comprises the steps: responding to a detected wake-up instruction of a user, judging whether the position of a sound source is credible through a camera, and the position of the sound source is the sound production position of the user calculated by the device; and if the sound source position is determined to be credible, horizontally turning to the sound source position. The method provided by the invention is helpful for improving the accuracy of sound source positioning.

Description

Technical Field

[0001] The present application relates to the field of intelligent robots, and in particular to a sound source localization method, device and storage medium. Background Art

[0002] After receiving the user's wake-up command, traditional devices (such as robots) may mis-locate the sound source due to sound reflection if there are obstacles around the device (such as walls). Taking a robot as an example, when the user wakes up the robot placed against the wall, if the robot locates the sound source correctly, the robot will face the user directly; however, due to sound reflection, the robot may mis-locate the sound source, causing the robot to face the user with its back, reducing the user experience. Summary of the invention

[0003] The present application provides a sound source localization method, device and storage medium, which are helpful to improve the accuracy of sound source localization.

[0004] A first aspect provides a sound source localization method, which is applied to a device, wherein the device includes a camera, including:

[0005] In response to detecting a wake-up command from a user, determining, by the camera, whether a sound source position is credible, the sound source position being a position of a user's voice calculated by the device;

[0006] If the sound source position is determined to be reliable, turn horizontally to the sound source position.

[0007] In the present application, by judging the credibility of the sound source position, steering to the sound source position only when the sound source position is credible can improve the accuracy of sound source positioning and avoid steering errors that reduce user experience.

[0008] In one possible implementation, determining whether the sound source position is credible by using the camera includes:

[0009] Determine whether the camera is in a credible interval, where the credible interval is used to represent an angle interval in which the device can successfully detect the user after turning to the sound source position.

[0010] In one possible implementation, the credible interval is determined by statistically analyzing detection results, where the detection results are the number of times the user is successfully detected after turning within a corresponding angle interval.

[0011] In the present application, the credible interval is dynamically updated through the statistics of the detection results, so that the accuracy of locating the sound source position can be continuously improved.

[0012] In one possible implementation, the method further includes: if it is determined that the sound source position is unreliable, turning to a position commonly used by the user.

[0013] In the present application, when the location of the sound source is unreliable, turning to the user's frequently used location can increase the success rate of detecting the user after turning, thereby improving the user experience.

[0014] In one possible implementation, the user's frequently used position is an angle at which the user appears frequently; or, the user's frequently used position is an angle range at which the user appears frequently.

[0015] In one possible implementation, after turning to the sound source position, the method further includes: if it is detected that there is no one at the sound source position, using the camera to turn in a vertical direction.

[0016] In the present application, through detection in the vertical direction, missed detection due to user's own reasons (for example, short height) can be avoided.

[0017] In one possible implementation, after turning to the sound source location, the method further includes: if it is detected that there is no one at the sound source location, determining whether there was anyone before the wake-up instruction was detected.

[0018] In one possible implementation manner, the method further includes: if it is determined that there is a person before the wake-up instruction is detected, turning horizontally to the position of the user detected before the wake-up instruction is detected.

[0019] In one possible implementation, the method further includes: if it is determined that there is no one before the wake-up instruction is detected, turning to a user's usual location.

[0020] A second aspect provides a sound source localization device, comprising one or more functional modules, wherein the one or more functional modules are used to implement the sound source localization method as described in the first aspect.

[0021] A third aspect provides a device, comprising: a processor and a memory, wherein the memory is used to store a program; and the processor is used to run the program to implement the sound source localization method as described in the first aspect.

[0022] A fourth aspect provides a readable storage medium, in which a program is stored. When the program is executed on a device, the device implements the sound source localization method as described in the first aspect.

[0023] The fifth aspect provides a program, which, when executed on a device, enables the device to execute the sound source localization method as described in the first aspect.

[0024] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of a flow chart of an embodiment of a sound source localization method provided in the present application;

[0026] Figure 2 A schematic diagram of the structure of a sound source localization device provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of the structure of the device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0028] In the embodiments of the present application, unless otherwise specified, the character " / " indicates that the objects before and after the association are in an or relationship. For example, A / B can represent A or B. "And / or" describes the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone.

[0029] It should be pointed out that the words "first", "second", etc. involved in the embodiments of the present application are only used to distinguish the description purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor can they be understood as indicating or implying order.

[0030] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. In addition, "at least one of the following" or similar expressions refers to any combination of these items, which may include any combination of single items or plural items. For example, at least one of A, B, or C may represent: A, B, C, A and B, A and C, B and C, or A, B and C. Among them, each of A, B, and C may be an element itself, or a set containing one or more elements.

[0031] In the embodiments of the present application, "exemplary", "in some embodiments", "in another embodiment", etc. are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0032] In the embodiments of the present application, "of", "corresponding, relevant" and "corresponding" may sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are consistent. In the embodiments of the present application, communication and transmission may sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are consistent. For example, transmission may include sending and / or receiving, and may be a noun or a verb.

[0033] The equal to involved in the embodiments of the present application can be used in conjunction with greater than, and is applicable to the technical solution adopted when greater than, and can also be used in conjunction with less than, and is applicable to the technical solution adopted when less than. It should be noted that when equal to is used in conjunction with greater than, it cannot be used in conjunction with less than; when equal to is used in conjunction with less than, it cannot be used in conjunction with greater than.

[0034] After a device (e.g., a robot) receives a wake-up command from a user, if there are obstacles around the device (e.g., walls), the device may mis-locate the sound source due to sound reflection. For example, when a user wakes up a robot placed against a wall, if the robot locates the sound source correctly, the robot will face the user directly; however, due to sound reflection, the robot may mis-locate the sound source, causing the robot to face the user with its back, which reduces the user experience.

[0035] Based on the above problems, an embodiment of the present application proposes a sound source localization method, which is applied to a device. The device may include a microphone, a camera, and a servo. Among them, the microphone is used to collect the user's voice to identify the user's instructions. For example, the instructions may include wake-up instructions or operation instructions, etc. The wake-up instructions are used to wake up the device, and the operation instructions are used to control the awakened device. The camera is used to perform face detection to identify whether there is a user. The servo is used to turn the device so that the camera can face the user for face tracking.

[0036] Now combined Figure 1 The sound source localization method provided in the embodiment of the present application is described.

[0037] like Figure 1 The figure is a flow chart of an embodiment of the sound source localization method provided by the present application, which specifically includes the following steps:

[0038] Step 101: In response to detecting a wake-up command from a user, the device determines through a camera whether the location of a sound source is credible.

[0039] Specifically, the sound source position is the position where the user makes a sound, and the sound source position can be calculated according to an existing sound source localization algorithm. The specific algorithm is not limited here and will not be described in detail in the embodiments of the present application.

[0040] The device can monitor the user's voice to identify the user's command. In response to detecting the user's wake-up command, the device can determine whether the sound source location is credible.

[0041] It is understandable that the device can be rotated by the steering gear so that the camera on the device rotates accordingly. When the camera is in different angle intervals, the accuracy of the device's positioning of the sound source is different. In other words, the device's camera can correspond to different confidence levels of the sound source position when it is in different angle intervals. According to its confidence level, it can be determined whether the sound source position recognized by the device in the current angle interval is credible.

[0042] Exemplarily, the device can pre-divide the rotatable angle into multiple angle intervals. For example, if the rotatable angle is 360 degrees and the range of each angle interval is 20 degrees, it can be divided into 18 angle intervals, for example, 0 to 20 degrees, 20 to 40 degrees, ..., 340 to 360 degrees. It can be understood that the above values ​​are only exemplary and do not constitute a limitation on the embodiments of the present application. In some embodiments, it can also be divided into angle intervals of other degrees to obtain multiple angle intervals; or, the range of the angle interval can also be expressed in other ways, for example, -180 to -160 degrees, -160 to -140 degrees, ..., 160-180 degrees.

[0043] Next, the detection results in each angle interval can be counted to determine whether the sound source position is credible. The detection result is the number of times the user is successfully detected after turning in a corresponding angle interval. The detection result can also be understood as the detection result of each detection process. The detection result can include successful detection or failed detection. Successful detection can be used to indicate that the device successfully detects the user after turning, and failed detection is used to indicate that the device does not detect the user after turning.

[0044] For any angle interval, by counting the detection results, if the number of successful detections within the angle interval is greater than or equal to the first threshold, the angle interval can be considered a credible interval, which is used to characterize the angle interval in which the device can successfully detect the user when turning to the sound source. Or,

[0045] For any angle interval, by counting the detection results, if the number of failed detections within the angle interval is greater than or equal to the second threshold, the angle interval can be considered as an unreliable interval. The unreliable interval is used to characterize the angle interval in which the device turns to the sound source position but fails to successfully detect the user.

[0046] It is understandable that the above method of judging whether the number of times is a credible interval is only an exemplary description and does not constitute a limitation on the embodiments of the present application. In some embodiments, other methods can also be used, for example, the number of successful identifications or the proportion of the number of failed identifications to the total number of times can be counted.

[0047] When the device determines that it is in the credible interval, it can be considered that the calculated sound source position is accurate, that is, the sound source position is credible, and step 104 can be executed. Or,

[0048] When the device is determined to be in the untrustworthy interval, it can be considered that the calculated sound source position is inaccurate, that is, the sound source position is untrustworthy, and step 102 can be executed.

[0049] In some optional embodiments, this step 102 is an optional step, that is, when the device is determined to be in an untrustworthy interval, it turns horizontally to the user's usual position.

[0050] Step 102: The device detects whether there is anyone at the current location.

[0051] Specifically, if the device determines that the location of the sound source is unreliable, it can detect through the camera at the current location to determine whether there is anyone at the current location.

[0052] If it is determined that there is someone at the current location, step 112 may be executed. Alternatively,

[0053] If it is determined that there is no one at the current location, step 103 may be executed.

[0054] Step 103: The device determines whether there is a frequently used location of the user.

[0055] Specifically, the common location can be determined by the user's location. The user's location can be the user's location each time when the user uses the device, that is, each time the user wakes up the device, the device can track the user through a camera to obtain the user's location, and the user's location can be represented by the deflection angle of the device.

[0056] Exemplarily, when the device is tracking a user, the user's location can be calculated within a first preset time period and according to a preset period. For example, if the first preset time period is 5 minutes and the preset period is 1 second, the device can record the user's location every 1 second within 5 minutes and can perform statistics based on the user's location.

[0057] After obtaining the statistical results, the user's frequently used location can be determined based on the statistical results. The frequently used location of the user can be considered as a location where the user appears with a high probability. The statistical results can be the multiple locations of the user obtained by the device during the process of tracking the user, and the user's location is the deflection angle of the user relative to the original angle of the device.

[0058] For example, when the device is initially placed, the original angle of the device can be set to 0 degrees, or to other angles, which are not specifically limited in the embodiments of the present application. When the device is rotated by a certain angle based on the initial angle and finds the user, the rotation angle relative to the initial angle can be used as the deflection angle.

[0059] It is understandable that the device can also be turned at any angle to find the user. In this case, the rotation angle relative to the initial angle is also used as the deflection angle.

[0060] In some optional embodiments, since the device usually detects the face of the user in the process of tracking the user, in order to determine the user's position. However, the result of face detection will be disturbed. In order to avoid the rotation of the device being too sensitive, the deflection angle of the device and the deflection angle of the user's face can be used as the user's position, so as to reduce the deviation caused by the steering jitter of the device. Among them, the deflection angle of the user's face is the deflection angle of the user's face relative to the direction of the device. The deflection angle of the user's face can be obtained by performing face recognition on the image taken by the user by the device, or it can be obtained by other methods. The embodiments of the present application do not specifically limit this.

[0061] In some optional embodiments, during the process of the device tracking the user, in order to reduce the impact of face jumps, in addition to the above-mentioned method of representing the user's position by angle, the user's position may also be represented by an angle interval.

[0062] The method of determining the user's frequently used location based on the statistical results may include the following two methods:

[0063] Method 1: Count the frequency of users appearing at any angle.

[0064] After the device obtains the positions of users at different angles within the first preset time period, the frequency of users appearing at corresponding angles can be counted.

[0065] If the frequency of the user appearing at any angle is greater than or equal to the third threshold, the angle corresponding to the angle greater than or equal to the third threshold can be used as the user's common position.

[0066] If the frequency of the user appearing at any angle is less than a third threshold, the angle corresponding to the angle less than the third threshold may not be used as the user's common position.

[0067] Method 2: Count the frequency of users appearing in any angle interval.

[0068] After the device obtains the positions of users at different angles within the first preset time period, it can count the frequency of users appearing within the corresponding angle interval.

[0069] It is understandable that, since the camera of the device has a preset field of view, illustratively, the preset field of view may be 60 degrees, or other angles, and the present application embodiment does not specifically limit this. The field of view of the camera may cover the above-mentioned angle interval. Taking the preset field of view of 60 degrees as an example, the frequency of users appearing in the angle interval less than or equal to 60 degrees may be counted.

[0070] If the frequency of the user appearing in any angle interval is greater than or equal to the fourth threshold, the angle interval corresponding to the angle greater than or equal to the fourth threshold can be used as the user's common location.

[0071] If the frequency of the user appearing in any angle interval is less than the fourth threshold, the angle interval corresponding to the angle interval less than the fourth threshold may not be used as the user's frequently used position.

[0072] It can be understood that, compared with method one, method two does not require multiple rotations. The electronics only need one rotation to detect multiple high-frequency user angles within the field of view, which can improve detection efficiency and also enhance the user experience.

[0073] If the device determines that the device has the user's usual location, step 105 is executed.

[0074] In some optional embodiments, after the device determines that it has a common location of the user, it can also count the number of failures, which is used to characterize the number of times the device fails to find the user after turning to the common location of the user within the second preset time period. If the number of failures is greater than or equal to the fifth threshold, step 104 can be performed; or, if the number of historical failures is less than the fifth threshold, the device can turn to the common location of the user. In the above manner, frequent failed turns of the device can be avoided, thereby improving the user experience.

[0075] If the device determines that it does not have the user's usual location, step 104 may be executed.

[0076] Step 104: The device turns to the location of the sound source.

[0077] Specifically, the steering gear can be used to horizontally turn to the sound source position so that the camera can capture the user at the sound source position to achieve tracking of the user.

[0078] When the device turns to the location of the sound source, step 106 may be executed.

[0079] Step 105: The device turns to the user's usual location.

[0080] Specifically, the device may be horizontally steered to the user's usual position through a servo, and when the device is horizontally steered to the user's usual position, step 106 may be executed.

[0081] Step 106, the device detects whether there is anyone at the position after turning.

[0082] Specifically, the position after steering in step 106 may be the sound source position or the user's usual position.

[0083] When the device turns to the user's usual location, the camera can be used to detect whether there is someone at the user's usual location, so as to determine whether the camera is facing the user after the device turns to the user's usual location. Or,

[0084] When the device turns to the location of the sound source, the camera can be used to detect whether there is a person at the location of the sound source, so as to determine whether the camera is facing the user after the device turns to the location of the sound source.

[0085] If it is determined that there is someone at the location after the turn, execute step 112. Or,

[0086] If it is determined that there is no one at the location after the turn, execute step 107.

[0087] In some optional embodiments, as described above, since the camera of the device is usually rotated in the horizontal direction, and the vertical field of view of the camera of the device is limited, for example, the vertical field of view is usually 40 degrees. Different users have different heights, and the camera of the device may not be able to detect shorter users. Therefore, when the camera of the device does not detect the user after rotating in the horizontal direction, the camera can be rotated in the vertical direction so that the shorter user can be detected. If no person is detected through the vertical detection, step 107 is executed; or, if a person is detected through the vertical detection, the detected user is tracked.

[0088] In some optional embodiments, in order to avoid errors caused by short-time detection, a third preset time period may be set. If no person is detected within the third preset time period, step 107 is executed. Alternatively, if a person is detected within the third preset time period, the detected user is tracked.

[0089] Step 107 , the device determines whether there is a person before detecting the wake-up command.

[0090] Specifically, before the device is awakened, the camera of the device may be in a normally-on state so as to detect surrounding users.

[0091] It is understandable that when the device detects a person through the camera, the position of the detected user can be stored. If the user is detected multiple times, the position of the user last detected before the device is awakened can be stored.

[0092] In some optional embodiments, the user's location detected at any time before the device is awakened may also be stored, and this embodiment of the present application does not specifically limit this.

[0093] If it is determined that there is a person before the wake-up command is detected, execute step 109. Or,

[0094] If it is determined that there is no one before the wake-up command is detected, execute step 108.

[0095] Step 108: The device determines whether there is a frequently used location of the user.

[0096] Specifically, the method of determining whether there is a user's frequently used location may refer to the relevant description in the above embodiment, which will not be described in detail here.

[0097] If it is determined that there is no user's usual location, step 109 is executed. Or,

[0098] If it is determined that there is a frequently used location of the user, step 110 is executed.

[0099] Step 109 : The device turns to the position of the user detected before the wake-up command is detected.

[0100] Specifically, reference may be made to the relevant description in the above embodiment for turning to the position of the user detected before the wake-up instruction is detected, which will not be repeated here.

[0101] When the device is turned horizontally to the position of the user detected before the wake-up command is detected, step 112 is executed.

[0102] Step 110: The device turns to the user's usual location.

[0103] The user's frequently used locations may be specifically referred to the relevant descriptions in the above embodiments, which will not be described in detail here.

[0104] Step 111, the device determines whether there is anyone at the position after turning.

[0105] Specifically, the position after the turn in step 111 may include a common position of the user or a position of the user detected before the wake-up instruction is detected.

[0106] When the device turns to the user's usual location or detects the user's location before the wake-up command is detected, it can be determined whether there is someone at the location after the turn.

[0107] If it is determined that there is someone at the position after the turn, execute step 112. Or,

[0108] If it is determined that there is no one at the position after the turn, step 113 is executed.

[0109] Step 112: The device tracks the detected user.

[0110] It is understandable that when the device successfully tracks the detected user, it means that the wake-up task is successful, and the device can continue to remain in the wake-up state and wait for the user's instructions.

[0111] Step 113: The device ends the current wake-up task.

[0112] Exemplarily, the device may re-enter the non-awakened state and wait to be awakened again.

[0113] In some embodiments, the device may also remain in the awake state and wait for user instructions, which is not particularly limited in the embodiments of the present application.

[0114] Figure 2 FIG. 1 is a schematic diagram of a structure of an embodiment of a sound source localization device of the present application. Figure 2 As shown, the above-mentioned sound source localization device 20 is applied to a device, and the device includes a camera. The sound source localization device 20 may include: a judgment module 21 and a steering module 22; wherein,

[0115] A judgment module 21, configured to judge whether the sound source position is credible through the camera in response to detecting a wake-up command of the user, wherein the sound source position is a position of the user's voice calculated by the device;

[0116] The steering module 22 is used to horizontally steer to the sound source position if it is determined that the sound source position is credible.

[0117] In one possible implementation, the judgment module 21 is further used to judge whether the camera is in a credible interval, where the credible interval is used to represent an angle interval in which the device can successfully detect the user after turning to the sound source position.

[0118] In one possible implementation, the credible interval is determined by statistically analyzing detection results, where the detection results are the number of times the user is successfully detected after turning within a corresponding angle interval.

[0119] In one possible implementation, the steering module 22 is further configured to, if it is determined that the sound source position is unreliable, steer to a position frequently used by the user.

[0120] In one possible implementation, the user's frequently used position is an angle at which the user appears frequently; or,

[0121] The user's frequently used position is an angle interval where the user appears frequently.

[0122] In one possible implementation, the steering module 22 is further configured to use the camera to steer in a vertical direction if it is detected that there is no person at the sound source location.

[0123] In one possible implementation, the judgment module 21 is further configured to determine whether there was a person before the wake-up instruction was detected if no person was detected at the sound source location.

[0124] In one possible implementation, the steering module 22 is further configured to, if it is determined that there is a person before the wake-up instruction is detected, horizontally turn to the position of the user detected before the wake-up instruction is detected.

[0125] In one possible implementation, the steering module 22 is further configured to, if it is determined that no one is present before the wake-up command is detected, steering to a location frequently used by the user. Figure 2 The sound source localization device 20 provided in the illustrated embodiment can be used to implement the technical solution of the method embodiment illustrated in the present application. Its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.

[0126] It should be understood that the above Figure 2 The division of the various modules of the sound source localization device 20 shown is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of the device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.

[0127] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0128] Combine the following Figure 3 The exemplary device provided in the embodiments of the present application is further introduced. Figure 3 A schematic structural diagram of the device 300 is shown.

[0129] The above-mentioned device 300 may include: at least one processor; and at least one memory communicatively connected to the above-mentioned processor, wherein: the above-mentioned memory stores program instructions that can be executed by the above-mentioned processor, and the processor calls the above-mentioned program instructions to execute the method provided by the embodiment shown in this application.

[0130] Figure 3 A block diagram of an exemplary device 300 suitable for implementing embodiments of the present application is shown. Figure 3 The device 300 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0131] like Figure 3 As shown, the device 300 is in the form of a device. The components of the device 300 may include, but are not limited to: one or more processors 310, a memory 320, a communication bus 340 connecting different system components (including the memory 320 and the processor 310), and a communication interface 330.

[0132] The communication bus 340 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0133] Device 300 typically includes a variety of system-readable media. These media can be any available media that can be accessed by the device, including volatile and non-volatile media, removable and non-removable media.

[0134] The memory 320 may include system-readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile system storage media. Figure 3 Not shown, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the communication bus 340 via one or more data medium interfaces. The memory 320 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0135] A program / utility having a set (at least one) of program modules may be stored in memory 320, such program modules including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules typically perform the functions and / or methods of the embodiments described herein.

[0136] Device 300 may also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), one or more devices that enable a user to interact with the device, and / or any device that enables the device to communicate with one or more other devices (e.g., network card, modem, etc.). Such communication may be performed via communication interface 330. In addition, device 300 may also communicate with a network adapter ( Figure 3 The network adapter can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the communication bus 340. It should be understood that although Figure 3Not shown, other hardware and / or software modules may be used in conjunction with device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Drives; hereinafter referred to as: RAID) systems, tape drives, and data backup storage systems.

[0137] The processor 310 executes various functional applications and data processing by running the programs stored in the memory 320, such as implementing the method provided in the embodiment of the present application.

[0138] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the device 300. In other embodiments of the present application, the device 300 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0139] In the above embodiments, the processor involved may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and an image signal processor (Image Signal Processing; hereinafter referred to as: ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASIC, or one or more integrated circuits for controlling the execution of the program of the technical solution of the present application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0140] An embodiment of the present application also provides a readable storage medium, which stores a computer program. When the computer program is run on a device, the device executes the method provided by the embodiment shown in the present application.

[0141] The embodiment of the present application also provides a program, which, when executed on a device, enables the device to execute the method provided by the embodiment shown in the present application.

[0142] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. A and B may be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c may be single or plural.

[0143] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0145] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0146] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A sound source localization method, characterized in that: Applied to a device, the device includes a camera, and the method includes: In response to detecting a wake-up command from a user, determining, by the camera, whether a sound source position is credible, the sound source position being a position of a user's voice calculated by the device; If the sound source position is determined to be credible, turn horizontally to the sound source position.

2. The method according to claim 1, characterized in that The determining whether the sound source position is credible by the camera includes: Determine whether the camera is in a credible interval, where the credible interval is used to represent an angle interval in which the device can successfully detect the user after turning to the sound source position.

3. The method according to claim 2, characterized in that The credible interval is determined by statistically analyzing detection results, where the detection results are the number of times a user is successfully detected after turning within a corresponding angle interval.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: If the location of the sound source is determined to be unreliable, redirect to the user's usual location.

5. The method according to claim 4, characterized in that The user's frequently used position is the angle at which the user appears most frequently; or, The user's frequently used position is an angle interval where the user appears frequently.

6. The method according to any one of claims 1 to 5, characterized in that: After the turning to the sound source position, the method further comprises: If it is detected that there is no one at the location of the sound source, the camera is used to turn in the vertical direction.

7. The method according to any one of claims 1 to 6, characterized in that: After the turning to the sound source position, the method further comprises: If it is detected that there is no person at the sound source location, it is determined whether there was a person before the wake-up instruction was detected.

8. The method according to claim 7, characterized in that The method further comprises: If it is determined that there is a person before the wake-up command is detected, turn horizontally to the position of the user detected before the wake-up command is detected.

9. The method according to claim 7, characterized in that: The method further comprises: If it is determined that there is no one before the wake-up command is detected, turn to the user's usual location.

10. A device, characterized in that: include: A processor and a memory, wherein the memory is used to store a program; and the processor is used to run the program to implement the sound source localization method according to any one of claims 1 to 9.

Images