A speaker control method, device, speaker and readable storage medium
By combining the sound source data and image information, the smart speaker accurately recognizes and locks the target user locations that interact with it, solving the problem of camera inaccuracy caused by noise and interference in the prior art, and improving the user experience.
Patent Information
- Application Number
- CN202110852856.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-27
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-07-27
AI Technical Summary
When existing smart speakers rotate and track people, they are susceptible to noise from other sources or interference from non-interactive personnel, resulting in inaccurate camera orientation.
By combining the audio source data collected by the microphone and the images captured by the camera, the target user location interacting with the speaker is determined, and the volume and face posture angles are used to sort and select the target orientation to ensure that the camera is accurately facing the interactive user.
Improves the accuracy of the speaker camera, ensuring that it always faces the target user that interacts with the speaker, and improves the user experience.
Smart Images

Figure CN113627290B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of smart speakers, and more specifically, to a speaker control method, a speaker control device, a speaker, and a readable storage medium. Background Art
[0002] Due to the rapid development of smart homes, people have higher and higher requirements for the quality of life, and smart voice control has gradually penetrated into family life. Domestic and foreign manufacturers have also launched corresponding smart voice products to seize the entrance to smart homes. These smart voice products have facilitated people's lives to a certain extent.
[0003] Existing intelligent voice products, such as smart speakers, already have a rotation function, which can control the camera to automatically face the direction where the person appears based on the direction of the sound and the position of the person in the image taken by the camera.
[0004] In the prior art, there are usually two ways for a smart speaker to control the camera to automatically face the direction where a person appears: the first solution is that the smart speaker uses the sound source data collected by the microphone to calculate the direction of the surrounding sound sources to locate the person; the second solution is that the smart speaker uses the camera to identify the person to achieve rotational tracking of the speaker.
[0005] However, in the first solution, the smart speaker will be disturbed by noise from other sound sources, and in the second solution, the smart speaker will be disturbed by other people who are not communicating with the speaker. In both solutions, the direction in which the camera of the smart speaker is facing is not the direction in which the person interacting with the smart speaker appears, making the direction of the camera of the smart speaker inaccurate. Summary of the invention
[0006] One object of the present disclosure is to provide a new technical solution for allowing a camera of a speaker to accurately face the direction where a target user interacting with the speaker appears.
[0007] According to a first aspect of the present disclosure, a speaker control method is provided, comprising:
[0008] Get the sound source data collected by the speaker's microphone;
[0009] Determining, based on the sound source data, a position to be detected of at least one sound source relative to the sound box;
[0010] Controlling the camera of the speaker to shoot toward the position to be detected, and obtaining a test image corresponding to the position to be detected;
[0011] According to the image to be tested, selecting a target position of a target user interacting with the sound box relative to the sound box from the positions to be detected;
[0012] The camera of the speaker is controlled to face the target direction.
[0013] Optionally, controlling the camera of the speaker to shoot toward the position to be detected to obtain the image to be detected corresponding to the position to be detected includes:
[0014] Traversing at least one of the positions to be detected;
[0015] Control the camera of the speaker to shoot toward the currently traversed position to be detected, and obtain the image to be detected of the currently traversed position to be detected;
[0016] The selecting, according to the image to be tested, a target position of a target user interacting with the sound box relative to the sound box from the positions to be detected comprises:
[0017] Determine the posture angle of the face in the image to be tested at the currently traversed position to be tested;
[0018] Determining whether the currently traversed position to be detected meets a preset condition according to the posture angle of the face in the image to be detected of the currently traversed position to be detected;
[0019] When the currently traversed position to be detected meets the preset condition, the currently traversed position to be detected is used as the target position, and the traversal is stopped.
[0020] Optionally, the attitude angle includes a heading angle and a pitch angle,
[0021] Determining whether the currently traversed position to be detected meets a preset condition according to the posture angle of the face in the image to be detected of the currently traversed position to be detected includes:
[0022] Determine a detection result of the currently traversed position to be detected according to the heading angle and the pitch angle of the face in the image to be detected of the currently traversed position to be detected;
[0023] In the case where the detection result of the currently traversed position to be detected is greater than or equal to a preset result threshold, determining that the currently traversed position to be detected meets the preset condition;
[0024] When the detection result of the currently traversed position to be detected is less than a preset result threshold, it is determined that the currently traversed position to be detected does not meet the preset condition.
[0025] Optionally, the attitude angle includes a heading angle and a pitch angle,
[0026] The step of determining the posture angle of the face in the image to be detected in the currently traversed orientation to be detected comprises:
[0027] According to the image to be tested at the currently traversed position to be tested, a spatial rectangular coordinate system of the face is constructed; a first coordinate axis of the spatial rectangular coordinate system of the face is parallel to the straight line where the eyes are located in the image to be tested at the currently traversed position to be tested; a second coordinate axis of the spatial rectangular coordinate system of the face is parallel to a first plane and perpendicular to the first coordinate axis, the first plane being a plane where at least three specified positions of the face in the image to be tested are located; a third coordinate axis of the spatial rectangular coordinate system of the face is perpendicular to the first coordinate axis and the second coordinate axis;
[0028] Constructing a reference coordinate system; the fourth coordinate axis of the reference coordinate system is parallel to the straight line where the eyes are located when the face of the person is facing the camera; the fifth coordinate axis of the reference coordinate system is perpendicular to the lens of the camera;
[0029] An angle between the first coordinate axis and the fourth coordinate axis is determined as the heading angle; an angle between the third coordinate axis and the fifth coordinate axis is determined as the pitch angle.
[0030] Optionally, the method further includes:
[0031] When the currently traversed position to be detected does not meet the preset condition, continue to traverse the next position to be detected.
[0032] Optionally, the method further includes:
[0033] When none of the to-be-detected orientations meets the preset condition, the to-be-detected orientation with the largest detection result is used as the target orientation.
[0034] Optionally, the method further includes:
[0035] Determining the volume of the at least one sound source according to the sound source data;
[0036] sorting the directions to be detected according to the volume of the at least one sound source;
[0037] The traversing at least one of the positions to be detected comprises:
[0038] At least one of the to-be-detected directions is traversed according to the sorting order of the to-be-detected directions.
[0039] According to a second aspect of the present disclosure, there is provided a speaker control device, comprising:
[0040] A sound source data acquisition module is used to acquire the sound source data collected by the microphone of the speaker;
[0041] A module for determining a position to be detected, used to determine a position to be detected of at least one sound source relative to the sound box according to the sound source data;
[0042] A module for obtaining an image to be tested, used for controlling the camera of the speaker to shoot toward the position to be tested, and obtaining an image to be tested corresponding to the position to be tested;
[0043] a target orientation determination module, configured to select, from the orientations to be detected, a target orientation of a target user interacting with the sound box relative to the sound box according to the image to be detected;
[0044] The camera control module is used to control the camera of the speaker to face the target direction.
[0045] According to a third aspect of the present disclosure, there is provided a speaker, comprising:
[0046] The device as described in the second aspect of the present disclosure; or,
[0047] A processor and a memory, wherein the memory is used to store instructions, and the instructions are used to control the processor to execute the method according to the first aspect of the present disclosure.
[0048] According to a fourth aspect of the present disclosure, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first aspect of the present disclosure is implemented.
[0049] Through the method of this embodiment, the target position of the target user interacting with the speaker relative to the speaker is determined by combining the sound source information and the image to be tested, and the camera of the speaker is controlled to face the target position. This allows the camera of the speaker to accurately face the position where the target user interacting with the speaker appears, thereby improving the user experience.
[0050] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0052] Figure 1 is a block diagram showing an example of a hardware configuration of a speaker that can be used to implement an embodiment of the present disclosure.
[0053] Figure 2 A flow chart of a speaker control method according to an embodiment of the present disclosure is shown.
[0054] Figure 3 A flowchart showing an example of a speaker control method according to an embodiment of the present disclosure is shown.
[0055] Figure 4 A block diagram of a speaker control device according to an embodiment of the present disclosure is shown.
[0056] Figure 5 A block diagram of a speaker according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0057] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.
[0058] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0059] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0060] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0061] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0062] <Hardware Configuration>
[0063] Figure 1 is a block diagram of an example of a hardware configuration of a speaker that can be used to implement an embodiment of the present disclosure.
[0064] The speaker 1000 may include but is not limited to a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. Among them, the processor 1100 may be a central processing unit CPU, a graphics processing unit GPU, a microprocessor MCU, etc., for executing a computer program, and the computer program may be written in an instruction set of an architecture such as x86, Arm, RISC, MIPS, SSE, etc. The memory 1200 may include, for example, a ROM (read-only memory), a RAM (random access memory), a non-volatile memory such as a hard disk, etc. The interface device 1300 may include, for example, a USB interface, a serial interface, a parallel interface, etc. The communication device 1400 may, for example, be capable of wired communication using an optical fiber or cable, or wireless communication, and may specifically include WiFi communication, Bluetooth communication, 2G / 3G / 4G / 5G communication, etc. The display device 1500 may be, for example, a liquid crystal display screen, a touch display screen, etc. The input device 1600 may include, for example, a touch screen, a keyboard, somatosensory input, etc. The speaker 1700 is used to output audio signals. The microphone 1800 is used to collect audio signals.
[0065] As used in the embodiments of the present disclosure, the memory 1200 of the speaker 1000 is used to store a computer program, which is used to control the processor 1100 to operate to implement the method according to the embodiments of the present disclosure. A technician can design the computer program according to the scheme disclosed in the present disclosure. How the computer program controls the processor to operate is well known in the art, so it will not be described in detail here. The speaker 1000 can be installed with an intelligent operating system (such as Windows, Linux, Android, IOS, etc.) and application software.
[0066] It should be understood by those skilled in the art that although Figure 1 , multiple devices of the speaker 1000 are shown, but the speaker 1000 of the embodiment of the present disclosure may only involve some of the devices, for example, only the processor 1100 and the memory 1200, etc.
[0067] Hereinafter, various embodiments and examples according to the present invention are described with reference to the accompanying drawings.
[0068] <Method Example>
[0069] In this embodiment, a method for controlling a speaker is provided. The method is implemented by a speaker. In one example, the speaker may be Figure 1 Speaker 1000 is shown.
[0070] The sound box in this embodiment may have a microphone and a camera, wherein the microphone may be a single microphone or a microphone array in any arrangement form, such as 2 mics in a straight line or 4 mics in a ring.
[0071] according to Figure 2 As shown, the speaker control method of this embodiment may include the following steps S2100 to S2500.
[0072] Step S2100, obtaining sound source data collected by the microphone of the speaker.
[0073] Step S2200: determining the position to be detected of at least one sound source relative to the sound box according to the sound source data.
[0074] In this embodiment, the sound source data collected by the microphone may be emitted by at least one sound source.
[0075] According to the sound source data, the position of each sound source to be detected relative to the speaker and the volume of each sound source can be determined.
[0076] In this embodiment, the directions to be detected correspond to the sound sources one by one, and the number of the directions to be detected finally obtained is the same as the number of the sound sources.
[0077] Step S2300, controlling the camera of the speaker to shoot toward the direction to be detected, and obtaining the image to be detected corresponding to the direction to be detected.
[0078] When there are multiple directions to be detected, the camera of the speaker can be controlled to shoot respectively toward each direction to be detected to obtain a test image of each direction to be detected.
[0079] In one embodiment of the present disclosure, controlling the camera of the speaker to shoot toward the direction to be detected, and obtaining the image to be detected corresponding to the direction to be detected may include the following steps S2310 to S2320:
[0080] Step S2310, traverse at least one direction to be detected.
[0081] In one embodiment of the present disclosure, before executing step S2310, the method may further include steps S2330 to S2340 as follows:
[0082] Step S2330: Determine the volume of at least one sound source according to the sound source data.
[0083] Step S2340: sort the directions to be detected according to the volume of at least one sound source.
[0084] In this embodiment, the directions to be detected may be sorted in descending order of the volume of the sound source.
[0085] For example, when the volume of sound source 1 is 50dB, the volume of sound source 2 is 30dB, and the volume of sound source 3 is 40dB, the sound source 1 is sorted in descending order of volume. The detected position 1 of sound source 1 relative to the speaker, the detected position 2 of sound source 2 relative to the speaker, and the detected position 3 of sound source 3 relative to the speaker are sorted. The sorting result may be detected position 1, detected position 3, and detected position 2.
[0086] On this basis, traversing at least one direction to be detected may be: traversing at least one direction to be detected according to the sorting order of the directions to be detected.
[0087] For example, when the sorting results are position 1 to be detected, position 3 to be detected, and position 2 to be detected, it is possible to first traverse to position 1 to be detected, then to position 3 to be detected, and then to position 2 to be detected.
[0088] Step S2320, controlling the camera of the speaker to shoot toward the currently traversed position to be detected, and obtaining the image to be detected of the currently traversed position to be detected.
[0089] In one embodiment of the present disclosure, step S2400 may be performed after obtaining the image to be detected at each position to be detected, or may be performed after obtaining the image to be detected at the currently traversed position to be detected.
[0090] Step S2400 , selecting a target position of a target user interacting with the speaker relative to the speaker from positions to be detected according to the image to be tested.
[0091] On the basis of the aforementioned steps S2310 to S2320, according to the image to be detected, selecting the target position of the target user interacting with the speaker relative to the speaker from the positions to be detected may include the following steps S2410 to S2430:
[0092] Step S2410, determining the posture angle of the face in the image to be tested in the currently traversed orientation to be tested.
[0093] In one embodiment of the present disclosure, the posture angle of the face in the test image may include a heading angle and a pitch angle.
[0094] Determining the posture angle of the face in the image to be detected in the currently traversed orientation to be detected may include the following steps S2411 to S2413:
[0095] Step S2411, constructing a spatial rectangular coordinate system for the face according to the image to be tested at the currently traversed orientation to be tested; a first coordinate axis of the spatial rectangular coordinate system for the face is parallel to the straight line where the eyes are located in the image to be tested at the currently traversed orientation to be tested; a second coordinate axis of the spatial rectangular coordinate system for the face is parallel to the first plane and perpendicular to the first coordinate axis, the first plane being the plane where at least three specified positions of the face in the image to be tested are located; a third coordinate axis of the spatial rectangular coordinate system for the face is perpendicular to the first coordinate axis and the second coordinate axis.
[0096] In this embodiment, the at least three designated positions may include the positions of two mouth corners and the position of the lowermost end of the chin. The plane formed by these three designated positions is the first plane.
[0097] Then, based on the positions of the eyes of the face and at least three specified positions of the face in the currently traversed test image to be detected, a spatial rectangular coordinate system of the face can be constructed, and the directions of the three coordinate axes in the constructed spatial rectangular coordinate system of the face are one-to-one corresponding to the image to be tested.
[0098] Step S2412, construct a reference coordinate system; the fourth coordinate axis of the reference coordinate system is parallel to the straight line where the eyes are located when the face is facing the camera; the fifth coordinate axis of the reference coordinate system is perpendicular to the camera lens.
[0099] In this embodiment, a reference coordinate system may be constructed based on the direction to be detected that the camera is currently traversing.
[0100] The reference coordinate system may be a spatial rectangular coordinate system, and further includes a sixth coordinate axis, and the sixth coordinate axis may be perpendicular to the fourth coordinate axis and the fifth coordinate axis.
[0101] When a person's face faces the camera, in the obtained image, the straight line where the two eyes are located is parallel to one side of the image.
[0102] In one example, since the roll angle of the camera usually does not change during the rotation of the speaker, when constructing a reference coordinate system, the fourth coordinate axis can be predetermined according to the setting posture of the camera.
[0103] Step S2413, determining the angle between the first coordinate axis and the fourth coordinate axis as the heading angle; determining the angle between the third coordinate axis and the fifth coordinate axis as the pitch angle.
[0104] In this embodiment, when the face is looking up or down, the pitch angle of the face is α, and the value of α is -π / 2 to π / 2. When the face is facing the camera, α is 0, and when the first plane is perpendicular to the lens of the camera, α is -π / 2 or π / 2. When the face rotates left and right, the heading angle of the face is β, and the value of β is also -π / 2 to π / 2. When the face is facing the camera, β is 0, and when the straight line where the eyes of the face are located is perpendicular to the fourth coordinate axis, β is -π / 2 or π / 2.
[0105] Step S2420, determining whether the currently traversed position to be detected meets a preset condition based on the posture angle of the face in the image to be detected at the currently traversed position to be detected.
[0106] In one embodiment of the present disclosure, the heading angle and the pitch angle may be expressed in radians. According to the posture angle of the face in the image to be detected of the currently traversed position to be detected, determining whether the currently traversed position to be detected meets the preset conditions may include the following steps S2421 to S2423:
[0107] Step S2421, determining the detection result of the currently traversed position to be detected according to the heading angle and pitch angle of the face in the image to be detected at the currently traversed position to be detected.
[0108] In one embodiment of the present disclosure, the detection result of the currently traversed position to be detected may be determined by the following formula:
[0109] Result=(π / 2-|α|)2*(π / 2-|β|)2
[0110] Among them, α is the pitch angle, β is the heading angle, and Result is the detection result of the current traversed direction to be detected.
[0111] Step S2422: When the detection result of the currently traversed position to be detected is greater than or equal to a preset result threshold, it is determined that the currently traversed position to be detected meets the preset condition.
[0112] In this embodiment, the result threshold may be set in advance according to the application scenario or specific requirements. For example, the result threshold may be 1. Then, when the detection result of the currently traversed position to be detected is greater than or equal to 1, it can be determined that the currently traversed position to be detected meets the preset condition; when the detection result of the currently traversed position to be detected is less than 1, it can be determined that the currently traversed position to be detected does not meet the preset condition.
[0113] Step S2423: when the detection result of the currently traversed position to be detected is less than a preset result threshold, it is determined that the currently traversed position to be detected does not meet the preset condition.
[0114] In another embodiment of the present disclosure, the heading angle and the pitch angle may be expressed in angles. According to the posture angle of the face in the image to be detected of the currently traversed position to be detected, determining whether the currently traversed position to be detected meets the preset condition may include:
[0115] Determine whether the heading angle and pitch angle of the face in the image to be detected at the currently traversed orientation to be detected are both less than or equal to the preset angle threshold. If so, determine that the currently traversed orientation to be detected meets the preset conditions; if not, determine that the currently traversed orientation to be detected does not meet the preset conditions.
[0116] In this embodiment, the angle threshold may be pre-set according to the application scenario or specific needs. For example, the angle threshold may be 30°. Then, when the heading angle and pitch angle of the face in the image to be detected of the currently traversed orientation to be detected are both less than or equal to 30°, it can be determined that the currently traversed orientation to be detected meets the preset conditions; when the heading angle and / or pitch angle of the face in the image to be detected of the currently traversed orientation to be detected is greater than 30°, it can be determined that the currently traversed orientation to be detected does not meet the preset conditions.
[0117] Step S2430, when the currently traversed position to be detected meets the preset conditions, the currently traversed position to be detected is taken as the target position, and the traversal is stopped.
[0118] In this embodiment, the positions to be detected are sorted in advance according to the volume of the sound source, and traversed in the sorted order. It can be considered that the sound source of the position to be detected that is traversed first is more likely to be the target user interacting with the speaker. Therefore, if the currently traversed position to be detected meets the preset conditions, the currently traversed position to be detected is taken as the target position, and the traversal is stopped.
[0119] Step S2440: When the currently traversed position to be detected does not meet the preset condition, continue to traverse the next position to be detected.
[0120] In one embodiment of the present disclosure, after the traversal is completed, if all the directions to be detected do not meet the preset conditions, the direction to be detected with the largest detection result may be used as the target direction.
[0121] Step S2500, control the camera of the speaker to face the target direction.
[0122] Through the method of this embodiment, the target position of the target user interacting with the speaker relative to the speaker is determined by combining the sound source information and the image to be tested, and the camera of the speaker is controlled to face the target position. This allows the camera of the speaker to accurately face the position where the target user interacting with the speaker appears, thereby improving the user experience.
[0123] <Example>
[0124] Figure 3 The flowchart is an example of a speaker control method according to an embodiment of the present disclosure.
[0125] according to Figure 3 As shown, the method may include steps S3001 to S3012 as shown below:
[0126] Step S3001, obtaining sound source data collected by the microphone of the speaker.
[0127] Step S3002: Determine, based on the sound source data, the position to be detected of at least one sound source relative to the sound box and the volume of at least one sound source.
[0128] Step S3003: sort the directions to be detected according to the volume of at least one sound source.
[0129] Step S3004: traverse at least one direction to be detected according to the sorting order of the directions to be detected.
[0130] Step S3005, controlling the camera of the speaker to shoot toward the currently traversed position to be detected, and obtaining the image to be detected of the currently traversed position to be detected.
[0131] Step S3006, determining the heading angle and pitch angle of the face in the image to be detected in the currently traversed orientation to be detected.
[0132] Step S3007, determining the detection result of the currently traversed position to be detected according to the heading angle and pitch angle of the face in the image to be detected at the currently traversed position to be detected.
[0133] Step S3008: When the detection result of the currently traversed position to be detected is greater than or equal to a preset result threshold, it is determined that the currently traversed position to be detected meets the preset condition.
[0134] Step S3009: when the detection result of the currently traversed position to be detected is less than the preset result threshold, it is determined that the currently traversed position to be detected does not meet the preset condition, and the next position to be detected is traversed.
[0135] Step S3010: When the currently traversed position to be detected meets the preset conditions, the currently traversed position to be detected is used as the target position, and the traversal is stopped.
[0136] Step S3011, after the traversal is completed, if all the directions to be detected do not meet the preset conditions, the direction to be detected with the largest detection result is taken as the target direction.
[0137] Step S3012, control the camera of the speaker to face the target direction.
[0138] <Device Example>
[0139] In this embodiment, a speaker control device 4000 is provided. Figure 4 As shown, it includes a sound source data acquisition module 4100, a module for determining a position to be detected 4200, a module for obtaining an image to be detected 4300, a module for determining a target position 4400, and a camera control module 4500. The sound source data acquisition module 4100 is used to acquire the sound source data collected by the microphone of the speaker; the module for determining the position to be detected 4200 is used to determine the position to be detected of at least one sound source relative to the speaker according to the sound source data; the module for obtaining the image to be detected 4300 is used to control the camera of the speaker to shoot toward the position to be detected to obtain the image to be tested corresponding to the position to be detected; the module for determining the target position 4400 is used to select the target position of the target user interacting with the speaker relative to the speaker from the position to be detected according to the image to be tested; and the camera control module 4500 is used to control the camera of the speaker to face the target position.
[0140] In one embodiment of the present disclosure, the module 4300 for obtaining the image to be tested may also be used for:
[0141] Traversing at least one position to be detected;
[0142] Control the camera of the speaker to shoot towards the currently traversed position to be detected, and obtain the image to be detected of the currently traversed position to be detected;
[0143] The target position determination module 4400 may also be used to:
[0144] Determine the posture angle of the face in the image to be tested at the currently traversed position to be tested;
[0145] Determine whether the currently traversed position to be detected meets the preset conditions according to the posture angle of the face in the image to be detected of the currently traversed position to be detected;
[0146] When the currently traversed position to be detected meets the preset conditions, the currently traversed position to be detected is taken as the target position and the traversal is stopped.
[0147] In one embodiment of the present disclosure, the attitude angle includes a heading angle and a pitch angle.
[0148] According to the posture angle of the face in the image to be detected at the currently traversed position to be detected, determining whether the currently traversed position to be detected meets the preset conditions includes:
[0149] Determine the detection result of the currently traversed position to be detected according to the heading angle and pitch angle of the face in the image to be detected of the currently traversed position to be detected;
[0150] When the detection result of the currently traversed position to be detected is greater than or equal to a preset result threshold, it is determined that the currently traversed position to be detected meets the preset condition;
[0151] When the detection result of the currently traversed position to be detected is less than a preset result threshold, it is determined that the currently traversed position to be detected does not meet the preset condition.
[0152] In one embodiment of the present disclosure, the attitude angle includes a heading angle and a pitch angle.
[0153] Determine the posture angle of the face in the image to be tested in the currently traversed orientation to be tested, including:
[0154] According to the image to be tested at the currently traversed position to be tested, a spatial rectangular coordinate system of the face is constructed; a first coordinate axis of the spatial rectangular coordinate system of the face is parallel to the straight line where the eyes are located in the image to be tested at the currently traversed position to be tested; a second coordinate axis of the spatial rectangular coordinate system of the face is parallel to the first plane and perpendicular to the first coordinate axis, the first plane being the plane where at least three specified positions of the face in the image to be tested are located; a third coordinate axis of the spatial rectangular coordinate system of the face is perpendicular to the first coordinate axis and the second coordinate axis;
[0155] A reference coordinate system is constructed; the fourth coordinate axis of the reference coordinate system is parallel to the straight line where the eyes are located when the face of the person is facing the camera; the fifth coordinate axis of the reference coordinate system is perpendicular to the lens of the camera;
[0156] The angle between the first coordinate axis and the fourth coordinate axis is determined as the heading angle; the angle between the third coordinate axis and the fifth coordinate axis is determined as the pitch angle.
[0157] In one embodiment of the present disclosure, the speaker control device 4000 may further include:
[0158] A module used to continue traversing the next position to be detected when the currently traversed position to be detected does not meet the preset conditions.
[0159] In one embodiment of the present disclosure, the speaker control device 4000 may further include:
[0160] A module for taking the direction to be detected with the largest detection result as the target direction when none of the directions to be detected meet the preset conditions.
[0161] In one embodiment of the present disclosure, the speaker control device 4000 may further include:
[0162] A module for determining the volume of at least one sound source based on the sound source data;
[0163] A module for sorting the positions to be detected according to the volume of at least one sound source;
[0164] Traversing at least one position to be detected includes:
[0165] At least one position to be detected is traversed according to the sorting order of the positions to be detected.
[0166] Those skilled in the art should understand that the speaker control device 4000 can be implemented in various ways. For example, the speaker control device 4000 can be implemented by configuring the processor with instructions. For example, the instructions can be stored in a ROM, and when the device is started, the instructions are read from the ROM into a programmable device to implement the speaker control device 4000. For example, the speaker control device 4000 can be solidified into a dedicated device (such as an ASIC). The speaker control device 4000 can be divided into independent units, or they can be combined together for implementation. The speaker control device 4000 can be implemented by one of the various implementations described above, or can be implemented by a combination of two or more of the various implementations described above.
[0167] In this embodiment, the speaker control device 4000 can have multiple implementation forms. For example, the speaker control device 4000 can be a functional module running in any software product or application that provides speaker control services, or it can be a peripheral embedded component, plug-in, patch, etc. of these software products or applications, or it can also be these software products or applications themselves.
[0168] <Speakers>
[0169] In this embodiment, a speaker 5000 is also provided.
[0170] In an example, the speaker 5000 may include the aforementioned speaker control device 4000 .
[0171] In another example, Figure 5 As shown, the speaker 5000 may include:
[0172] A memory 5100 and a processor 5200, wherein the memory 5100 is used to store executable instructions; the instructions are used to control the processor 5200 to execute the aforementioned speaker control method.
[0173] <Readable Storage Medium>
[0174] In this embodiment, a readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the speaker control method of any embodiment of the present disclosure is implemented.
[0175] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
[0176] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0177] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0178] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0179] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0180] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0181] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0182] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.
[0183] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.
Claims
1. A speaker control method, It is characterized in that include: Get the sound source data collected by the speaker's microphone; The sound source data is used to determine the position to be detected and the volume of each sound source relative to the sound box; Determining, based on the sound source data, a position to be detected of at least one sound source relative to the sound box; Determining the volume of the at least one sound source according to the sound source data; sorting the directions to be detected according to the volume of the at least one sound source; Traversing at least one of the to-be-detected directions according to the sorting order of the to-be-detected directions; Control the camera of the speaker to shoot toward the currently traversed position to be detected, and obtain the image to be detected of the currently traversed position to be detected; Determine the posture angle of the face in the image to be tested at the currently traversed position to be tested; Determining whether the currently traversed position to be detected meets a preset condition according to the posture angle of the face in the image to be detected of the currently traversed position to be detected; When the currently traversed position to be detected meets the preset condition, the currently traversed position to be detected is used as the target position, and the traversal is stopped; The camera of the speaker is controlled to face the target direction.
2. The method according to claim 1, It is characterized in that The attitude angle includes heading angle and pitch angle. Determining whether the currently traversed position to be detected meets a preset condition according to the posture angle of the face in the image to be detected of the currently traversed position to be detected includes: Determine a detection result of the currently traversed position to be detected according to the heading angle and the pitch angle of the face in the image to be detected of the currently traversed position to be detected; In the case where the detection result of the currently traversed position to be detected is greater than or equal to a preset result threshold, determining that the currently traversed position to be detected meets the preset condition; When the detection result of the currently traversed position to be detected is less than a preset result threshold, it is determined that the currently traversed position to be detected does not meet the preset condition.
3. The method according to claim 1, It is characterized in that The attitude angle includes heading angle and pitch angle. The step of determining the posture angle of the face in the image to be detected in the currently traversed orientation to be detected comprises: According to the image to be tested at the currently traversed position to be tested, a spatial rectangular coordinate system of the face is constructed; a first coordinate axis of the spatial rectangular coordinate system of the face is parallel to the straight line where the eyes are located in the image to be tested at the currently traversed position to be tested; a second coordinate axis of the spatial rectangular coordinate system of the face is parallel to a first plane and perpendicular to the first coordinate axis, the first plane being a plane where at least three specified positions of the face in the image to be tested are located; a third coordinate axis of the spatial rectangular coordinate system of the face is perpendicular to the first coordinate axis and the second coordinate axis; Constructing a reference coordinate system; the fourth coordinate axis of the reference coordinate system is parallel to the straight line where the eyes are located when the face of the person is facing the camera; the fifth coordinate axis of the reference coordinate system is perpendicular to the lens of the camera; An angle between the first coordinate axis and the fourth coordinate axis is determined as the heading angle; an angle between the third coordinate axis and the fifth coordinate axis is determined as the pitch angle.
4. The method according to claim 1, It is characterized in that The method further comprises: When the currently traversed position to be detected does not meet the preset condition, continue to traverse the next position to be detected.
5. The method according to claim 2, It is characterized in that The method further comprises: When none of the to-be-detected orientations meets the preset condition, the to-be-detected orientation with the largest detection result is used as the target orientation.
6. A speaker control device, It is characterized in that include: A sound source data acquisition module is used to acquire the sound source data collected by the microphone of the speaker; The sound source data is used to determine the position to be detected and the volume of each sound source relative to the sound box; A module for determining a position to be detected, used to determine a position to be detected of at least one sound source relative to the sound box according to the sound source data; A module for obtaining an image to be tested, used for controlling the camera of the speaker to shoot toward the position to be tested, and obtaining an image to be tested corresponding to the position to be tested; a target orientation determination module, configured to select, from the orientations to be detected, a target orientation of a target user interacting with the sound box relative to the sound box according to the image to be detected; A camera control module, used for controlling the camera of the speaker to face the target direction; A module for determining the volume of the at least one sound source according to the sound source data; a module for sorting the directions to be detected according to the volume of the at least one sound source, The module for obtaining the image to be tested is specifically used for: Traversing at least one of the to-be-detected directions according to the sorting order of the to-be-detected directions; Control the camera of the speaker to shoot toward the currently traversed position to be detected, and obtain the image to be detected of the currently traversed position to be detected; The target position determination module is specifically used for: Determine the posture angle of the face in the image to be tested at the currently traversed position to be tested; Determining whether the currently traversed position to be detected meets a preset condition according to the posture angle of the face in the image to be detected of the currently traversed position to be detected; When the currently traversed position to be detected meets the preset condition, the currently traversed position to be detected is used as the target position, and the traversal is stopped.
7. A speaker, It is characterized in that include: The device as claimed in claim 6; or, A processor and a memory, wherein the memory is used to store instructions, and the instructions are used to control the processor to execute the method according to any one of claims 1-5.
8. A readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 5 when executed by a processor.
Citation Information
Patent Citations
Sound box control method, device and equipment and computer readable medium
CN108513207A
A method and apparatus for interactive shooting
CN109040584A
Sound source positioning method of audio equipment and audio equipment
CN110691196A
Image shooting method and device, terminal and computer storage medium
CN111093022A
Method and device for controlling camera to rotate based on sound source localization
CN112492207A
