Pickup assembly and control method and control device thereof
By using multiple phased arrays and image recognition technologies in the sound pickup component, the phased area of the tone-picking target is determined and the working state of the phased array is controlled, and the problem of high power consumption of the sound pickup component in public places is solved, and the realization of the realization of the effect of the sound pickup component is achieved. Accurate sound pickup and low noise effects.
Patent Information
- Application Number
- CN202411303307.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-09-18
AI Technical Summary
The existing sound pickup components consume high power in public places, making it difficult to effectively reduce noise and improve sound pickup efficiency.
Using multiple phased array sound pickup components, the phased area of the target to be picked up is determined through image information and the working state of the phased array is controlled. Only the phased array with the most suitable target sound position is started to pick up sounds, and other arrays are other arrays. In stop state to reduce power consumption.
It realizes accurate sound pickup while effectively reducing the power consumption of the sound pickup assembly, improves the sound pickup efficiency and reduces the introduction of noise.
Smart Images

Figure CN118900374A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of sound pickup assembly control, and in particular to a sound pickup assembly and a control method and a control device thereof. Background Art
[0002] In order to improve the sound pickup quality of users speaking in some public places, such as conference rooms and meeting rooms, sound pickup components are often installed in such places. The sound pickup components collect the user's speaking sound through the phased array microphones therein.
[0003] Currently, in the field of sound pickup components, how to reduce the power consumption of the sound pickup components has become a major exploration direction for those skilled in the art. Summary of the invention
[0004] The main purpose of the present invention is to provide a sound pickup component control method, which aims to reduce the power consumption of the sound pickup component, while improving the sound pickup efficiency through phase control and reducing noise introduction.
[0005] To achieve the above object, the present invention proposes a method for controlling a sound pickup assembly, wherein the sound pickup assembly includes a plurality of phased arrays, each of the phased arrays is provided with at least two microphones arranged in a horizontal linear manner, and a preset sound pickup area of the sound pickup assembly is divided into a plurality of phased areas. The method includes: Step S100, acquiring image information in the sound pickup area; Step S200: when it is determined according to the image information that a user is a target to be picked up, a phase control area where the target to be picked up is located is determined and set as a target phase control area; Step S300, obtaining the distances between the target phased area and the plurality of phased arrays respectively to obtain a plurality of distances, and determining the target phased array according to the size order of the plurality of distances, and controlling the target phased array to be in a working state and controlling the other phased arrays among the plurality of phased arrays except the target phased array to be in a stopped state, so as to achieve accurate sound pickup by the sound pickup component while effectively reducing the power consumption of the sound pickup component.
[0006] Optionally, the step of determining, according to the image information, that there is a user target to be picked up specifically includes: Step S210: determining a user action according to the image information; Step S220: When it is determined that a user has made a lip movement according to the user action, the user is determined as a target for sound pickup.
[0007] Optionally, the step S220 specifically includes: When it is determined based on the user action that the user maintains lip movements within a preset first time length and the opening and closing angles of the upper lip and the lower lip are within a preset angle range, the user is determined as a target to be picked up.
[0008] Optionally, the sound pickup assembly includes a sound pickup device, and the plurality of phased arrays are integrated in the sound pickup device; the step of acquiring the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances specifically includes: Step S310, acquiring distances between the target phased area and the plurality of phased arrays respectively to obtain a plurality of distances; Step S320, arranging the plurality of distances from small to large to obtain a first order; Step S330: According to the first order, determine the phased array corresponding to the smallest one of the multiple distances as a target phased array.
[0009] Optionally, the step S330 specifically includes: Step S331, according to the first order, determining the phased array corresponding to the smallest one of the plurality of distances as the target phased array; Step S332, obtaining an angle between a line between a target to be picked up and the target phased array and a central axis of the target phased array; Step S333: if the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is less than the first preset difference, or the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is less than the second preset difference, or the included angle is not within the preset sound pickup angle range, then in the first order, the second smallest one of the multiple distances is used as the new target phased array and steps S332 to S333 are repeatedly performed until the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is greater than the first preset difference, the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is greater than the second preset difference, and the included angle is within the preset sound pickup angle range; Step S334: determine the target phased array as the target phased array.
[0010] Optionally, the sound pickup assembly includes a plurality of sound pickup devices, each of which is integrated with a plurality of phased arrays; the step of acquiring the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances specifically includes: Step S340, acquiring the facial movement of the target to be sound-collected, and determining the sound-collecting device that the face of the target to be sound-collected is facing as the target sound-collecting device; Step S350, obtaining distances between the target phased area and a plurality of phased arrays in the target sound pickup device, to obtain a plurality of distances; Step S360, arranging the plurality of distances from small to large to obtain a second order; Step S370: According to the second order, determine the phased array corresponding to the smallest one of the multiple distances as the target phased array.
[0011] Optionally, the step S370 specifically includes: Step S371, according to the second order, determining the phased array corresponding to the smallest one of the plurality of distances as the target phased array; Step S372, obtaining an angle between a line between a target to be picked up and the target phased array and a central axis of the target phased array; Step S373: if the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is less than the first preset difference, or the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is less than the second preset difference, or the included angle is not within the preset sound pickup angle range, then in the second order, the second smallest one of the multiple distances is used as the new target phased array and steps S372 to S373 are repeatedly performed until the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is greater than the first preset difference, the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is greater than the second preset difference, and the included angle is within the preset sound pickup angle range; Step S374: determine the target phased array as the target phased array.
[0012] Optionally, the step of acquiring the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances further comprises: If the difference between the angles between the connecting line between any of the target to be sound picked up and the target phased array and the central axis of the target phased array is less than a first preset difference from the upper limit of the preset sound pickup angle range of the target phased array or less than a second preset difference from the lower limit of the preset sound pickup angle range of the target phased array, then the binaural connecting line of the target to be sound picked up is used as a reference, and the side to which the face of the target to be sound picked up faces is taken as the first side; The one of the plurality of sound pickup devices on the first side that is at the shortest distance from the target phase control area is determined as the target sound pickup device, and steps S350 to S374 are executed.
[0013] The present invention also provides a control device, which includes: memory; and, processor; A sound pickup assembly control program stored in the memory and executed by the processor, wherein when the sound pickup assembly control program is executed by the processor, the sound pickup assembly control method as described in any one of the above items is implemented.
[0014] The present invention also proposes a sound pickup assembly, comprising the control device as described above and multiple phased arrays, each of the phased arrays is provided with at least two microphones arranged along a horizontal linear arrangement, and the preset sound pickup area of the sound pickup assembly is divided into multiple phased areas.
[0015] The present invention proposes a method for controlling a sound pickup component, including: first obtaining image information in a sound pickup area, then determining, based on the image information, that a user is a target to be picked up, determining the phased area where the target to be picked up is located and setting it as a target phased area, and finally determining the target phased array based on the relative position of the target phased area and a plurality of phased arrays, and controlling the target phased array to be in a working state and controlling the other phased arrays except the target phased array in the plurality of phased arrays to be in a stopped state. Thus, in practical applications, when the sound pickup component determines that there is a target to be picked up in the current scene, it will only start the phased array with the most appropriate relative position to pick up the sound emitted by the target to be picked up, thereby effectively reducing the power consumption of the sound pickup component while achieving accurate sound pickup, and at the same time improving the sound pickup efficiency and reducing the introduction of noise through phase control. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.
[0017] Figure 1 It is a flow chart of an embodiment of a method for controlling a sound pickup assembly of the present invention; Figure 2 It is a flow chart of another embodiment of the method for controlling a sound pickup assembly of the present invention; Figure 3 A schematic flow chart of another embodiment of the method for controlling a sound pickup assembly of the present invention; Figure 4A flowchart of another embodiment of the method for controlling a sound pickup assembly of the present invention; Figure 5 It is a flow chart of another embodiment of the method for controlling a sound pickup assembly of the present invention; Figure 6 It is a flow chart of another embodiment of the method for controlling a sound pickup assembly of the present invention; Figure 7 A schematic diagram of a sound pickup device in one embodiment of the sound pickup assembly of the present invention; Figure 8 It is a schematic diagram of another embodiment of the pickup assembly of the present invention including multiple pickup devices.
[0018] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0020] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0021] In order to improve the sound pickup quality of users speaking in some public places, such as conference rooms and meeting rooms, sound pickup components are often installed in such places. The sound pickup components collect the user's speaking sound through the phased array microphones therein.
[0022] Currently, in the field of sound pickup components, how to reduce the power consumption of the sound pickup components has become a major exploration direction for those skilled in the art.
[0023] To this end, the present invention proposes a control method for a sound pickup component, wherein the sound pickup component includes a plurality of phased arrays, each of which is provided with at least two microphones arranged in a horizontal linear manner, and the preset sound pickup area of the corresponding sound pickup component is divided into a plurality of phased areas. Among them, the plurality of phased areas can be divided in advance by the research and development personnel for the sound pickup area of the sound pickup component, or can be set by the user according to the needs. The phased arrays can be arranged in the front-to-back direction and / or in the vertical direction. The plurality of microphones in each phased array can achieve sound enhancement and thus achieve accurate sound pickup by adjusting the phase difference. It can be understood that the sound pickup component also includes a control device for executing and storing the following method, and the control device can be implemented by a main controller, such as MCU, DSP (Digital Signal Process, digital signal processing chip), FPGA (Field Programmable Gate Array, programmable logic gate array chip), PLC, SOC (System On Chip, system-on-chip), etc.
[0024] In one embodiment of the present invention, reference Figure 1 , methods include: Step S100, acquiring image information in the sound pickup area; In this embodiment, the sound pickup assembly may include a camera module, such as a camera and an image processing module, for capturing an image in the current sound pickup area, and outputting corresponding image information to the control device after digital processing.
[0025] Step S200: when it is determined according to the image information that a user is a target to be picked up, a phase control area where the target to be picked up is located is determined and set as a target phase control area; Optionally, in one embodiment, reference Figure 2 The step of determining that there is a user to be picked up according to the image information specifically includes: Step S210: determining a user action according to the image information; Step S220: When it is determined that a user has made a lip movement according to the user action, the user is determined as a target for sound pickup.
[0026] In this embodiment, the control device recognizes the image information acquired and output by the camera module through an image recognition chip electrically connected thereto or an image processing unit integrated therein, so as to determine the user actions of all users in the conference room.
[0027] When it is confirmed based on the above user actions that a certain user has lip movements, such as the upper and lower lips opening and closing, it will be confirmed that the user is currently speaking, and the control device will identify the user as a target to be picked up.
[0028] It should be understood that in actual situations, users may also make lip movements that are not related to speaking needs, such as yawning, smacking lips, etc. In the above cases, users may be easily misjudged as targets to be picked up.
[0029] To this end, based on the above embodiments, in one embodiment, step S220 specifically includes: When it is determined based on the user action that the user maintains lip movements within a preset first time length and the opening and closing angles of the upper lip and the lower lip are within a preset angle range, the user is determined as a target to be picked up.
[0030] In this embodiment, the control device can determine the opening and closing angles of the user's upper and lower lips according to the user actions obtained in the above embodiment process. If the opening and closing angles of the upper and lower lips are within the preset angle range, such as 20°-40°, and the upper and lower lips have continuous lip movement, then the control device will determine that the current user does have a need to speak, and will determine it as a target to be picked up. Similarly, if the opening and closing angles of the upper and lower lips of the user are outside the preset angle range, or the upper and lower lips do not have continuous opening and closing movements, it means that the current user has no intention to speak, and the control device will not use it as a target to be picked up.
[0031] After the control device determines the target to be picked up, it will determine the target phase control area where the target to be picked up is located based on the image of the sound pickup area captured above and the multiple phase control areas divided in advance.
[0032] Step S300, obtaining the distances between the target phased area and the plurality of phased arrays respectively to obtain a plurality of distances, and determining the target phased array according to the size order of the plurality of distances, and controlling the target phased array to be in a working state and controlling the other phased arrays among the plurality of phased arrays except the target phased array to be in a stopped state, so as to achieve accurate sound pickup by the sound pickup component while effectively reducing the power consumption of the sound pickup component.
[0033] In this embodiment, after confirming the target phased-control area, the control device will determine the phased-control array with better sound pickup effect according to the positional relationship between the target phased-control area and multiple phased-control arrays, such as distance, angle, etc., and use it as the target phased-control array. Then, the control device will only control the target phased-control array to be in a working state and control the other phased-control arrays except the target phased-control array in the multiple phased-control arrays to be in a stopped state, so that the sound pickup component can pick up the sound emitted by the sound pickup target while effectively reducing the power consumption of the sound pickup component. If the power supply of the sound pickup component is an independent energy storage module, the battery life of the sound pickup component can also be effectively improved.
[0034] For example, the relative position is the distance between the target phased-control area and the multiple phased-control arrays. After confirming the target phased-control area, the control device can determine the distance between the current target phased-control area and the multiple phased-control arrays according to the phased-control area-phased-control array distance mapping table pre-stored by the R&D personnel, or calculate and determine the distance between the current target phased-control area and the multiple phased-control arrays according to the image of the above-mentioned sound pickup area. The phased-control array corresponding to the one with the smallest distance is used as the target phased-control array, and only the target phased-control array is controlled to be in a working state, and the other phased-control arrays except the target phased-control array in the multiple phased-control arrays are controlled to be in a stopped state, so as to ensure that the sound pickup component can pick up the sound emitted by the sound pickup target while effectively reducing the power consumption of the sound pickup component.
[0035] The present invention proposes a method for controlling a sound pickup component, including: first obtaining image information in a sound pickup area, then determining, based on the image information, that a user is a target to be picked up, determining the phased area where the target to be picked up is located and setting it as a target phased area, and finally determining the target phased array based on the relative position of the target phased area and a plurality of phased arrays, and controlling the target phased array to be in a working state and controlling the other phased arrays except the target phased array in the plurality of phased arrays to be in a stopped state. Thus, in practical applications, when the sound pickup component determines that there is a target to be picked up in the current scene, it will only start the phased array with the most appropriate relative position to pick up the sound emitted by the target to be picked up, thereby effectively reducing the power consumption of the sound pickup component while achieving accurate sound pickup, and at the same time improving the sound pickup efficiency and reducing the introduction of noise through phase control.
[0036] It is important to understand that in practice, reference Figure 7 The sound pickup assembly includes a sound pickup device, and a plurality of the phased arrays are integrated in the sound pickup device.
[0037] In one embodiment of the present invention, reference Figure 3 Based on the above-mentioned sound pickup assembly including a sound pickup device, the step of obtaining the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances specifically includes: Step S310, acquiring distances between the target phased area and the plurality of phased arrays respectively to obtain a plurality of distances; Step S320, arranging the plurality of distances from small to large to obtain a first order; Step S330: According to the first order, determine the phased array corresponding to the smallest one of the multiple distances as a target phased array.
[0038] In this embodiment, the control device can determine multiple distances between the target phased-control area and multiple phased-control arrays based on the method described in the above embodiment. Then, the control device will arrange these multiple distances from small to large to obtain a first order, and use the one with the smallest distance in the first order as the target phased-control array, that is, the phased-control array closest to the target to be picked up (also the target phased-control area) as the target phased-control array. In this way, in the subsequent process of controlling only the target phased-control array to start and perform the sound pickup work, it can not only ensure that the sound pickup component can pick up the sound emitted by the target to be picked up, but also effectively reduce the power consumption of the sound pickup component.
[0039] It should be understood that for each phased array, the angle at which it can achieve accurate sound pickup is limited, that is, the target to be picked up must be within a certain angle ± based on the central axis of the phased array, and the phased array can pick up the sound. For example, ±7°, where ±7° is also the preset sound pickup angle range corresponding to the phased array. It can be seen from the above embodiments that in order to ensure the sound pickup effect, the phased array closest to the target to be picked up will be selected as the target phased array, but the target to be picked up may not be within the preset sound pickup angle range of the target phased array, or although it is within the preset sound pickup angle range, its position is close to the boundary value of the preset sound pickup angle range. At this time, if the target phased array starts working, it cannot guarantee a sufficiently good sound pickup effect. For example, refer to Figure 7 From the above embodiments, it can be seen that the arrangement of the multiple phased arrays in the sound pickup device can be arranged along the front-to-back direction and / or along the left-to-right direction. The multiple phased arrays are arranged along the front-to-back direction and along the left-to-right direction at the same time, so that the multiple phased arrays, for example Figure 7 Settings shown. Figure 7 The sound pickup device 10 has three phased arrays 20A, 20B and 20C arranged as shown in the figure, and their preset sound pickup angle ranges are all [-α / 2, α / 2], for example, α is 14°. It is clear from the figure that the phased array closest to the target to be picked up (target phased area) is the phased array 20B, but its position is close to the edge of the preset sound pickup angle range of the phased array (close to the position corresponding to -α / 2). At this time, although the phased array 20B is closest to the target to be picked up, it cannot meet the requirements of the sound pickup effect.
[0040] For this purpose, refer to Figure 4 , based on the above Figure 3 In one embodiment of the present invention, step S330 specifically includes: Step S331, according to the first order, determining the phased array corresponding to the smallest one of the plurality of distances as the target phased array; Step S332, obtaining an angle between a line between a target to be picked up and the target phased array and a central axis of the target phased array; Step S333: if the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is less than the first preset difference, or the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is less than the second preset difference, or the included angle is not within the preset sound pickup angle range, then in the first order, the second smallest one of the multiple distances is used as the new target phased array and steps S332 to S333 are repeatedly performed until the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is greater than the first preset difference, the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is greater than the second preset difference, and the included angle is within the preset sound pickup angle range; Step S334: determine the target phased array as the target phased array.
[0041] In this embodiment, after the above embodiment process, the smallest one of the multiple distances is first used as the target phased array. Then, according to the image information of the above sound pickup area, the angle between the connecting line between the target and the target phased array and the central axis of the target phased array is determined.
[0042] If the current angle is within the preset sound pickup angle range, and the difference between the current angle and the upper limit of the preset sound pickup angle range of the target phased array is greater than the first preset difference, and the difference between the current angle and the lower limit of the preset sound pickup angle range of the target phased array is greater than the second preset difference, it means that the target to be picked up is not only within the preset sound pickup angle range of the target phased array, but also not close to the boundary of the preset sound pickup angle range, wherein the first preset difference and the second preset difference may be the same or different, and are obtained and stored by the R&D personnel through multiple tests during the R&D period. At this time, the control device confirms that the current target sound pickup device can meet the sound pickup requirements, and will use the target sound pickup device as the final target sound pickup device.
[0043] If the difference between the upper limit of the preset sound pickup angle range of the phased array to be targeted is less than the first preset difference, or the difference between the angle and the lower limit of the preset sound pickup angle range of the phased array to be targeted is less than the second preset difference, or the angle is not within the preset sound pickup angle range, it means that the target to be picked up is not within the preset sound pickup angle range of the phased array to be targeted, or is close to the boundary of the preset sound pickup angle range, then the control device will set the next phased array of the current phased array to be targeted as the new phased array to be targeted according to the first sequence confirmed in the above embodiment, that is, relative to the distance between the current phased array to be targeted and the target to be picked up, the phased array corresponding to the distance one greater than the distance is set as the new phased array to be targeted according to the first sequence, and then repeat the above angle judgment process until the angle between the connecting line between the phased array to be targeted and the target to be picked up and the central axis of the phased array to be targeted is within the preset sound pickup angle range and is not close to the boundary value of the preset sound pickup angle range. Such a setting further ensures the sound pickup effect of the sound pickup component.
[0044] For example, refer to Figure 7 , the control device determines that although the target to be picked up is closest to the phased array 20B, its position is close to the boundary value of the preset sound pickup angle range of the phased array 20B, then the control device will believe that the current phased array 20B does not meet the sound pickup requirements, so according to the first order, the phased array 20A that is second closest to the target to be picked up is used as the new target phased array, and according to the above angle judgment, it is determined that the position of the target to be picked up is within the preset sound pickup angle range and is not close to the boundary value of the preset sound pickup angle range, then the phased array 20A will be used as the target phased array, and only the phased array 20A will be controlled to start, and the other phased arrays 20B and 20C will stop working, thereby achieving accurate sound pickup while effectively reducing the power consumption of the sound pickup components.
[0045] It is important to understand that in practice, reference Figure 8 The pickup assembly may further include a plurality of pickup devices, each of which is integrated with a plurality of phased arrays.
[0046] In one embodiment of the present invention, reference Figure 5 Based on the fact that the sound pickup assembly includes a plurality of sound pickup devices, the step of obtaining the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances specifically includes: Step S340, acquiring the facial movement of the target to be sound-collected, and determining the sound-collecting device that the face of the target to be sound-collected is facing as the target sound-collecting device; Step S350, obtaining distances between the target phased area and a plurality of phased arrays in the target sound pickup device, to obtain a plurality of distances; Step S360, arranging the plurality of distances from small to large to obtain a second order; Step S370: According to the second order, determine the phased array corresponding to the smallest one of the multiple distances as the target phased array.
[0047] It can be understood that, since there are multiple sound pickup devices in the sound pickup assembly, the multiple sound pickup devices can be set at different positions. At the same time, the sound pickup effect of the sound pickup device facing the face of the target to be picked up is better. Therefore, in this embodiment, the control device will first determine the facial movement of the target to be picked up according to the image information in the above embodiment, so as to determine the sound pickup device facing the face of the target to be picked up as the target sound pickup device, thereby effectively ensuring the sound pickup effect of the sound pickup assembly.
[0048] Then, after determining the target sound pickup device, the control device will determine the distances between the multiple phased arrays in the target sound pickup device and the target to be picked up, as in the embodiment in which the sound pickup assembly includes a single sound pickup device, and arrange them in order from small to large to obtain a second order, and according to the second order, the phased array corresponding to the one with the smallest distance, that is, the phased array closest to the target to be picked up, is used as the target phased array, thereby ensuring the subsequent sound pickup effect.
[0049] Based on the same reasons as in the above embodiments, in order to ensure that the position of the target to be picked up is within the preset pickup angle range of the target phased array and is not close to the edge position, the reference Figure 6 In one embodiment of the present invention, step S370 specifically includes: Step S371, according to the second order, determining the phased array corresponding to the smallest one of the plurality of distances as the target phased array; Step S372, obtaining an angle between a line between a target to be picked up and the target phased array and a central axis of the target phased array; Step S373: if the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is less than the first preset difference, or the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is less than the second preset difference, or the included angle is not within the preset sound pickup angle range, then in the second order, the second smallest one of the multiple distances is used as the new target phased array and steps S372 to S373 are repeatedly performed until the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is greater than the first preset difference, the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is greater than the second preset difference, and the included angle is within the preset sound pickup angle range; Step S374: determine the target phased array as the target phased array.
[0050] In this embodiment, the specific judgment process can refer to the above Figure 4 The process of the embodiment described above will not be described in detail here. The control device will determine a target phased array that can ensure the sound pickup effect according to the process of the embodiment described above, and in the subsequent control process, only control the target phased array in the target sound pickup device to start and control other sound pickup devices to stop working, so as to ensure the sound pickup effect of the target sound pickup and reduce the total power consumption of the sound pickup component.
[0051] It should be understood that when the sound pickup assembly includes multiple sound pickup devices, the corresponding sound pickup area is the sum of the areas of the sound pickup areas corresponding to the multiple sound pickup devices. In other words, although the target to be picked up is located in the sound pickup area of the sound pickup assembly, it is not necessarily located in the sound pickup areas of all the sound pickup devices. For example, it can be seen from the content of the above embodiment that in order to ensure the sound pickup effect, a sound pickup device facing the face of the target to be picked up will first be selected as the target sound pickup device, and then the target phased array will be selected therein. However, the position of the target to be picked up is not necessarily within the sound pickup area of the target sound pickup device (the pickup area of the sound pickup device is the sum of the pickup ranges of the multiple phased arrays therein). For example, refer to Figure 8 The facial orientation of the target to be picked up is indicated by the arrow. Although it is facing the sound pickup device 10B, it is obviously not within the preset sound pickup angle range of any phased array of the sound pickup device 10B. Therefore, the sound pickup device 10B cannot guarantee the sound pickup effect for the target to be picked up.
[0052] To this end, based on the above Figure 6 In the embodiment described above, in one embodiment of the present invention, the step of acquiring the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances further includes: If the difference between the angles between the connecting line between any of the target to be sound picked up and the target phased array and the central axis of the target phased array is less than a first preset difference from the upper limit of the preset sound pickup angle range of the target phased array or less than a second preset difference from the lower limit of the preset sound pickup angle range of the target phased array, then the binaural connecting line of the target to be sound picked up is used as a reference, and the side to which the face of the target to be sound picked up faces is taken as the first side; The one of the plurality of sound pickup devices on the first side that is at the shortest distance from the target phase control area is determined as the target sound pickup device, and steps S350 to S374 are executed.
[0053] In this embodiment, if it is determined according to the process of step S371 to step S373 that the target to be picked up is not within the preset pickup angle range of any phased array in the target pickup device, that is, the target to be picked up is not within the pickup range of the target pickup device, then based on the image information of the pickup area obtained in the above embodiment, the binaural line of the target to be picked up is used as a reference, and the side to which the face of the target to be picked up is facing is taken as the first side, and the one with the shortest distance to the target phased area among the multiple pickup devices on the first side is taken as the target pickup device.
[0054] For example, refer to Figure 8 When the control device determines that the target to be picked up is not within the sound pickup range of the sound pickup device 10B, it will determine, according to the above process, that there is a sound pickup device 10A closest to the target in the area in front of the target to be picked up, that is, within the range of the first side, and will use the sound pickup device 10A as the new target sound pickup device, and then determine the most suitable target phased array in the sound pickup device 10A according to the above embodiment process (the target to be picked up is within the preset sound pickup angle range of the target phased array in the sound pickup device 10A, and is not close to the boundary value of the preset sound pickup angle range, and the distance between the target phased array and the target phased array is as close as possible). This arrangement further improves the sound pickup effect when the sound pickup component includes multiple sound pickup devices, and reduces the power consumption of the sound pickup component while ensuring the sound pickup effect of the target to be picked up.
[0055] The present invention also proposes a control device, which includes: memory; and, processor; A sound pickup assembly control program stored in the memory and executed by the processor, wherein when the sound pickup assembly control program is executed by the processor, the sound pickup assembly control program implements any of the above-mentioned sound pickup assembly control methods.
[0056] It is worth noting that since the control device of the present invention is based on the above-mentioned sound pickup component control method, the embodiments of the control device of the present invention include all technical solutions of all embodiments of the above-mentioned sound pickup component control method, and the technical effects achieved are also exactly the same, which will not be repeated here.
[0057] The present invention also proposes a sound pickup assembly, comprising the control device as described above and multiple phased arrays, each of the phased arrays is provided with at least two microphones arranged along a horizontal linear arrangement, and the preset sound pickup area of the sound pickup assembly is divided into multiple phased areas.
[0058] It is worth noting that since the sound pickup assembly of the present invention is based on the above-mentioned control device, the embodiments of the sound pickup assembly of the present invention include all technical solutions of all embodiments of the above-mentioned control device, and the technical effects achieved are also exactly the same, which will not be repeated here.
[0059] The above contents are only optional embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.
Claims
1. A method for controlling a sound pickup assembly, characterized in that: The sound pickup assembly includes a plurality of phased arrays, each of the phased arrays is provided with at least two microphones arranged in a horizontal linear manner, and a preset sound pickup area of the sound pickup assembly is divided into a plurality of phased areas. The method includes: Step S100, acquiring image information in the sound pickup area; Step S200: when it is determined according to the image information that a user is a target to be picked up, a phase control area where the target to be picked up is located is determined and set as a target phase control area; Step S300, obtaining the distances between the target phased area and the plurality of phased arrays respectively to obtain a plurality of distances, and determining the target phased array according to the size order of the plurality of distances, and controlling the target phased array to be in a working state and controlling the other phased arrays among the plurality of phased arrays except the target phased array to be in a stopped state, so as to achieve accurate sound pickup by the sound pickup component while effectively reducing the power consumption of the sound pickup component.
2. The pickup assembly control method according to claim 1, characterized in that: The step of determining that there is a user target to be picked up according to the image information specifically includes: Step S210: determining a user action according to the image information; Step S220: When it is determined that a user has made a lip movement according to the user action, the user is determined as a target for sound pickup.
3. The method for controlling a pickup assembly as claimed in claim 2, wherein: The step S220 specifically includes: When it is determined based on the user action that the user maintains lip movements within a preset first time length and the opening and closing angles of the upper lip and the lower lip are within a preset angle range, the user is determined as a target to be picked up.
4. The method for controlling a pickup assembly as claimed in claim 3, wherein: The sound pickup assembly includes a sound pickup device, and the plurality of phased arrays are integrated in the sound pickup device; the step of obtaining the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances specifically includes: Step S310, acquiring distances between the target phased area and the plurality of phased arrays respectively to obtain a plurality of distances; Step S320, arranging the plurality of distances from small to large to obtain a first order; Step S330: According to the first order, determine the phased array corresponding to the smallest one of the multiple distances as a target phased array.
5. The method for controlling a pickup assembly as claimed in claim 4, wherein: The step S330 specifically includes: Step S331, according to the first order, determining the phased array corresponding to the smallest one of the plurality of distances as the target phased array; Step S332, obtaining an angle between a line between a target to be picked up and the target phased array and a central axis of the target phased array; Step S333: if the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is less than the first preset difference, or the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is less than the second preset difference, or the included angle is not within the preset sound pickup angle range, then in the first order, the second smallest one of the multiple distances is used as the new target phased array and steps S332 to S333 are repeatedly performed until the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is greater than the first preset difference, the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is greater than the second preset difference, and the included angle is within the preset sound pickup angle range; Step S334: determine the target phased array as the target phased array.
6. The method for controlling a pickup assembly as claimed in claim 3, wherein: The sound pickup assembly includes a plurality of sound pickup devices, each of which is integrated with a plurality of phased arrays; the step of acquiring the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances specifically includes: Step S340, acquiring the facial movement of the target to be sound-collected, and determining the sound-collecting device that the face of the target to be sound-collected is facing as the target sound-collecting device; Step S350, obtaining distances between the target phased area and a plurality of phased arrays in the target sound pickup device, to obtain a plurality of distances; Step S360, arranging the plurality of distances from small to large to obtain a second order; Step S370: According to the second order, determine the phased array corresponding to the smallest one of the multiple distances as the target phased array.
7. The method for controlling a pickup assembly as claimed in claim 6, wherein: The step S370 specifically includes: Step S371, according to the second order, determining the phased array corresponding to the smallest one of the plurality of distances as the target phased array; Step S372, obtaining an angle between a line between a target to be picked up and the target phased array and a central axis of the target phased array; Step S373: if the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is less than the first preset difference, or the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is less than the second preset difference, or the included angle is not within the preset sound pickup angle range, then in the second order, the second smallest one of the multiple distances is used as the new target phased array and steps S372 to S373 are repeatedly performed until the difference between the included angle and the upper limit of the preset sound pickup angle range of the target phased array is greater than the first preset difference, the difference between the included angle and the lower limit of the preset sound pickup angle range of the target phased array is greater than the second preset difference, and the included angle is within the preset sound pickup angle range; Step S374: determine the target phased array as the target phased array.
8. The method for controlling a pickup assembly as claimed in claim 7, wherein: The step of acquiring the distances between the target phased area and the plurality of phased arrays to obtain a plurality of distances, and determining the target phased array according to the order of the plurality of distances further includes: If the difference between the angles between the connecting line between any of the target to be sound picked up and the target phased array and the central axis of the target phased array is less than a first preset difference from the upper limit of the preset sound pickup angle range of the target phased array or less than a second preset difference from the lower limit of the preset sound pickup angle range of the target phased array, then the binaural connecting line of the target to be sound picked up is used as a reference, and the side to which the face of the target to be sound picked up faces is taken as the first side; The one of the plurality of sound pickup devices on the first side that is at the shortest distance from the target phase control area is determined as the target sound pickup device, and steps S350 to S374 are executed.
9. A control device, characterized in that: The control device comprises: memory; and, processor; A sound pickup assembly control program stored in the memory and executed by the processor, wherein when the sound pickup assembly control program is executed by the processor, the sound pickup assembly control method according to any one of claims 1 to 8 is implemented.
10. A sound pickup assembly, characterized in that: It comprises the control device as claimed in claim 9 and a plurality of phased arrays, each of the phased arrays is provided with at least two microphones arranged along a horizontal linear pattern, and the preset sound pickup area corresponding to the sound pickup assembly is divided into a plurality of phased areas.
Citation Information
Patent Citations
Artificial intelligence-based control method and control system for intelligent interaction equipment
CN105159111A
Microphone array pickup angle adjusting method and device, and electronic equipment
CN107491101A
Pickup sensitivity adjusting method and device
CN112015364A
Pickup structure, multimedia equipment and pickup method
CN113115171A
Directional pickup method and device based on human lip detection
CN115762553A
