Conference speaking user positioning system
By using multiple first microphone arrays in larger conference rooms to connect to the display device and combining the audio signals collected by the second microphone array, the problem of insufficient positioning accuracy in the prior art is solved, and accurate user positioning in larger conference rooms is achieved.
Patent Information
- Application Number
- CN202311387269.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-24
- Publication Date
- 2025-04-25
AI Technical Summary
The prior art cannot effectively locate the conference speaker user in larger conference rooms scenarios because the effective sound pickup distance of the microphone array is insufficient, resulting in low positioning accuracy.
A plurality of first microphone arrays are connected to the display device, and the distance of the first microphone array is greater than or equal to the effective sound pickup distance of the second microphone array. The first audio signal is collected through the second microphone array, and the second audio signal is collected through the first microphone array, and the positioning is carried out by combining the volume threshold.
It realizes accurate positioning of conference speaker users in larger conference room scenarios, improving the accuracy and applicability of positioning.
Smart Images
Figure CN119893382A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to speech positioning technology, and more specifically, to a conference speaking user positioning system. Background Art
[0002] With the development of technology, more and more people use video conferencing to hold meetings. In video conferencing, in order to highlight the conference speaker on the big screen, the conference speaker needs to be located.
[0003] In the prior art, the positioning of conference speakers is usually carried out in a small conference room scenario by the audio and video module in the display device. The microphone array in the audio and video module collects sound signals for processing, obtains the relative angle between the conference speaker and the real device, and completes the positioning.
[0004] However, in larger conference room scenarios, because the distance between some conference speaking users and the microphone exceeds the effective pickup distance of the microphone array, the positioning accuracy is low, so the existing positioning methods are not suitable for larger conference room scenarios. Therefore, there is an urgent need for a conference speaking user positioning system and method that can achieve positioning in larger conference room scenarios. Summary of the invention
[0005] The embodiment of the present application provides a conference speaking user positioning system, which is used to solve the problem that the positioning method in the prior art is not suitable for larger conference room scenarios. Therefore, there is an urgent need for a conference speaking user positioning system and method that can achieve positioning in larger conference room scenarios.
[0006] In a first aspect, an embodiment of the present application provides a conference speaking user positioning system, comprising:
[0007] A display device and a plurality of first microphone arrays; the display device includes an audio and video module, and the audio and video module includes a second microphone array;
[0008] The audio and video module is connected to the plurality of first microphone arrays;
[0009] The distance between the first microphone array and the display device is greater than or equal to the effective sound pickup distance of the second microphone array;
[0010] The audio and video module is configured as follows:
[0011] collecting a first audio signal through the second microphone array, and collecting a second audio signal through the first microphone array;
[0012] Determining whether the volume of the first audio signal is less than a first preset volume threshold;
[0013] If the volume of the first audio signal is less than the first preset volume threshold, adopting the first setting mode to determine the position data of the conference speaking user;
[0014] If the volume of the first audio signal is greater than or equal to the first preset volume threshold, the second setting mode is adopted to determine the position data of the conference speaking user;
[0015] The first setting mode determines the position data of the conference speaking user according to the first audio signal and the second audio signal, and the second setting mode determines the position data of the conference speaking user according to the first audio signal.
[0016] Beneficial effects of this embodiment: The conference speaking user positioning system includes a display device and multiple first microphone arrays, and the distance between the first microphone array and the display device is greater than the effective sound pickup distance of the second microphone array in the display device. After the audio and video module in the display device collects the first audio signal collected by the second microphone array and the second audio signal collected by the first microphone array, it is determined whether the volume of the first audio signal is less than the first preset volume threshold; if the volume of the first audio signal is less than the first preset volume threshold, the first setting mode is used according to the first audio signal and the second audio signal to determine the location data of the conference speaking user; if the volume of the first audio signal is greater than or equal to the first preset volume threshold, the second setting mode is used according to the first audio signal to determine the location data of the conference speaking user. This solution realizes positioning in a larger conference room scene through two microphone arrays and positioning according to the volume of the first audio signal.
[0017] In some embodiments of the present application, the audio and video module is configured to adopt the first setting mode, and when determining the location data of the conference speaking user, it is specifically configured as follows:
[0018] Determining a target angle range according to the volume of the first audio signal at a plurality of preset angles;
[0019] Determining, from the plurality of first microphone arrays, a candidate microphone array located within the target angle range according to preset position data of the plurality of first microphone arrays relative to the display device;
[0020] Determining a target microphone array from the microphone arrays to be selected according to the volume of the second audio signal corresponding to the microphone arrays to be selected;
[0021] Determining an intermediate angle of the conference speaking user relative to the target microphone array according to the volume of the second audio signal corresponding to the target microphone array at multiple preset angles;
[0022] Based on the preset position data of the target microphone array relative to the display device, the intermediate angle, and the preset position data of the user's fixed position relative to the target microphone array, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
[0023] The beneficial effects of this embodiment are as follows: when the volume of the first audio signal is less than the first preset volume threshold, the target angle range is determined through the first audio signal, and then the candidate microphone array within the target angle range is determined, and then the target microphone array is determined therefrom; then, based on the second audio signal corresponding to the target microphone array, the middle angle of the conference speaking user relative to the target microphone array is determined, and then combined with the preset position data of the target microphone array relative to the display device and the preset position data of the user's fixed position relative to the target microphone array, the angle of the conference speaking user relative to the display device is determined, thereby effectively improving the positioning accuracy.
[0024] In some embodiments of the present application, when the audio and video module is configured to determine the target angle range according to the volume of the first audio signal at multiple preset angles, it is specifically configured as follows:
[0025] Determining a target volume greater than a second preset volume threshold from volumes of the first audio signal at a plurality of preset angles, the second preset volume threshold being less than the first preset volume threshold;
[0026] The target angle range is generated according to a minimum angle and a maximum angle among the preset angles corresponding to the target volume.
[0027] The beneficial effect of this embodiment is that the target angle range is determined according to the volume of the first audio signal at multiple preset angles and the second preset volume threshold, which effectively improves the accuracy of the determined target angle range.
[0028] In some embodiments of the present application, the target microphone array is the microphone array corresponding to the second audio signal with the highest volume among the microphone arrays to be selected.
[0029] The beneficial effect of this embodiment is that the microphone array corresponding to the second audio signal with the highest volume in the candidate microphone arrays is used as the target microphone array, thereby improving the accuracy of the target microphone array.
[0030] In some embodiments of the present application, the audio and video module is configured to determine the angle of the conference speaking user relative to the display device based on the preset position data of the target microphone array relative to the display device, the intermediate angle, and the preset position data of the fixed position of the user relative to the target microphone array, and when obtaining the position data of the conference speaking user, is specifically configured as follows:
[0031] Determine the distance between the conference speaking user and the target microphone array according to the intermediate angle and the preset position data of the fixed position of the user relative to the target microphone array;
[0032] According to the preset position data of the target microphone array relative to the display device, the intermediate angle, and the distance between the conference speaking user and the target microphone array, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
[0033] The beneficial effects of this embodiment are as follows: by first determining the distance between the conference speaking user and the target microphone array, and then combining the preset position data and intermediate angle of the target microphone array relative to the display device, the position data of the conference speaking user is obtained, thereby improving the accuracy of the position data of the conference speaking user.
[0034] In some embodiments of the present application, the audio and video module is configured to adopt the second setting mode, and when determining the location data of the conference speaking user, it is specifically configured as follows:
[0035] According to the volume of the first audio signal at multiple preset angles, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
[0036] The beneficial effect of this embodiment is that when the volume of the first audio signal is greater than or equal to the first preset volume threshold, the volume of the first audio signal at multiple preset angles and the position data of the conference speaking user ensure the positioning accuracy at close range.
[0037] In some embodiments of the present application, the audio and video module is configured to determine the angle of the conference speaking user relative to the display device according to the volume of the first audio signal at multiple preset angles, and obtain the position data of the conference speaking user, and is specifically configured as follows:
[0038] The preset angle with the largest volume among the volumes of the first audio signal at multiple preset angles is used as the angle of the conference speaking user relative to the display device to obtain position data of the conference speaking user.
[0039] The beneficial effect of this embodiment is that the preset angle with the maximum volume is used as the angle of the conference speaking user relative to the display device, thereby ensuring the positioning accuracy at a close distance.
[0040] In some embodiments of the present application, the display device further includes a processor and a display screen, and the audio and video module further includes a wide-angle camera and a telephoto camera;
[0041] The audio and video module is also configured as follows:
[0042] Acquiring a conference image according to a result of determining whether the volume of the first audio signal is less than the first preset volume threshold;
[0043] Obtaining an image of the user who spoke at the conference according to the conference image and the location data of the user who spoke at the conference;
[0044] The processor is configured to control the display screen to display the image of the user who speaks at the conference.
[0045] The beneficial effects of this embodiment are as follows: the conference image is acquired by the volume of the first audio signal, and then the image of the user speaking at the conference is determined and then displayed, thereby highlighting the user speaking at the conference and improving the conference effect.
[0046] In some embodiments of the present application, when acquiring the conference image according to the judgment result of whether the volume of the first audio signal is less than the first preset volume threshold, it is specifically configured as follows:
[0047] If the volume of the first audio signal is less than the first preset volume threshold, acquiring the conference image through the telephoto camera;
[0048] If the volume of the first audio signal is greater than or equal to the first preset volume threshold, the conference image is acquired through the wide-angle camera.
[0049] The beneficial effects of this embodiment are as follows: when the volume of the first audio signal is less than the first preset volume threshold, the conference image is acquired through the telephoto camera; when the volume of the first audio signal is greater than or equal to the first preset volume threshold, the conference image is acquired through the wide-angle camera, thereby effectively improving the clarity of the conference speaking user in the conference image.
[0050] In some embodiments of the present application, when the audio and video module is configured to obtain the image of the conference speaking user according to the conference image and the location data of the conference speaking user, it is specifically configured as follows:
[0051] An image of a position corresponding to the position data of the user who spoke at the conference is captured from the conference image to obtain an image of the user who spoke at the conference.
[0052] The beneficial effect of this embodiment is as follows: by intercepting the image of the position corresponding to the position data of the conference speaking user, the image of the conference speaking user is obtained, thereby highlighting the conference speaking user and the conference effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.
[0054] Figure 1 Schematic diagram of the small conference room scenario provided for this application;
[0055] Figure 2 A schematic diagram of the architecture of the conference speech user positioning system provided for this application;
[0056] Figure 3 Schematic diagram of the application scenario of the conference speaking user positioning system provided for this application;
[0057] Figure 4 A flowchart of the method for locating conference speaking users provided in this application;
[0058] Figure 5 A schematic diagram of the preset angles provided for this application;
[0059] Figure 6 A schematic diagram of the process of determining the location data of the user speaking at the conference provided by this application;
[0060] Figure 7 A schematic diagram of the relative positions of the first microphone array and the display device provided in this application;
[0061] Figure 8 A schematic diagram for determining the angle of a conference speaking user relative to a display device provided by the present application;
[0062] Fig. 9 A schematic diagram of the process of displaying the image of the user speaking at the conference provided by this application;
[0063] Fig.10 A schematic diagram of the conference image division provided for this application;
[0064] Fig.11 This is a structural diagram of an embodiment of a conference speaking user positioning device provided in this application. DETAILED DESCRIPTION
[0065] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.
[0066] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.
[0067] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein.
[0068] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.
[0069] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0070] With the development of technology, more and more people use video conferencing to hold meetings. In video conferencing, in order to highlight the conference speaker on the big screen, the conference speaker needs to be located.
[0071] In the prior art, the positioning of conference speakers is usually carried out in a small conference room scenario by the audio and video module in the display device. The microphone array in the audio and video module collects sound signals for processing, obtains the relative angle between the conference speaker and the real device, and completes the positioning.
[0072] For example, Figure 1 A schematic diagram of a small conference room scenario provided for this application, such as Figure 1 As shown, in a small conference room scenario, the display device includes a display screen 11 and an audio and video module 12, and a conference user 13 sits in front of the display device for a video conference. When a user speaks, the audio and video module 12 performs positioning according to the acquired audio signal.
[0073] However, in larger conference room scenarios, because the distance between some conference speaking users and the microphone may exceed the effective pickup distance of the microphone array, the existing positioning methods are not suitable for larger conference room scenarios. Therefore, there is an urgent need for a conference speaking user positioning system and method that can achieve positioning in larger conference room scenarios.
[0074] In response to the problems existing in the prior art, the inventors discovered during the study of the conference speaking user positioning method that in order to be able to achieve positioning in a larger conference room scenario, it can be achieved through a conference speaking user positioning system. The conference speaking user positioning system includes a display device and multiple first microphone arrays. The distance between the first microphone array and the display device is greater than or equal to the effective pickup distance of the second microphone array in the audio and video module in the display device.
[0075] The audio and video module determines to use different setting modes for positioning based on the volume of the first audio signal collected by the second microphone array. When the volume is low, it means that the speaking user is far away from the display device, and positioning is performed based on the first audio signal and the second audio signal collected by the first microphone array.
[0076] When the volume is high, it means that the speaking user is close to the display device, and the user is positioned according to the first audio signal. The positioning of the user who speaks at the conference can be realized in a large conference room. Based on the above invention concept, the conference speaking user positioning solution in this application is designed.
[0077] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0078] For example, Figure 2 The schematic diagram of the architecture of the conference speech user positioning system provided for this application is as follows: Figure 2 As shown, the conference speaking user positioning system includes a display device 21 and a plurality of microphone arrays 22. Each microphone array 22 is composed of a plurality of microphones.
[0079] The display device 21 includes an audio and video module 23, a processor 24 and a display screen 25. The audio and video module 23 is connected to the processor 24 and each first microphone array 22 respectively.
[0080] The audio and video module 23 includes a second microphone array, a wide-angle camera 26 and a telephoto camera 27 , and the second microphone array is composed of multiple microphones 28 .
[0081] The distance between the first microphone array 22 and the display device 21 is greater than or equal to the effective sound pickup distance of the second microphone array.
[0082] It should be noted that the connection method between each first microphone array 22 and the audio and video module 23 can be a wired connection or a wireless connection. The embodiment of the present application does not limit the connection method between the first microphone array 22 and the audio and video module 23, and can be set according to actual conditions.
[0083] For example, Figure 3 The application scenario diagram of the conference speech user positioning system provided for this application is as follows: Figure 3 As shown, the conference speaking user positioning system is applied in a larger conference room scenario, and the first microphone array 22 is installed in the conference room at a distance from the display device, so that the distance between the first microphone array 22 and the display device is greater than or equal to the effective pickup distance of the second microphone array.
[0084] When the user speaks, the audio and video module 23 collects audio signals according to the first microphone array and the second microphone array 22, and then performs positioning according to the audio signals.
[0085] The audio and video module 23 captures images through a wide-angle camera or a telephoto camera according to the positioning data, and then intercepts and obtains the image of the user speaking at the conference and sends it to the processor, which controls the display screen 25 to display the image.
[0086] Figure 4 This is a flow chart of the conference speaking user positioning method provided in this application. The executor of the method is a display device. The method in this embodiment can be implemented by software, hardware, or a combination of software and hardware.
[0087] like Figure 4 As shown, the audio and video module in the display device is configured to perform the following steps:
[0088] S401: Collect a first audio signal through the second microphone array, and collect a second audio signal through the first microphone array.
[0089] After the display device is started, the user conducts a video conference through the display device, the audio and video module in the display device locates the user speaking in the conference, the audio and video module collects the first audio signal through the second microphone array, and collects the second audio signal through the first microphone array.
[0090] S402: Determine whether the volume of the first audio signal is less than a first preset volume threshold. If the volume of the first audio signal is less than the first preset volume threshold, execute step S403; if the volume of the first audio signal is greater than or equal to the first preset volume threshold, execute step S404.
[0091] After the audio and video module collects the first audio signal, it determines whether the volume of the first audio signal is less than a first preset volume threshold to determine the distance between the conference speaking user and the display device.
[0092] It should be noted that the first preset volume threshold may be 40 decibels, 50 decibels, 70 decibels, etc. The embodiment of the present application does not limit the first preset volume threshold and may be set according to actual conditions.
[0093] It should be noted that before the audio and video module makes a judgment, it can also detect the first audio signal to determine whether human voice is collected. When it is determined that human voice is collected, it is determined whether the volume of the first audio signal is less than the first preset volume threshold; when it is determined that human voice is not collected, the judgment process is not performed and the first audio signal continues to be collected.
[0094] S403: Adopting the first setting mode, determining the location data of the user who speaks at the conference.
[0095] When the volume of the first audio signal is less than the first preset volume threshold, it means that the distance between the conference speaking user and the display device is far. For more accurate positioning, the audio and video module adopts the first setting mode to determine the position data of the conference speaking user based on the first audio signal and the second audio signal.
[0096] S404: Adopting the second setting mode, determining the location data of the user who speaks at the conference.
[0097] When the volume of the first audio signal is greater than or equal to the first preset volume threshold, it indicates that the distance between the conference speaking user and the display device is relatively close. The audio and video module adopts the second setting mode to determine the position data of the conference speaking user according to the first audio signal.
[0098] Specifically, according to the volume of the first audio signal at multiple preset angles, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
[0099] That is, the preset angle at which the volume of the first audio signal at a plurality of preset angles is the largest is used as the angle of the conference speaking user relative to the display device to obtain the position data of the conference speaking user.
[0100] For example, Figure 5 The preset angle diagram provided for this application is as follows: Figure 5 As shown, within the 180° range in front of the display device, a preset angle is set at intervals of 5°. The preset angles can also be set at intervals of 1°, 10°, etc. The preset angles are not limited in the present embodiment and can be set according to actual conditions.
[0101] The conference speaking user positioning system provided in this embodiment includes a display device and multiple first microphone arrays, and the distance between the first microphone array and the display device is greater than the effective sound pickup distance of the second microphone array in the display device. When the volume of the first audio signal collected by the second microphone array in the display device is less than the first preset volume threshold, the audio and video module in the display device uses the first setting mode to determine the position data of the conference speaking user based on the first audio signal and the second audio signal collected by the first microphone array; when the volume of the first audio signal is greater than or equal to the first preset volume threshold, the second setting mode is used to determine the position data based on the first audio signal. This solution uses two microphone arrays and positioning based on the volume of the first audio signal to achieve positioning in a larger conference room scene.
[0102] Furthermore, based on the above-mentioned embodiments, the embodiments of the present application illustrate the situation where the audio and video module adopts the first setting mode to determine the location data of the user who speaks at the conference. Figure 6 A schematic diagram of a process for determining the location data of a conference speaking user provided in this application, wherein the audio and video module in the display device is configured to perform the following steps:
[0103] S601: Determine a target angle range according to the volume of a first audio signal at multiple preset angles.
[0104] When the volume of the first audio signal is less than the first preset volume threshold, the first setting mode is adopted to determine the position data of the conference speaking user. The audio and video module determines the target angle range according to the volume of the first audio signal at multiple preset angles.
[0105] Specifically, the audio and video module determines a target volume greater than a second preset volume threshold from the volumes of the first audio signal at multiple preset angles.
[0106] It should be noted that the second preset volume threshold is smaller than the first preset volume threshold. The second preset volume threshold may be 20 decibels, 30 decibels, 50 decibels, etc. The embodiment of the present application does not limit the second preset volume threshold and may be set according to actual conditions.
[0107] The audio and video module generates a target angle range according to the minimum angle and the maximum angle among the preset angles corresponding to the target volume.
[0108] Exemplarily, the target volume is 45 decibels, 50 decibels, 52 decibels, 57 decibels, and 60 decibels, the angle corresponding to 45 decibels is 30°, the angle corresponding to 45 decibels is 45°, the angle corresponding to 50 decibels is 32°, the angle corresponding to 52 decibels is 35°, the angle corresponding to 57 decibels is 40°, and the angle corresponding to 60 decibels is 37°.
[0109] The minimum angle of the preset angles corresponding to the target volume is 30°, and the maximum angle is 45°, so 30°-45° is used as the target angle range.
[0110] S602: Determine, from the plurality of first microphone arrays, a candidate microphone array located within a target angle range according to preset position data of the plurality of first microphone arrays relative to the display device.
[0111] After the audio and video module determines the target angle range, since the audio and video module stores the preset position data of each first microphone array relative to the display device, including the angle of the first microphone array relative to the display device, the selected microphone array within the target angle range can be determined from multiple first microphone arrays based on the preset position data.
[0112] For example, Figure 7 A schematic diagram of the relative positions of the first microphone array and the display device provided in this application, such as Figure 7 As shown, the distance between the first microphone array and the display device is greater than or equal to the effective sound pickup distance of the second microphone array.
[0113] In the figure, 30°-45° is used as the target angle range, and the black first microphone array represents the microphone array to be selected.
[0114] S603: Determine a target microphone array from the microphone arrays to be selected according to the volume of the second audio signal corresponding to the microphone arrays to be selected.
[0115] After the audio and video module determines the microphone arrays to be selected, it is also necessary to determine the target microphone array closest to the conference speaker. The target microphone array is determined from the microphone arrays to be selected according to the volume of the second audio signal corresponding to the microphone arrays to be selected.
[0116] The microphone array corresponding to the second audio signal with the loudest volume among the microphone arrays to be selected is used as the target microphone array.
[0117] It should be noted that, if there are multiple microphone arrays corresponding to the second audio signal with the highest volume among the microphone arrays to be selected, one microphone array may be randomly selected as the target microphone array.
[0118] S604: Determine an intermediate angle of the conference speaking user relative to the target microphone array according to the volume of the second audio signal corresponding to the target microphone array at multiple preset angles.
[0119] After the audio and video module determines the target microphone array, the preset angle with the largest volume among the volumes of the second audio signal corresponding to the target microphone array at multiple preset angles is used as the middle angle of the conference speaking user relative to the target microphone array.
[0120] S605: Determine the angle of the conference speaking user relative to the display device based on the preset position data of the target microphone array relative to the display device, the intermediate angle, and the preset position data of the user's fixed position relative to the target microphone array, and obtain the position data of the conference speaking user.
[0121] After the audio and video module obtains the middle angle of the conference speaking user relative to the target microphone array, it determines the distance between the conference speaking user and the target microphone array based on the middle angle and the preset position data of the user's fixed position relative to the target microphone array.
[0122] Because after the first microphone array is installed, the position of each first microphone array is fixed, and the position of the conference user is also fixed, the preset position data of the user's fixed position relative to the first microphone array can be determined and stored in the audio and video module.
[0123] The preset position data of the user's fixed position relative to the first microphone array includes the angle and distance of the user's fixed position relative to the first microphone array. Therefore, in the preset position data of the user's fixed position relative to the target microphone array, the preset position data including the intermediate angle can be found, and then the distance therein can be found to obtain the distance between the conference speaking user and the target microphone array.
[0124] According to the preset position data of the target microphone array relative to the display device, the intermediate angle, and the distance between the conference speaking user and the target microphone array, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
[0125] For example, Figure 8 A schematic diagram for determining the angle of the conference speaking user relative to the display device provided by this application, such as Figure 8 As shown, the angle of the target microphone array relative to the display device is α, the angle of the conference speaking user relative to the display device is β, and combined with the distance m between the conference speaking user and the target microphone array, the angle γ of the conference speaking user relative to the display device can be obtained.
[0126] The conference speaking user positioning system provided in this embodiment adopts a first setting mode to determine the target microphone array closest to the conference speaking user when the volume of the first audio signal is less than a first preset volume threshold, and then determines the middle angle of the conference speaking user relative to the target microphone array. Then, the position data of the conference speaking user is determined by combining the preset position data of the target microphone array relative to the shown device and the preset position data of the user's fixed position relative to the target microphone array, thereby improving the accuracy of the position data of the conference speaking user.
[0127] Furthermore, based on the above-mentioned embodiments, the embodiments of the present application illustrate the situation in which the audio and video module determines the image of the user who speaks at the conference and the display screen displays the image. Fig. 9 A schematic diagram of a process for displaying an image of a user speaking at a conference provided by the present application, wherein the audio and video module in the display device is configured to perform the following steps:
[0128] S901: Acquire a conference image according to a result of determining whether the volume of a first audio signal is less than a first preset volume threshold.
[0129] After the audio and video module obtains the location data of the user speaking at the conference, it can also highlight the image of the user speaking at the conference to improve the conference effect.
[0130] The conference image is acquired according to a determination result of whether the volume of the first audio signal is less than a first preset volume threshold.
[0131] When the volume of the first audio signal is less than the first preset volume threshold, it means that the conference speaking user is far away from the display device, and the image taken by the telephoto lens in the audio and video module at a farther distance is clearer, so the conference image is obtained through the telephoto camera.
[0132] When the volume of the first audio signal is greater than or equal to the first preset volume threshold, it means that the conference speaking user is close to the display device, and the image taken by the wide-angle lens in the audio and video module at a closer distance is clearer, so the conference image is obtained through the wide-angle camera.
[0133] S902: Obtain an image of the user who spoke at the conference based on the conference image and the location data of the user who spoke at the conference.
[0134] After the audio and video module obtains the conference image, it obtains the image of the conference speaking user based on the conference image and the location data of the conference speaking user.
[0135] The conference image can be divided into multiple images, each image corresponding to the angle of the conference speaking user relative to the display device.
[0136] For example, Fig.10The conference image division diagram provided for this application is as follows: Fig.10 As shown, the conference image is divided into 18 parts, from left to right, the first part corresponds to 10°, the second part corresponds to 20°, and so on.
[0137] Therefore, an image of the position corresponding to the position data of the conference speaking user is captured from the conference image, that is, an image of the position corresponding to the angle of the conference speaking user relative to the display device, to obtain the image of the conference speaking user.
[0138] The processor in the display device is configured to control the display screen in the display device to display the image of the user who speaks at the conference.
[0139] The conference speaker positioning system provided in this embodiment obtains the position data of the conference speaker, and the audio and video module obtains the conference image through the telephoto camera or the wide-angle camera, and then determines the image of the conference speaker according to the position data of the conference speaker. Then, the processor in the display device controls the display screen to display the image of the conference speaker, so as to highlight the conference speaker, improve the clarity of the image, and improve the conference effect.
[0140] In this application, a method for locating a conference speaking user is also provided, and the steps may be:
[0141] The first audio signal is collected through the second microphone array, and the second audio signal is collected through the first microphone array.
[0142] It is determined whether the volume of the first audio signal is less than a first preset volume threshold.
[0143] If the volume of the first audio signal is less than the first preset volume threshold, the first setting mode is adopted to determine the position data of the conference speaking user.
[0144] If the volume of the first audio signal is greater than or equal to the first preset volume threshold, the second setting mode is adopted to determine the position data of the conference speaking user.
[0145] For specific implementation steps and technical effects, please refer to the above embodiments, and this application will not repeat them.
[0146] Fig.11 A schematic diagram of the structure of an embodiment of a conference speaking user positioning device provided in this application; Fig.11 As shown, the conference speaking user positioning device 1100 includes:
[0147] An acquisition module 1101 is configured to collect a first audio signal through the second microphone array, and collect a second audio signal through the first microphone array;
[0148] The processing module 1102 is used to:
[0149] Determining whether the volume of the first audio signal is less than a first preset volume threshold;
[0150] If the volume of the first audio signal is less than the first preset volume threshold, adopting the first setting mode to determine the position data of the conference speaking user;
[0151] If the volume of the first audio signal is greater than or equal to the first preset volume threshold, the second setting mode is adopted to determine the position data of the conference speaking user;
[0152] The first setting mode determines the position data of the conference speaking user according to the first audio signal and the second audio signal, and the second setting mode determines the position data of the conference speaking user according to the first audio signal.
[0153] Furthermore, the processing module 1102 is specifically configured to:
[0154] Determining a target angle range according to the volume of the first audio signal at a plurality of preset angles;
[0155] Determining, from the plurality of first microphone arrays, a candidate microphone array located within the target angle range according to preset position data of the plurality of first microphone arrays relative to the display device;
[0156] Determining a target microphone array from the microphone arrays to be selected according to the volume of the second audio signal corresponding to the microphone arrays to be selected;
[0157] Determining an intermediate angle of the conference speaking user relative to the target microphone array according to the volume of the second audio signal corresponding to the target microphone array at multiple preset angles;
[0158] Based on the preset position data of the target microphone array relative to the display device, the intermediate angle, and the preset position data of the user's fixed position relative to the target microphone array, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
[0159] Furthermore, the processing module 1102 is specifically configured to:
[0160] Determining a target volume greater than a second preset volume threshold from volumes of the first audio signal at a plurality of preset angles, the second preset volume threshold being less than the first preset volume threshold;
[0161] The target angle range is generated according to a minimum angle and a maximum angle among the preset angles corresponding to the target volume.
[0162] Furthermore, the target microphone array is a microphone array corresponding to the second audio signal with the highest volume among the microphone arrays to be selected.
[0163] Furthermore, the processing module 1102 is specifically configured to:
[0164] Determine the distance between the conference speaking user and the target microphone array according to the intermediate angle and the preset position data of the fixed position of the user relative to the target microphone array;
[0165] According to the preset position data of the target microphone array relative to the display device, the intermediate angle, and the distance between the conference speaking user and the target microphone array, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
[0166] Furthermore, the processing module 1102 is specifically configured to:
[0167] According to the volume of the first audio signal at multiple preset angles, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
[0168] Furthermore, the processing module 1102 is specifically configured to:
[0169] The preset angle with the largest volume among the volumes of the first audio signal at multiple preset angles is used as the angle of the conference speaking user relative to the display device to obtain position data of the conference speaking user.
[0170] Furthermore, the acquisition module 1101 is further configured to acquire the conference image according to a determination result of whether the volume of the first audio signal is less than the first preset volume threshold;
[0171] Furthermore, the processing module 1102 is further configured to obtain an image of the conference speaking user according to the conference image and the location data of the conference speaking user;
[0172] The display module 1103 is used to display the image of the user who speaks at the conference.
[0173] Furthermore, the acquisition module 1101 is specifically used for:
[0174] If the volume of the first audio signal is less than the first preset volume threshold, acquiring the conference image through the telephoto camera;
[0175] If the volume of the first audio signal is greater than or equal to the first preset volume threshold, the conference image is acquired through the wide-angle camera.
[0176] Furthermore, the processing module 1102 is specifically configured to capture an image of a position corresponding to the position data of the conference speaking user from the conference image, to obtain an image of the conference speaking user.
[0177] The conference speaking user positioning device provided in this embodiment is used to implement the technical solution in any of the above embodiments. Its implementation principle and technical effects are similar and will not be described in detail here.
[0178] An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the technical solution provided in any of the aforementioned embodiments is implemented.
[0179] An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement the technical solution provided by any of the aforementioned embodiments when executed by a processor.
[0180] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
[0182] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.
Claims
1. A conference speaking user positioning system, characterized in that: include: a display device and a plurality of first microphone arrays; The display device includes an audio and video module, and the audio and video module includes a second microphone array; The audio and video module is connected to the plurality of first microphone arrays; The distance between the first microphone array and the display device is greater than or equal to the effective sound pickup distance of the second microphone array; The audio and video module is configured as follows: collecting a first audio signal through the second microphone array, and collecting a second audio signal through the first microphone array; Determining whether the volume of the first audio signal is less than a first preset volume threshold; If the volume of the first audio signal is less than the first preset volume threshold, adopting the first setting mode to determine the position data of the conference speaking user; If the volume of the first audio signal is greater than or equal to the first preset volume threshold, the second setting mode is adopted to determine the position data of the conference speaking user; The first setting mode determines the position data of the conference speaking user according to the first audio signal and the second audio signal, and the second setting mode determines the position data of the conference speaking user according to the first audio signal.
2. The conference speaking user positioning system according to claim 1, characterized in that: The audio and video module is configured to adopt the first setting mode, and when determining the location data of the conference speaking user, it is specifically configured as follows: Determining a target angle range according to the volume of the first audio signal at a plurality of preset angles; Determining, from the plurality of first microphone arrays, a candidate microphone array located within the target angle range according to preset position data of the plurality of first microphone arrays relative to the display device; Determining a target microphone array from the microphone arrays to be selected according to the volume of the second audio signal corresponding to the microphone arrays to be selected; Determining an intermediate angle of the conference speaking user relative to the target microphone array according to the volume of the second audio signal corresponding to the target microphone array at multiple preset angles; Based on the preset position data of the target microphone array relative to the display device, the intermediate angle, and the preset position data of the user's fixed position relative to the target microphone array, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
3. The conference speaking user positioning system according to claim 2, characterized in that: When the audio and video module is configured to determine the target angle range according to the volume of the first audio signal at multiple preset angles, it is specifically configured as follows: Determining a target volume greater than a second preset volume threshold from volumes of the first audio signal at a plurality of preset angles, the second preset volume threshold being less than the first preset volume threshold; The target angle range is generated according to a minimum angle and a maximum angle among the preset angles corresponding to the target volume.
4. The conference speaking user positioning system according to claim 2, characterized in that: The target microphone array is a microphone array corresponding to the second audio signal with the highest volume among the microphone arrays to be selected.
5. The conference speaking user positioning system according to claim 2, characterized in that: The audio and video module is configured to determine the angle of the conference speaking user relative to the display device according to the preset position data of the target microphone array relative to the display device, the intermediate angle, and the preset position data of the fixed position of the user relative to the target microphone array, and when obtaining the position data of the conference speaking user, is specifically configured as follows: Determine the distance between the conference speaking user and the target microphone array according to the intermediate angle and the preset position data of the fixed position of the user relative to the target microphone array; According to the preset position data of the target microphone array relative to the display device, the intermediate angle, and the distance between the conference speaking user and the target microphone array, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
6. The conference speaking user positioning system according to claim 1, characterized in that: The audio and video module is configured to adopt the second setting mode, and when determining the location data of the conference speaking user, it is specifically configured as follows: According to the volume of the first audio signal at multiple preset angles, the angle of the conference speaking user relative to the display device is determined to obtain the position data of the conference speaking user.
7. The conference speaking user positioning system according to claim 6, characterized in that: The audio and video module is configured to determine the angle of the conference speaking user relative to the display device according to the volume of the first audio signal at multiple preset angles, and obtain the position data of the conference speaking user, and is specifically configured as follows: The preset angle with the largest volume among the volumes of the first audio signal at multiple preset angles is used as the angle of the conference speaking user relative to the display device to obtain position data of the conference speaking user.
8. The conference speaking user positioning system according to claim 1, characterized in that: The display device further includes a processor and a display screen, and the audio and video module further includes a wide-angle camera and a telephoto camera; The audio and video module is also configured as follows: Acquiring a conference image according to a result of determining whether the volume of the first audio signal is less than the first preset volume threshold; Obtaining an image of the user who spoke at the conference according to the conference image and the location data of the user who spoke at the conference; The processor is configured to control the display screen to display the image of the user who speaks at the conference.
9. The conference speaking user positioning system according to claim 8, characterized in that: When acquiring the conference image according to the judgment result of whether the volume of the first audio signal is less than the first preset volume threshold, the specific configuration is as follows: If the volume of the first audio signal is less than the first preset volume threshold, acquiring the conference image through the telephoto camera; If the volume of the first audio signal is greater than or equal to the first preset volume threshold, the conference image is acquired through the wide-angle camera.
10. The conference speaking user positioning system according to claim 8, characterized in that: When the audio and video module is configured to obtain the image of the conference speaking user according to the conference image and the position data of the conference speaking user, the audio and video module is specifically configured as follows: An image of a position corresponding to the position data of the user who spoke at the conference is captured from the conference image to obtain an image of the user who spoke at the conference.