Audio processing method, related device, equipment and computer storage medium

Abstract

The invention discloses an audio processing method, a related device, equipment and a computer storage medium. According to the method, a plurality of sound source acquisition elements are arranged in an audio acquisition device to acquire sound information at different positions in a target scene, so that an audio playing device distinguishes and identifies sound information of a target spokesman and other conference participants based on relative distances between the conference participants and the sound source acquisition elements; and the sound information of the target spokesman is obtained. Therefore, according to the embodiment of the invention, the voice information of the target spokesman can be determined according to the relative distance to obtain clear and accurate single-person voice information, it is ensured that useful audio information of a network conference is clear and is not interfered by sounds of other conference participants, and the problem that in the related technology, sounds of other people and other noises are possibly collected is effectively solved.

Description

Technical Field

[0001] The present application relates to the field of signal processing technology, and in particular to an audio processing method, related devices, equipment and computer storage medium. Background Art

[0002] At this stage, online meetings have become the mainstream mode. Online meetings can collect voice information of participants through conference audio equipment and interact through the Internet.

[0003] When there are multiple people attending a meeting in an offline conference room, the conference audio equipment will not only collect the speaker's effective voice information, but also the voices of other participants and environmental sounds, which may cause noise in the online conference room. Summary of the invention

[0004] The embodiments of the present application provide an audio processing method, related devices, equipment and computer storage medium, which can obtain clear and accurate single-person voice information, ensuring that useful audio information of a network conference is clear without being disturbed by the voices of other participants.

[0005] In a first aspect, an embodiment of the present application provides an audio processing method, which is used for the audio playback device, and the audio playback device communicates with an audio acquisition device; the method includes:

[0006] Receiving first sound information in a target scene during a meeting sent by an audio collection device; wherein the meeting period refers to a time period corresponding to a meeting in which multiple participants participate, and the first sound information is obtained through multiple sound source collection elements located at different positions in the audio collection device;

[0007] Determining the voice information of the participants in the target scene based on the first voice information in the target scene during the meeting;

[0008] Based on the voice information of the participants and the relative distances between each participant and each sound source collecting element, the voice information of a target speaker among the multiple participants is obtained.

[0009] In a second aspect, an embodiment of the present application provides an audio processing method, which is used for an audio acquisition device, wherein the audio acquisition device includes a plurality of sound source acquisition elements; each sound source acquisition element is located at a different position in the audio acquisition device; the audio acquisition device communicates with an audio playback device, and the method includes:

[0010] Acquire first sound information in a target scene during a meeting; wherein the meeting period represents a time period corresponding to a meeting in which multiple participants participate, and the first sound information is obtained through multiple sound source collection elements;

[0011] The first sound information is sent to the audio playback device, so that the audio playback device can obtain the sound information of the target speaker among the multiple participants based on the first sound information and the relative distances between each participant and each sound source collection element.

[0012] In a third aspect, an embodiment of the present application provides an audio processing device, which is used for the audio playback device, and the audio playback device communicates with the audio acquisition device; the device includes:

[0013] A first sound information receiving module is used to receive first sound information in a target scene during a meeting sent by an audio collection device; wherein the meeting period represents a time period corresponding to a meeting in which multiple participants participate, and the first sound information and the relative distance are obtained by multiple sound source collection elements arranged at different positions in the audio collection device;

[0014] A participant voice information determination module, used to determine the voice information of the participant in the target scene based on the first voice information in the target scene during the meeting;

[0015] The target speaker sound information obtaining module is used to obtain the sound information of the target speaker among multiple participants based on the sound information of the participants and the relative distances between each participant and each sound source collecting element.

[0016] In a fourth aspect, an embodiment of the present application provides an audio processing device, which is used for an audio acquisition device, wherein the audio acquisition device includes a plurality of sound source acquisition elements; each sound source acquisition element is located at a different position in the audio acquisition device; the audio acquisition device communicates with an audio playback device, and the device includes:

[0017] A first sound information acquisition module is used to acquire first sound information in a target scene during a meeting; wherein the relative distance is obtained through multiple sound source acquisition elements;

[0018] The first sound information sending module is used to send the first sound information to the audio playback device, so that the audio playback device can obtain the sound information of the target speaker among multiple participants based on the first sound information and the relative distance between each participant and each sound source collection element.

[0019] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0020] The embodiment of the present application can collect sound information at different positions in the target scene by setting multiple sound source collection elements in the audio collection device, so that the audio playback device can obtain the sound information of the target speaker based on the relative distance between each participant and each sound source collection element. Therefore, the embodiment of the present application can determine the voice information of the target speaker according to the relative distance to obtain clear and accurate single-person voice information, ensure that the useful audio information of the network conference is clear without being interfered by the voices of other participants, and effectively solve the noise problem such as the possible collection of other people's voices in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 Application scenarios of conference equipment in related technologies;

[0023] Figure 2 A system architecture diagram of an audio processing method provided in an embodiment of the present application;

[0024] Figure 3A A flowchart of an audio processing method provided in an embodiment of the present application;

[0025] Figure 3B An application scenario diagram of another audio processing method provided in an embodiment of the present application;

[0026] Figure 3C An application scenario diagram of another audio processing method provided in an embodiment of the present application;

[0027] Figure 3D An application scenario diagram of another audio processing method provided in an embodiment of the present application;

[0028] Figure 4A A flowchart of another audio processing method provided in an embodiment of the present application;

[0029] Figure 4B An application scenario diagram of another audio processing method provided in an embodiment of the present application;

[0030] Figure 4C A schematic diagram of a module structure corresponding to another audio processing method provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of the structure of another audio processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of systems and methods consistent with some aspects of the present application as detailed in the attached claims.

[0034] In the description of the present application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0035] Figure 1 The example shows the application scenario of conference equipment in the related technology. Among them, the conference site is located at location 1, and the online conferences in locations 2 to 5 can be connected to conference equipment (for example, conference tablets, conference audio, etc.) through the network to receive or send conference-related information (for example, audio information, image information, file information, etc.).

[0036] In the current conference scenario, in order to ensure that all participants in the meeting can clearly hear the important information of the current speaker, the only way is to turn off the microphones of non-speakers to prevent the input of invalid noise. However, during the meeting, such operations can easily cause the voices of participants to be missed, important audio information may not be synchronized and thus be missed, and manually turning the audio on or off may affect the overall efficiency of the meeting. In addition, in general, audio transceiver equipment can collect low, medium, and high-order audio information before leaving the factory to ensure normal system frequency response. Therefore, the user end can no longer adjust the audio response later, that is, it cannot filter out related noise information.

[0037] Figure 2The system architecture diagram of the audio processing method of the embodiment of the present application is shown. The audio processing method provided by the embodiment of the present application can be applied to multiple conference devices. Specifically, conference device 1 can send conference related information to conference devices 2-4 through the network. Each conference device can include an audio playback device 21 and an audio acquisition device 22. The audio acquisition device can include multiple sound source acquisition elements, and the sound source acquisition elements are located at different positions in the audio acquisition device ( Figure 2 The audio acquisition equipment includes, but is not limited to, computer microphones, lavalier microphones, condenser microphones, wireless microphones, wired microphones, handheld microphones, gooseneck microphones, interface microphones, headphones, speakers, etc.

[0038] Specifically, the audio playback device includes, but is not limited to, a touch screen, an all-in-one machine, a personal computer, a tablet computer, a vehicle-mounted device, a smart phone, a computing device, or other processing devices connected to a wireless modem. Display devices may be called different names in different networks, such as user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), terminal in the fifth generation mobile communication technology (5th generation mobile networks, 5G) network or future evolution network, etc. The terminal system refers to an operating system that can be run on the terminal, which is a program for managing and controlling terminal hardware and terminal applications, and is an indispensable system application for the terminal. The system includes but is not limited to Android system, IOS system, Windows phone (WP) system and Ubuntu mobile version operating system, etc.

[0039] It should be understood that Figure 2 The number of conference devices and network servers in the example is only for illustration. Any number of conference devices and servers may be provided according to actual needs. For example, a server may be a server cluster composed of multiple servers. Users may use the server to interact with the conference device through the server to obtain an optimized version, etc.

[0040] Next, combine Figure 1 Application scenarios and Figure 2 The system architecture diagram of the audio processing method of the embodiment of the present application is shown to introduce the audio processing method provided by the embodiment of the present application.

[0041] In one embodiment, Figure 3A A flowchart of an audio processing method is provided as shown in FIG. Figure 3AThe audio processing method shown may include the following steps:

[0042] S301: An audio acquisition device obtains first sound information in a target scene during a conference.

[0043] Among them, the meeting period refers to the time period corresponding to the meeting participated by multiple participants, for example, the pre-agreed time period for holding an online meeting. For example, you can use relevant software such as conference applications to pre-set an online discussion meeting from 9:00 to 11:00 a.m. on Tuesday and invite relevant personnel to participate in the meeting. Then, Tuesday morning from 9:00 to 11:00 is the meeting period, and the relevant personnel participating in the meeting are the participants.

[0044] Specifically, the target scene in the embodiment of the present application represents any spatial scene using conference equipment, such as conference rooms, conversation rooms, compartments, offices, classrooms, computer rooms, study rooms, living rooms, etc. The first sound information represents the sound information in the target scene received by the audio collection device during the meeting, and the first sound information includes but is not limited to: human voices (for example, the voices of participants, human voices outdoors, etc.), the sound of drinking water, the sound of typing, the sound of weather (for example, rain, thunder, wind, etc.), etc.

[0045] S302: The audio collection device sends first sound information to the audio playback device.

[0046] Possibly, the audio acquisition device may send the first sound information to the audio playback device via a transmission method such as Bluetooth or a wireless network.

[0047] Possibly, the audio playback device may also receive the first sound information sent by the audio acquisition device via a transmission method such as Bluetooth or a wireless network.

[0048] S303: The audio playback device determines the sound information of the participants in the target scene based on the first sound information in the target scene during the meeting.

[0049] Possibly, the embodiment of the present application can filter the first sound information in the target scene during the meeting to obtain the meeting sound information in the target scene; based on the meeting sound information in the target scene, determine the sound information of the participants in the target scene.

[0050] It can be understood that the embodiment of the present application can perform primary filtering on the first sound information collected by multiple sound source collection elements to filter out obvious noise information in the first sound information, such as typing, drinking water, coughing, noisy human voices outdoors, etc., to obtain the sound information of the participants in the target scene.

[0051] S304: The audio playback device obtains the voice information of a target speaker among the multiple participants based on the voice information of the participants and the relative distances between each participant and each sound source collection element.

[0052] Specifically, the relative position between the audio collection device and the target speaker remains unchanged. For example, a wearable microphone is generally worn on the target speaker's clothes, and a movable microphone is generally held by the target speaker or located near the target speaker. Therefore, the relative position between each sound source collection element in the audio collection device and the target speaker is also relatively fixed.

[0053] Furthermore, the multiple sound source collection elements in the embodiment of the present application may include: a first sound source collection element and at least one second sound source collection element. Specifically, based on the relative distance between each participant and each sound source collection element, the first sound source collection element and at least one second sound source collection element corresponding to the target speaker can be determined, and based on the sound information collected by the second sound source collection element, the sound information of the participant is filtered to obtain the sound information corresponding to the first sound source collection element; based on the sound information corresponding to the first sound source collection element, the sound information of the target speaker is obtained.

[0054] It is understandable that, since the location of the sound information source in the target scene is not fixed and may be located in any direction in the target scene, the embodiment of the present application sets each sound source collection element at a different position of the audio collection device, so as to first determine the first sound source collection element and its second sound source collection element closest to each participant, and then determine the sound information of the target speaker among multiple participants based on the sound information collected by the first sound source collection element and the sound information collected by other second sound source collection elements. The specific sound source collection element can use a microphone (MIC) to convert the sound signal into an electrical signal.

[0055] Possibly, the embodiment of the present application can set an image acquisition element on the audio playback device end, and determine the relative distance between each participant and each sound source acquisition element through the image information in the target scene acquired by the image acquisition element. The image acquisition element can be a camera, and the image information in the target scene can be acquired by adding a camera to the audio playback device.

[0056] See also Figure 3B In a specific example, the target scene includes: an audio acquisition device (circular microphone) and three meeting participants. The audio acquisition device is provided with three MICs located at different positions, one MIC is provided approximately every 120 degrees, and the three meeting participants are located at different positions of the target scene. The embodiment of the present application can shoot the target scene through the camera in the audio playback device to obtain Figure 3BThe relative distance relationship diagram of each participant and the audio collection device in the target scene shown in the figure can determine the relative distance between participant 2 and the audio collection device by image recognition ( Figure 3B The dotted line marked with the mark is the smallest, that is, participant 2 is the target speaker. It can be further determined that the relative distance between participant 2 and MIC2 is smaller than that between MIC1 and MIC3. Therefore, MIC2 is the first sound source collection element, and MIC1 and MIC3 are the second sound source collection elements.

[0057] Possibly, the embodiment of the present application can also determine the relative distance between each participant and each sound source collection element by setting a position collection element on the audio collection device end through a distance measuring element. The distance measuring element can be a distance measuring element such as a laser rangefinder (measured by emitting laser into the target scene), a thermal infrared remote sensing device (measured by measuring the heat source corresponding to the participant), etc. By using the distance measuring element in the audio collection device, the relative distance between each participant and the audio collection device is determined and sent to the audio playback device.

[0058] Possibly, the laser rangefinder identifies each participant by emitting laser light into the target scene to form a point cloud, and measures the distance between each participant and each sound source collection element. The thermal infrared remote sensing device measures the distance by measuring the heat source corresponding to the participant, and the detected heat source corresponding to each participant is different.

[0059] See also Figure 3C In a specific example, the target scene may include: an audio collection device and two participants. The audio collection device includes: a laser rangefinder and three sound source collection elements. The three sound source collection elements are located at different positions of the audio collection device, namely MIC1, MIC2, and MIC3. The laser rangefinder can be used to measure the relative distance between participant 1 and sound collection unit 1 ( Figure 3C The relative distance between participant 1 and sound collection unit 2 is 2 meters, the relative distance between participant 1 and sound collection unit 3 is 1.6 meters, the relative distance between participant 2 and sound collection unit 1 is 1 meter, the relative distance between participant 2 and sound collection unit 2 is 0.6 meters, and the relative distance between participant 2 and sound collection unit 3 is 0.8 meters. In this way, it can be determined that participant 2 is the target speaker, MIC2 is the first sound source collection element, and MIC1 and MIC3 are the second sound source collection elements.

[0060] See also Figure 3DThe conference equipment application scenario diagram shown in FIG. The application scenario may include: an audio playback device, an audio acquisition device, and 4 participants. Among them, the audio acquisition device is equipped with a thermal infrared remote sensing device and a sound source acquisition element ( Figure 3D (not shown in the figure), through the measurement information of the thermal infrared remote sensing device, it can be determined that the heat source position of participant 2 and the heat source position of participant 3 are not fixed (the corresponding heat source positions change, for example, participant 2 corresponds to two heat source positions, namely, heat source position 1 of participant 2 and heat source position 2 of participant 2), the distance between the position of participant 1 and the audio collection device, and the distance between the position of participant 4 and the audio collection device are relatively fixed, but because the relative distance between the position of participant 1 and the audio collection device is greater than the relative distance between the position of participant 4 and the audio collection device. Therefore, participant 4 can be determined as the target speaker.

[0061] It is understandable that, since the relative distance between the first sound source collection element and the target speaker is smaller than the relative distance between the second sound source collection element and the target speaker, the sound component of the target speaker collected by the first sound source collection element will be higher than the sound component of the target speaker collected by the second sound source collection element. Therefore, the embodiment of the present application can filter the sound information of the participants, filter out the sound information collected by the second sound source collection element, and retain the sound information collected by the first sound source collection element.

[0062] Possibly, the embodiment of the present application may also add a negative feedback system to adjust the surrounding audio state in real time, enhance the effective voice information according to the specific frequency directionality information of the voice signal and the sound pressure level information of a specific frequency band, and ensure that the useful audio information for online meetings is clear and not interfered with by the outside world.

[0063] Furthermore, the embodiment of the present application can also perform harmonic analysis on the sound information corresponding to the first sound source collecting element to obtain the harmonics corresponding to the first sound source collecting element; filter the harmonics corresponding to the first sound source collecting element to obtain the sound harmonics of the target speaker; compensate for the sound harmonics of the target speaker to obtain the sound information of the target speaker.

[0064] Specifically, the embodiment of the present application can perform multiple harmonic discrete analysis on the sound information collected by the first sound source collection element to obtain harmonic components, reduce the sound pressure level power of the specified frequency point, and filter the sound information corresponding to the second sound source collection element to obtain the sound information corresponding to the first sound source collection element.

[0065] It can be understood that, since the positions of the participants are different and the sound frequencies of the participants are different, the sound information of different participants can be distinguished in terms of parameters such as the frequency domain and sound pressure level according to the inherent delay, directionality, and sound frequency of the sound.

[0066] Possibly, when filtering out the secondary sound source information corresponding to other participants, since the frequency domain points of human voices are similar, it is possible to filter out the secondary harmonic data of some of the main sound source information while ensuring that the overall human voice is not distorted. Therefore, the corresponding part of the data needs to be sound compensated under a scene algorithm database such as the audio equalizer algorithm.

[0067] In a specific example, the audio collection device in the conference equipment collects the first sound information in the conference room through multiple sound source collection elements, and determines the relative distance between each sound source collection element and each participant through a camera, wherein the relative distance between the target speaker among the participants and each sound collection element remains unchanged, and further, the sound source collection element closest to the target speaker can be determined as the first sound source collection element, and the other sound source collection elements are the second sound source collection elements. The first sound information is filtered based on the sound information corresponding to the second sound source collection element to obtain the sound information corresponding to the first sound source collection element, and then the sound information corresponding to the first sound source collection element is subjected to harmonic analysis, reduced human voice power, and filtered out secondary sound source information, and the sound information of the target speaker can be obtained.

[0068] The embodiment of the present application can collect sound information at different positions in the target scene by setting multiple sound source collection elements in the audio collection device, so that the audio playback device can determine the sound information of the target speaker based on the relative distance between each participant and each sound source collection element. Therefore, the embodiment of the present application can determine the voice information of the target speaker based on the relative distance to obtain clear and accurate single-person voice information, ensure that the useful audio information of the network conference is clear without being interfered by the voices of other participants, and effectively solve the noise problem such as the possible collection of other people's voices in the related technology.

[0069] Figure 4A A flowchart of another audio processing method is provided as shown in FIG. Figure 4A The audio processing method shown may include the following steps:

[0070] S401: An audio collection device obtains second sound information of a target scene during a non-meeting period.

[0071] The non-meeting period refers to a time period without a meeting, such as a time period when the conference equipment is in standby mode, when online conference related software is not started, etc. The second sound information refers to sound information that may exist in the target scene during the time period without a meeting, such as noise information in the external environment.

[0072] It is understandable that the noise information of the external environment in the target scene is relatively fixed. For example, in a conference room facing the street, most of its noise information is noisy human voices, noise from vehicles, etc. Therefore, the purpose of the audio acquisition device in the embodiment of the present application to obtain the second sound information of the target scene during the non-meeting period is to calibrate the second sound information in the target scene during the non-meeting period based on the noise type, and obtain the noise information in the target scene during the non-meeting period. The noise information is then used to further perform noise reduction processing on the meeting sound information of the target scene during the meeting.

[0073] For example, an audio acquisition device can be turned on in the target scene during non-meeting periods to collect the second sound information in real time. Specifically, the device can be adjusted to an environmental simulation state for real-time recording to collect noise that may exist in the target scene during non-meeting periods. S402, the audio acquisition device sends the second sound information of the target scene during non-meeting periods to the audio playback device.

[0074] S403: The audio playback device calibrates the second sound information in the target scene during the non-meeting period based on the noise type to obtain the noise information in the target scene during the non-meeting period.

[0075] Among them, by continuously accumulating noise audio data and using machine learning models, a corresponding scene noise database is formed internally. Then, in the subsequent meeting stage, the noise information is compared with the real-time meeting sound information on site during the meeting to assist in audio cancellation and correction in the sound calibration process.

[0076] See also Figure 4BThe application scenario diagram of the audio processing method shown. The conference device can be placed in different scenes. When the conference device is placed in scene 1, the audio acquisition device will collect the second sound information in scene 1 during non-conference periods (the sound that may exist in scene 1 during non-conference periods). When the conference device is placed in scene 2, the audio acquisition device will collect the second sound information in scene 2 during non-conference periods. When the conference device is placed in scene 3, the audio acquisition device will collect the second sound information in scene 3 during non-conference periods. Further, the audio acquisition device sends the second sound information to a digital signal processor (DSP) module. The DSP module may include: a second sound data distinguishing unit, a second sound data calibration unit, and an audio data updating unit. Among them, the second sound information distinguishing unit is used to distinguish the noise type of the second sound information. The second sound information calibration unit is used to calibrate the noise type of the second sound information, that is, to mark different second sound information. The audio data updating unit is used to store and periodically update the noise information with the mark. It can be understood that different scenes correspond to different types of noise information.

[0077] S404: The audio collection device obtains first sound information in the target scene during the conference, and relative distances between multiple conference participants and the audio collection device.

[0078] Specifically, S404 is consistent with S301 and will not be described in detail here.

[0079] S405: The audio collection device sends the first sound information and the relative distance to the audio playback device.

[0080] Specifically, S405 is consistent with S302 and will not be described in detail here.

[0081] S406: The audio playback device determines the voice information of the meeting participants in the target scene based on the meeting voice information in the target scene and the noise information in the target scene during non-meeting periods.

[0082] Possibly, the embodiment of the present application may obtain the sound information of the participants in the target scene by canceling the conference sound information in the target scene and the noise information in the target scene during non-meeting periods.

[0083] Specifically, in the real-time audio processing process of the audio playback device, the first sound information of the audio acquisition device is firstly filtered, and then secondary filtered using the noise information type determined during the non-meeting period to achieve real-time output of participant information.

[0084] S407: The audio playback device obtains the voice information of the target speaker based on the voice information and relative distance of the participants.

[0085] Specifically, S407 is consistent with S304 and will not be described in detail here.

[0086] See also Figure 4C The schematic diagram of the module structure corresponding to the audio processing method shown. During non-conference periods, multiple sound source collection elements set in the audio collection device collect the sound information in the target scene and send it to the audio playback device. The algorithm database in the audio playback device can use a pre-set algorithm to distinguish and calibrate the sound information to obtain the scene noise information corresponding to the target scene, and further filter the sound information during the meeting based on the scene noise information, that is, after the sound information during the meeting after the primary filtering is converted into analog to digital, the feedback system is used to adjust the surrounding audio state in real time, and the effective audio information is enhanced according to the specific frequency directionality information of the voice information and the sound pressure level information of the specific frequency band. After the DSP module is used to adjust the real-time sound spectrum to retain all the human voice information, the sound pressure reduction and multi-level harmonics are used for re-filtering to obtain the sound information of the target speaker. Finally, the power amplifier function is realized by the power amplifier, and the weak electrical signal corresponding to the sound information of the target speaker is amplified to drive the speaker to make a sound.

[0087] Therefore, the embodiment of the present application can further filter the scene noise information that may exist during the meeting through the environmental noise information collected during non-meeting periods, so as to optimize the sound source and filter out specific noise, thereby enhancing useful audio information. In addition, the embodiment of the present application also uses artificial intelligence algorithms as an aid to optimize the use scenarios of conference equipment and gradually improve the user experience of online meetings.

[0088] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0089] Figure 5 1 is a schematic diagram of the structure of the audio processing device provided by the present application. The device is used for an audio playback device, the audio playback device communicates with an audio acquisition device, and the device is used to execute the audio processing application method of any of the above embodiments of the present specification. Figure 5 As shown, the audio processing device may include:

[0090] The first sound information receiving module 51 is used to receive the first sound information in the target scene during the meeting sent by the audio collection device; wherein the meeting period refers to the time period corresponding to the meeting participated by multiple participants, and the first sound information and the relative distance are obtained by multiple sound source collection elements arranged at different positions in the audio collection device;

[0091] A participant voice information determination module 52, configured to determine voice information of a participant in a target scene based on first voice information in the target scene during a meeting;

[0092] The target speaker sound information obtaining module 53 is used to obtain the sound information of the target speaker among multiple participants based on the sound information of the participants and the relative distances between each participant and each sound source collecting element.

[0093] In some implementations, the participant voice information determination module 52 includes:

[0094] A conference sound information obtaining unit, used to filter the first sound information in the target scene during the conference to obtain the conference sound information in the target scene;

[0095] The voice information determination unit of the conference participants is used to determine the voice information of the conference participants in the target scene based on the conference voice information in the target scene.

[0096] In some implementations, before the first sound information receiving module 51, the device further includes:

[0097] The second sound information receiving module is used to receive the second sound information of the target scene during the non-meeting period sent by the audio acquisition device; wherein the non-meeting period refers to the time period without meetings;

[0098] A first noise information obtaining module is used to calibrate the second sound information in the target scene during the non-meeting period based on the noise type to obtain the noise information in the target scene during the non-meeting period;

[0099] The voice information determination unit of the participants is specifically used for:

[0100] Based on the conference sound information in the target scene and the noise information in the target scene during non-meeting periods, the sound information of the participants in the target scene is determined.

[0101] In some embodiments, the relative position between each sound source collection element and the target speaker remains unchanged; the plurality of sound source collection elements include: a first sound source collection element and at least one second sound source collection element; the relative distance between the first sound source collection element and the target speaker is the smallest;

[0102] The target speaker voice information obtaining module 53 includes:

[0103] A first sound information obtaining unit is used to filter the sound information of the participants based on the sound information collected by the second sound source collecting element to obtain the sound information corresponding to the first sound source collecting element;

[0104] The second sound information obtaining unit is used to obtain the sound information of the target speaker based on the sound information corresponding to the first sound source collecting element.

[0105] In some implementations, the second sound information obtaining unit is specifically configured to:

[0106] Perform harmonic analysis on the sound information corresponding to the first sound source collection element to obtain the harmonics corresponding to the first sound source collection element; filter the harmonics corresponding to the first sound source collection element to obtain the sound harmonics of the target speaker; compensate for the sound harmonics of the target speaker to obtain the sound information of the target speaker.

[0107] In some embodiments, the audio playback device includes: an image acquisition element;

[0108] Before the target speaker voice information obtaining module 53, the device further includes:

[0109] The first relative distance determination module is used to determine the relative distance between each participant and each sound source collection element through the image information in the target scene collected by the image collection element.

[0110] In some implementations, before the target speaker voice information obtaining module 53, the device further includes:

[0111] The relative distance receiving module is used to receive the relative distance between each participant and each sound source collection element sent by the audio collection device.

[0112] Figure 6 1 is a schematic diagram of the structure of the audio processing device provided by the present application. The device is used in an audio acquisition device, and the audio acquisition device includes multiple sound source acquisition components; each sound source acquisition component is located at a different position in the audio acquisition device; the audio acquisition device communicates with the audio playback device, and the device is used to execute the audio processing application method of any of the above embodiments of the present specification. Figure 6 As shown, the audio processing device may include:

[0113] A first sound information acquisition module 61 is used to acquire first sound information in a target scene during a meeting; wherein the relative distance is obtained through a plurality of sound source acquisition elements;

[0114] The first sound information sending module 62 is used to send the first sound information to the audio playback device, so that the audio playback device can obtain the sound information of the target speaker among multiple participants based on the first sound information and the relative distance between each participant and each sound source collection element.

[0115] In some implementations, before the first sound information acquisition module 61, the device further includes:

[0116] The second noise information obtaining module is used to obtain the second sound information of the target scene during the non-meeting period, so as to calibrate the second sound information in the target scene during the non-meeting period based on the noise type through the audio playback device to obtain the noise information in the target scene during the non-meeting period.

[0117] In some embodiments, the audio acquisition device further includes a distance measurement element;

[0118] Before the first sound information acquisition module 61, the device further includes:

[0119] A second relative distance determination module is used to determine the relative distance between each participant and each sound source collection element through a distance measurement element in the audio collection device;

[0120] The first sound information sending module 62 is specifically used for:

[0121] The first sound information and the relative distance are sent to the audio playing device.

[0122] It should be noted that the audio processing device provided in the above embodiment only uses the division of the above functional modules as an example when executing the audio processing method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the audio processing device provided in the above embodiment and the audio processing method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0123] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0124] The embodiments described above are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the protection scope determined by the claims of the present application.

Claims

1. An audio processing method, the method is used for the audio playback device, the audio playback device communicates with the audio acquisition device; characterized in that: The method comprises: Receiving first sound information in a target scene during a meeting sent by the audio collection device; wherein the meeting period represents a time period corresponding to a meeting in which multiple participants participate, and the first sound information is obtained by multiple sound source collection elements located at different positions in the audio collection device; Determining the voice information of the participants in the target scene based on the first voice information in the target scene during the meeting; Based on the voice information of the participants and the relative distances between each participant and each sound source collecting element, the voice information of a target speaker among the multiple participants is obtained.

2. The method according to claim 1, characterized in that The determining, based on the first sound information in the target scene during the meeting, the sound information of the participants in the target scene includes: Filtering the first sound information in the target scene during the meeting to obtain the meeting sound information in the target scene; Based on the conference sound information in the target scene, the sound information of the participants in the target scene is determined.

3. The method according to claim 1 or 2, characterized in that: Before receiving the first sound information in the target scene during the meeting sent by the audio acquisition device, the method further includes: Receiving second sound information of the target scene during a non-meeting period sent by the audio acquisition device; wherein the non-meeting period refers to a time period without a meeting; Calibrate the second sound information in the target scene during the non-meeting period based on the noise type to obtain the noise information in the target scene during the non-meeting period; The determining, based on the conference sound information in the target scene, the sound information of the participants in the target scene includes: Based on the conference sound information in the target scene and the noise information in the target scene during a non-meeting period, the sound information of the participants in the target scene is determined.

4. The method according to claim 1, characterized in that: The relative distance between each of the sound source collecting elements and the target speaker remains unchanged; The plurality of sound source collecting elements include: a first sound source collecting element and at least one second sound source collecting element; the relative distance between the first sound source collecting element and the target speaker is the smallest; The obtaining of the sound information of the target speaker among the multiple participants based on the sound information of the participants and the relative distances between each participant and each sound source collecting element includes: Determine, based on the relative distances between each of the conference participants and each of the sound source collecting elements, a first sound source collecting element and at least one second sound source collecting element corresponding to the target speaker; Based on the sound information collected by the second sound source collection element, the sound information of the conference participants is filtered to obtain the sound information corresponding to the first sound source collection element; Based on the sound information corresponding to the first sound source collecting element, the sound information of the target speaker is obtained.

5. The method according to claim 4, characterized in that The obtaining the sound information of the target speaker based on the sound information corresponding to the first sound source collecting element includes: Performing harmonic analysis on the sound information corresponding to the first sound source collecting element to obtain harmonics corresponding to the first sound source collecting element; Filtering the harmonics corresponding to the first sound source collection element to obtain the sound harmonics of the target speaker; The sound harmonics of the target speaker are compensated to obtain the sound information of the target speaker.

6. The method according to claim 1, characterized in that The audio playback device comprises: an image acquisition element; Before obtaining the voice information of a target speaker among the multiple participants based on the voice information of the participants and the relative distances between each participant and each sound source collecting element, the method further includes: The relative distance between each of the conference participants and each of the sound source collecting elements is determined through the image information in the target scene collected by the image collecting element.

7. The method according to claim 1, characterized in that Before obtaining the voice information of a target speaker among the multiple participants based on the voice information of the participants and the relative distances between each participant and each sound source collecting element, the method further includes: Receive the relative distance between each of the conference participants and each of the sound source collection elements sent by the audio collection device.

8. An audio processing method, characterized in that: The audio processing method is used in an audio acquisition device, and the audio acquisition device includes a plurality of sound source acquisition elements; Each of the sound source collection elements is located at a different position in the audio collection device; The audio acquisition device communicates with the audio playback device, and the method includes: Acquire first sound information in a target scene during a meeting; wherein the meeting period represents a time period corresponding to a meeting in which multiple participants participate, and the first sound information is obtained through the multiple sound source collection elements; The first sound information is sent to the audio playback device, so that the audio playback device can obtain the sound information of a target speaker among the multiple participants based on the first sound information and the relative distances between each participant and each sound source collecting element.

9. The method according to claim 8, characterized in that Before acquiring the first sound information in the target scene during the meeting, the method further includes: The second sound information of the target scene during the non-meeting period is obtained, so that the second sound information in the target scene during the non-meeting period is calibrated based on the noise type by the audio playback device to obtain the noise information in the target scene during the non-meeting period.

10. The method according to claim 8 or 9, characterized in that: The audio acquisition device also includes a distance measurement element; Before sending the first sound information to the audio playback device, the method further includes: Determine the relative distance between each of the participants and each of the sound source collecting elements by using the distance measuring element; The sending the first sound information to the audio playback device includes: The first sound information and the relative distance are sent to the audio playing device.

11. An audio processing device, the device is used for the audio playback device, the audio playback device communicates with the audio acquisition device; characterized in that: The device comprises: A first sound information receiving module is used to receive the first sound information in the target scene during the meeting sent by the audio collection device; wherein the meeting period represents a time period corresponding to the meeting participated by multiple participants, and the first sound information and the relative distance are obtained by multiple sound source collection elements arranged at different positions in the audio collection device; A participant voice information determination module, used to determine the voice information of the participant in the target scene based on the first voice information in the target scene during the meeting; The target speaker sound information obtaining module is used to obtain the sound information of the target speaker among the multiple participants based on the sound information of the participants and the relative distances between each participant and each sound source collecting element.

12. An audio processing device, the audio processing device being used in an audio acquisition device, characterized in that: The audio collection device includes a plurality of sound source collection elements; Each of the sound source collection elements is located at a different position in the audio collection device; The audio acquisition device communicates with the audio playback device; the device comprises: A first sound information acquisition module is used to acquire first sound information in a target scene during a meeting; wherein the relative distance is obtained by the multiple sound source acquisition elements; The first sound information sending module is used to send the first sound information to the audio playback device, so that the audio playback device can obtain the sound information of the target speaker among the multiple participants based on the first sound information and the relative distance between each participant and each sound source collection element.

Images