Audio processing method and device, audio playing method and device, equipment and storage medium

Abstract

The invention provides an audio processing method and device, an audio playing method and device, equipment and a storage medium. The method comprises the steps of obtaining position information of each piece of audio playing equipment; receiving a first audio collected by a first audio playing device in the audio playing devices, and generating a second audio based on the first audio; based on the position information of each audio playing device, determining the distance from each second audio playing device except the first audio playing device in each audio playing device to the first audio playing device; based on the distance, determining the volume of the second audio played by each second audio playing device; and the second audio and the volume for playing the second audio are sent to each second audio playing device, so that each second audio playing device plays the second audio according to the volume for playing the second audio, and the nearby speaking sound can be clearly heard in a noisy environment.

Description

Technical Field

[0001] The present application belongs to the field of audio processing technology, and in particular, relates to an audio processing method, an audio playing method, a device, a equipment and a storage medium. Background Art

[0002] Noise is a common problem in industrial production environments. The high-decibel mechanical operation noise not only poses a potential threat to employees' hearing, but also seriously affects the communication efficiency between employees. In such an environment, traditional devices such as walkie-talkies and earplugs often cannot meet the needs. For example, when using earplugs and in such a noisy environment, it is basically impossible to hear the voices of nearby people clearly. Summary of the invention

[0003] The embodiments of the present application provide an audio processing method, an audio playback method, an apparatus, a device and a storage medium, which can enable the sound of nearby conversations to be clearly heard in a noisy environment.

[0004] According to a first aspect of the present application, an embodiment of the present application provides an audio processing method, which is applied to an audio processing device, including:

[0005] Get the location information of each audio playback device;

[0006] receiving a first audio captured by a first audio playback device among the audio playback devices, and generating a second audio based on the first audio;

[0007] Based on the position information of each of the audio playback devices, determining the distances from each of the second audio playback devices except the first audio playback device to the first audio playback device;

[0008] Based on the distance, determining the volume of the second audio played by each of the second audio playback devices;

[0009] The second audio and the volume at which the second audio is played are sent to each of the second audio playback devices, so that each of the second audio playback devices plays the second audio at the volume at which it plays the second audio.

[0010] In some embodiments, determining the volume of each of the second audio playback devices includes:

[0011] Obtaining a pre-established correspondence table between distance and volume;

[0012] The volume of the second audio played by each second audio playback device is determined from the correspondence table according to the distance from each second audio playback device to the first audio playback device.

[0013] In some embodiments, the distance in the correspondence table is divided into an adjacent first distance interval and a second distance interval; the volume corresponding to the first distance interval decreases as the corresponding distance increases, the volume corresponding to the second distance interval remains unchanged, and the first distance interval is smaller than the second distance interval, and the volume corresponding to the first distance interval is greater than the volume corresponding to the second distance interval.

[0014] In some embodiments, obtaining the location information of each audio playback device includes:

[0015] Obtaining the location information of the current audio playback device uploaded by each of the audio playback devices;

[0016] The location information is determined by each of the audio playback devices performing UWB communication with at least three base stations at fixed locations and positioning the current audio playback device based on the strength of the UWB signals fed back by each of the base stations.

[0017] In some embodiments, positioning the current audio playback device by using the strength of the UWB signal fed back by each base station includes:

[0018] The strength of the UWB signal fed back by each base station for the current audio playback device is input into the positioning model, so that the positioning model determines the position information of the current audio playback device based on the three-point positioning principle.

[0019] In some embodiments, generating the second audio based on the first audio includes:

[0020] performing noise suppression processing on the first audio to obtain a noise-suppressed first audio;

[0021] Perform speech enhancement processing on the noise-suppressed first audio to obtain the second audio.

[0022] In some embodiments, the method further comprises:

[0023] The second audio is sent to the first audio playback device, so that the first audio playback device plays the second audio at a default volume, where the default volume is the same as the volume of the first audio.

[0024] According to a second aspect of the present application, the present application provides an audio playback method, which is applied to an audio playback device, and the audio playback method includes:

[0025] The audio playback device acquires its own position information and transmits it to the audio processing device, and when collecting the first audio, acts as the first audio playback device to send the first audio to the audio processing device;

[0026] The audio playback device, as a second audio playback device, receives the second audio and the volume of the second audio sent by the audio processing device, and plays the second audio at the volume corresponding to the volume of the second audio played by itself.

[0027] The second audio is obtained by the audio processing device performing audio processing on the first audio collected by the first audio playback device and sent to the audio processing device, and the volume of playing the second audio is determined by the audio processing device based on the distance between the first audio playback device and the second audio playback device as the receiver.

[0028] In some embodiments, the audio playback device obtains its own location information, including:

[0029] After performing UWB communication with at least three base stations at fixed locations, the current audio playback device is positioned by the strength of the UWB signals fed back by each of the base stations to determine the location information of the current audio playback device.

[0030] According to a third aspect of the present application, an embodiment of the present application provides an audio processing device, including:

[0031] A location information acquisition module is used to acquire the location information of each audio playback device;

[0032] A first receiver receives a first audio captured by a first audio player among the audio players, and generates a second audio based on the first audio;

[0033] a control module, based on the position information of each of the audio playback devices, determining the distances from each of the second audio playback devices except the first audio playback device to the first audio playback device; and based on the distances, determining the volume of the second audio played by each of the second audio playback devices;

[0034] The first transmitter sends the second audio and the volume of playing the second audio to each of the second audio playback devices, so that each of the second audio playback devices plays the second audio at the volume of the second audio played by itself.

[0035] According to a fourth aspect of the present application, there is provided an audio playback device, a position information acquisition module, which acquires its own position information;

[0036] A second transmitter transmits its position information to the audio processing device;

[0037] An audio collection device, used for collecting sound and sending the collected audio as the first audio to the audio processing device;

[0038] a second receiver, receiving the second audio and the volume of playing the second audio sent by the audio processing device, and playing the second audio at the volume corresponding to the volume of playing the second audio by itself,

[0039] The second audio is obtained by the audio processing device performing audio processing on the first audio collected by the first audio playback device and sent to the audio processing device, and the volume of playing the second audio is determined by the audio processing device based on the distance between the first audio playback device and the second audio playback device as each receiver.

[0040] According to the fifth aspect of the present application, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the computer program.

[0041] According to the sixth aspect of the present application, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above items is implemented.

[0042] According to the seventh aspect of the present application, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the electronic device to execute any of the methods described above.

[0043] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0044] The audio processing method provided in the embodiment of the present application obtains the position information of each audio playback device; receives the first audio collected by the first audio playback device in each of the audio playback devices, and generates the second audio based on the first audio; determines the distance from each of the second audio playback devices except the first audio playback device to the first audio playback device based on the position information of each of the audio playback devices; determines the volume of the second audio played by each of the second audio playback devices based on the distance; sends the second audio and the volume of playing the second audio to each of the second audio playback devices, so that each of the second audio playback devices plays the second audio at the volume of its own playing of the second audio, so that the voices of nearby colleagues can be clearly heard in a noisy environment.

[0045] It can be understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A schematic diagram of an implementation flow of an audio processing method provided for the implementation of this application;

[0048] Figure 2 A schematic diagram of an implementation flow of an audio playback method provided for the implementation of this application;

[0049] Figure 3 A schematic diagram of the structure of an audio processing system provided in an embodiment of the present application;

[0050] Figure 4 A flowchart of another audio processing method provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of the corresponding relationship between distance and volume in an audio processing method provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application;

[0053] Figure 7 A schematic diagram of the structure of an audio playback device provided in an embodiment of the present application;

[0054] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0056] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0057] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0058] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to audio processing, depending on the context. Similarly, the phrases "if it is determined" or "if the audio is processed to" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce the audio is processed to" or "in response to audio processing, depending on the context.

[0059] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0060] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0061] Based on the technical problems of the related technology, the embodiments of the present application provide an audio processing method that can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific type of electronic devices. The electronic device can be used as an audio processing device, and the audio processing device can be a background system.

[0062] The functions implemented by the audio processing method provided in the embodiment of the present application can be implemented by calling program codes by a processor of an electronic device, wherein the program codes can be stored in a computer storage medium.

[0063] Figure 1A schematic diagram of an implementation flow of an audio processing method provided for the implementation of this application is intended to achieve intelligent distribution and volume control of audio information among multiple devices in the room to simulate the sound propagation effect in a real scene, such as Figure 1 As shown, the audio processing method includes:

[0064] Step S101, obtaining location information of each audio playback device.

[0065] In the embodiment of the present application, the audio playback device may be a smart headset, a microphone array or a portable recording device. In the embodiment of the present application, the electronic device may obtain the location information of each audio playback device. As an example, the electronic device obtains the location information of each audio playback device through UWB (ultra-wideband) technology.

[0066] Step S102: receiving a first audio captured by a first audio playback device among the audio playback devices, and generating a second audio based on the first audio.

[0067] In the embodiment of the present application, the first audio is the audio information of the surrounding environment, including the voice signals of nearby colleagues, and the first audio playback device can collect the audio information through the audio sensor thereon.

[0068] In an embodiment of the present application, the first audio is preprocessed to obtain the second audio to ensure audio quality, remove noise and redundant information, and prepare for subsequent audio distribution and volume control. Preprocessing includes noise suppression and denoising, echo cancellation, volume equalization, etc. After the above preprocessing steps, the original first audio will be converted into a processed second audio, which has better sound quality, less noise and redundant information, and a format and characteristics that are more suitable for subsequent processing and playback. It should be noted that the selection and combination of the above preprocessing steps depends on the specific application scenario and requirements.

[0069] Step S103: determining the distances from each second audio playback device, except the first audio playback device, to the first audio playback device based on the position information of each audio playback device.

[0070] In the embodiment of the present application, determining the distances from each second audio playback device other than the first audio playback device to the first audio playback device may depend on the geographic location coordinates of the first audio playback device and each second audio playback device, wherein the location coordinates generally include longitude and latitude.

[0071] In the embodiment of the present application, the Haversine formula may be used to calculate the distance between two points.

[0072] Step S104: determining the volume of the second audio played by each of the second audio playback devices based on the distance.

[0073] In the embodiment of the present application, the electronic device needs to determine the volume of the second audio played by each second audio playback device according to the distance information between each second audio playback device and the first audio playback device.

[0074] In an embodiment of the present application, a volume attenuation model is stored in the electronic device, wherein the volume attenuation model includes a correspondence table between distance and volume. The electronic device can determine the volume of the second audio played by each second audio playback device through the distance information between each second audio playback device and the first audio playback device.

[0075] Step S105: Send the second audio and the volume of playing the second audio to each of the second audio playback devices, so that each of the second audio playback devices plays the second audio at its own volume of playing the second audio.

[0076] In an embodiment of the present application, the electronic device can communicate with each second audio playback device through network communication (Wi-Fi, Bluetooth, local area network (LAN) or other wireless / wired communication methods), construct a control instruction containing volume setting information according to the requirements of the communication protocol, and send the constructed control instruction to the corresponding second audio playback device. Through the above steps, each second audio playback device can be effectively controlled to output the second audio at their respective corresponding volumes, thereby providing users with a more personalized and comfortable audio experience.

[0077] The audio processing method provided in the embodiment of the present application obtains the position information of each audio playback device; receives the first audio collected by the first audio playback device in each of the audio playback devices, and generates the second audio based on the first audio; determines the distance from each of the second audio playback devices except the first audio playback device to the first audio playback device based on the position information of each of the audio playback devices; determines the volume of the second audio played by each of the second audio playback devices based on the distance; sends the second audio and the volume of playing the second audio to each of the second audio playback devices, so that each of the second audio playback devices plays the second audio at the volume of its own playing of the second audio, so that the voices of nearby colleagues can be clearly heard in a noisy environment.

[0078] According to the present application, the audio processing method also includes pre-establishing a correspondence table between distance and volume to achieve the purpose of determining that each second audio playback device plays the second audio according to the distance information between each second audio playback device and the first audio playback device.

[0079] In some embodiments, step S104, determining the volume of each of the second audio playback devices playing the second audio, includes:

[0080] Step S1041, obtaining a pre-established correspondence table between distance and volume.

[0081] In this step, the correspondence table may be pre-stored in the electronic device, wherein the correspondence between the distance and the volume is stored in the correspondence table.

[0082] In this step, the correspondence table may be stored in a memory of the electronic device, in the cloud, or in a memory card so as to be easily accessed when needed.

[0083] Step S1042: determining the volume of the second audio played by each second audio playback device from the correspondence table according to the distance from each second audio playback device to the first audio playback device.

[0084] In this step, the correspondence table is traversed to find the entry that matches the distance of each second audio playback device. This may involve determining which interval the distance belongs to, or performing more complex calculations to find the closest match. Based on the correspondence found, the volume value that each second audio playback device should play is determined. If the correspondence table is given in the form of intervals, it may be necessary to perform interpolation calculations based on the actual distance of the device to obtain a more accurate volume value. The determined volume value can be applied to the corresponding second audio playback device by sending a volume control instruction to the corresponding second audio playback device.

[0085] In these embodiments, the volume of the second audio played by each second audio playback device can be determined according to the distance between each second audio playback device and the first audio playback device and the pre-established correspondence table between the distance and the volume. In a noisy environment, employees can clearly hear the voices of nearby colleagues, reducing misunderstandings and information miscommunication. In addition, indoor users can communicate effectively and transmit safety information and operating instructions in a timely manner, reducing the occurrence of accidents and improving production safety. In addition, the working environment and communication experience can be improved, thereby improving employee job satisfaction and enthusiasm.

[0086] In some embodiments, the distance in the correspondence table is divided into an adjacent first distance interval and a second distance interval; the volume corresponding to the first distance interval decreases as the corresponding distance increases, the volume corresponding to the second distance interval remains unchanged, and the first distance interval is smaller than the second distance interval, and the volume corresponding to the first distance interval is greater than the volume corresponding to the second distance interval.

[0087] In this step, the first distance interval and the second distance interval can be set according to actual conditions. As an example, the preset broadcast range can be a certain range centered on the first audio playback device. Specifically, the first distance interval can be within 10 meters, and the second distance interval can be outside 10 meters. By dividing the intervals, it is possible to plan which areas of the audience can hear the audio at what volume. Table 1 is a table of the corresponding relationship between distance and volume provided in an embodiment of the present application, as shown in Table 1:

[0088] Table 1 Example of the correspondence between distance and volume

[0089] Serial number equipment Distance from device 1 (m) Volume 1 Device 2 1 9 2 Device 3 2 8 3 Device 4 4 6 4 Device 5 6 4 5 Device 6 10 0 6 Device 7 15 0 7 Device 8 20 0

[0090] In this step, within the first distance interval, the volume gradually decreases as the distance increases. In the second distance interval, no matter how far the distance is, the volume remains at 0 to avoid interference to listeners out of range and to avoid information leakage.

[0091] In these embodiments, it is defined how the volume should be adjusted within different distance ranges, thereby ensuring that listeners within a preset broadcast range can receive audio at an appropriate volume, while listeners outside the range will not be disturbed or receive unnecessary audio.

[0092] In some embodiments, step S101, obtaining location information of each audio playback device, includes:

[0093] The location information of the current audio playback device uploaded by each of the audio playback devices is obtained, wherein the location information is determined by each of the audio playback devices after locating the current audio playback device through the strength of the UWB signal fed back by each of the base stations after communicating with at least three base stations at fixed locations via UWB.

[0094] In this step, ultra-wideband (UWB) signals are used to determine location information. UWB signals are sent periodically or on demand by a UWB transmitter built into or connected to the audio playback device. When the audio playback device sends UWB signals, these signals have extremely short pulse widths and extremely high bandwidths, allowing them to measure the time of signal propagation very accurately. These signals can be received by the audio playback device or other devices used for positioning. The received UWB signals are used to calculate and determine the precise location of the audio playback device.

[0095] Specifically, the electronic device measures the time difference (Time of Flight, ToF) between sending the UWB signal and receiving the signal. The time difference is proportional to the distance the signal propagates, because the propagation speed of the UWB signal in the air is close to the speed of light.

[0096] Multilateration: The UWB signal from the audio playback device is received by at least three UWB receivers at known locations (these receivers can be fixed base stations or other devices with known locations), and a multilateration algorithm (such as trilateration) is used to calculate the location of each device. Multilateration uses the distances measured from multiple points to estimate the location of an unknown point.

[0097] Based on the time difference and multi-lateration positioning algorithm, the location information of the audio playback device can be calculated. This location information is usually expressed in the form of coordinates (such as longitude and latitude, X / Y / Z coordinates, etc.).

[0098] In these embodiments, ultra-wideband (UWB) signals are used to determine location information, which has the advantages of high-precision positioning, strong anti-interference ability, strong penetration, low power consumption, wide application range, high security and simple system structure, so that indoor environment positioning can be used to ensure the accuracy, timeliness and security of the audio processing method.

[0099] In some embodiments, positioning the current audio playback device according to the strength of the UWB signal fed back by each base station includes:

[0100] The strength of the UWB signal fed back by each base station for the current audio playback device is input into the positioning model, so that the positioning model determines the location information of the current audio playback device based on the three-point positioning principle.

[0101] In these embodiments, the positioning model may be a software model based on an algorithm. By processing the UWB signal using the positioning model, accurate position information of the first audio playback device and each second audio playback device can be calculated.

[0102] In an embodiment of the present application, the preprocessed UWB signal (including possible information such as timestamp, signal strength, signal arrival angle, etc.) can be passed as input data to the positioning model. These input data usually contain signal information received from UWB receivers (or anchor points, base stations) at multiple known locations. One or more positioning algorithms will run inside the positioning model, such as multilateral positioning algorithms (such as trilateral measurement, least squares method, etc.), fingerprint positioning algorithms (matching based on signal features), machine learning algorithms (such as neural networks, support vector machines, etc.), etc. These algorithms will calculate the position of the audio playback device based on the input UWB signal information and possible other auxiliary information (such as map data, environmental models, etc.). After the positioning algorithm is processed, the location information of the audio playback device will be output. These location information are usually expressed in the form of coordinates, such as longitude and latitude, X / Y / Z coordinates, etc., depending on the application scenario and the design requirements of the positioning system.

[0103] In these embodiments, by presetting the positioning model, the location information of the audio playback device can be acquired more quickly, and timely and effective audio processing can be ensured.

[0104] According to the present application, step S102 generates a second audio based on the first audio to make it more suitable for subsequent processing or playback. This process is subdivided into two sub-steps: noise suppression processing and speech enhancement processing.

[0105] In some embodiments, step S102, generating a second audio based on the first audio, includes:

[0106] Step S1021: Perform noise suppression processing on the first audio to obtain a noise-suppressed first audio.

[0107] In this step, the main purpose of noise suppression processing is to reduce or eliminate background noise in the audio, which may come from the environment (such as wind, traffic, crowd noise, etc.) or from the noise of the recording equipment itself (such as circuit noise, mechanical noise, etc.).

[0108] In this step, noise suppression can be performed in a variety of ways. For example, the estimated noise spectrum can be subtracted from the spectrum of the original audio by analyzing the spectral characteristics of the noise. For another example, Wiener filtering, Kalman filtering, etc. can be used to filter the signal by establishing a statistical model of audio and noise to reduce the noise component. For another example, noise suppression methods based on neural networks have also made significant progress, such as using models such as convolutional neural networks (CNN) and recurrent neural networks (RNN) for noise suppression.

[0109] In this step, the background noise in the audio is significantly reduced after noise suppression processing.

[0110] Step S1022: Perform speech enhancement processing on the noise-suppressed first audio to obtain the second audio.

[0111] In this step, speech enhancement processing is based on noise suppression processing to further improve the quality of audio, especially to improve the intelligibility and clarity of speech.

[0112] In this step, speech enhancement processing can be performed in a variety of ways. For example, the clarity of speech can be improved by adjusting the spectral characteristics of the audio, such as enhancing the spectral peak of speech and suppressing the spectrum of non-speech components. Time domain processing can be performed, such as dynamic range compression and automatic gain control, to adjust the dynamic range of the audio to make it more suitable for the auditory characteristics of the human ear. Spatial domain processing can be performed. In the case of multiple microphones, beamforming technology can be used to enhance speech signals from a specific direction while suppressing noise from other directions.

[0113] In this step, after the speech enhancement processing, the speech component in the first audio is further highlighted, and the intelligibility and clarity of the speech are significantly improved, while maintaining good naturalness and sound quality.

[0114] In these embodiments, the first audio is processed through two sub-steps of noise suppression processing and speech enhancement processing to obtain the second audio, which is superior to the first audio in terms of sound quality, intelligibility and clarity.

[0115] According to the present application, the audio processing method further includes processing the control instruction and the third audio from the third audio playback device, and sending the processed third audio to the designated target audio playback device. The following is a detailed explanation of these newly added steps:

[0116] In some embodiments, the audio processing method further includes:

[0117] The second audio is sent to the first audio playback device, so that the first audio playback device plays the second audio at a default volume, where the default volume is the same as the volume of the first audio.

[0118] Based on the above embodiments, the present application also provides an audio playback method, which is applied to an audio playback device. Figure 2 A schematic diagram of an implementation flow of an audio playback method provided in an embodiment of the present application includes:

[0119] Step S201, the audio playback device obtains its own position information and transmits it to the audio processing device, and when collecting the first audio, acts as the first audio playback device to send the first audio to the audio processing device;

[0120] Step S202: the audio playback device, as a second audio playback device, receives the second audio and the volume of the second audio sent by the audio processing device, and plays the second audio at the volume corresponding to the second audio played by itself.

[0121] The second audio is obtained by the audio processing device performing audio processing on the first audio collected by the first audio playback device and sent to the audio processing device, and the volume of playing the second audio is determined by the audio processing device based on the distance between the first audio playback device and the second audio playback device as the receiver.

[0122] In an embodiment of the present application, the first audio playback device collects the first audio (for example, ambient sound or the user's voice) and sends it to the audio processing device. After receiving the first audio, the audio processing device performs audio processing (such as noise reduction, reverberation, etc.) to generate the second audio. The audio processing device calculates the appropriate playback volume based on the distance between the first audio playback device and the second audio playback device.

[0123] In some embodiments, the audio playback device obtains its own location information, including:

[0124] After performing UWB communication with at least three base stations at fixed locations, the current audio playback device is positioned by the strength of the UWB signals fed back by each of the base stations to determine the location information of the current audio playback device.

[0125] In an embodiment of the present application, the audio playback device: has UWB communication capability and is used to interact with the base station for signal. Responsible for receiving signal strength information from the base station to determine its location information. At least three base stations at fixed positions can send and receive UWB signals. Each base station can measure the received signal strength and feed it back to the audio playback device. Each base station feeds back the measured signal strength to the audio playback device. The audio playback device calculates its current location information based on the received signal strength information using triangulation or other positioning algorithms (such as least squares method, weighted positioning, etc.).

[0126] Based on the above embodiments, the present application provides a method for building an audio processing system including audio processing and audio playback functions, and the specific implementation method is as follows:

[0127] In order to ensure that this system can operate effectively in a noisy production environment, the detailed implementation process includes four main stages: hardware design and manufacturing, background system development, deployment and testing, and use and maintenance. The specific steps of each stage are as follows:

[0128] 1. Hardware design and manufacturing.

[0129] In the embodiment of the present application, Figure 3 This is a schematic diagram of the structure of the audio processing system provided in the embodiment of the present application. Figure 3 As shown, the audio processing system includes: a sound pickup end, an electronic device and a playback end. The electronic device can be divided into: an audio processing module, an AI algorithm module, a positioning module, a function selection module, an MCU main control and a power supply module according to different functions. The audio processing module is used to act as the audio processing device in the aforementioned embodiment, and the audio processing module can call other modules according to its own functional requirements to implement the audio processing method in the above embodiment.

[0130] The sound pickup end (equivalent to the first audio playback device mentioned in the above method embodiment) uses a high-performance audio sensor to ensure that the wearer's voice can be accurately captured in a noisy environment.

[0131] Networking module: used to realize the communication connection between electronic devices and the sound pickup and playback ends. The networking module adopts low-power wide area network (LPWAN) modules, such as LoRa, NB-IoT, etc., to ensure that the devices can operate stably for a long time with low power consumption. It can also realize the connection between the device and the gateway, and the gateway transmits data to the background system in real time. Positioning module: adopts ultra-wideband (UWB) precise positioning technology to ensure that the device can accurately locate the wearer's position in the indoor environment.

[0132] Playing end (equivalent to the second audio playing device mentioned in the above embodiment): Choose high-quality headphones or bone conduction headphones to ensure the clarity and comfort of audio playback. In addition, through bone conduction technology and noise isolation, the hearing health of employees is protected and the occurrence of occupational diseases is reduced.

[0133] In addition, all the above-mentioned audio devices are managed through a network, and electronic devices can communicate with audio devices, receive and send audio and command information. Electronic devices can send audio and command information to selected audio devices separately according to judgment conditions, and can send audio information of different volumes to audio devices in different locations.

[0134] 2. Equipment control software development.

[0135] Develop embedded software to coordinate the work of various hardware modules. The software needs to support real-time audio acquisition, processing and transmission, and be able to receive and play audio content sent by the background system, as well as information communicated through fixed-position base stations. The background uses AI learning algorithms to achieve precise indoor positioning through three-point positioning and mobile trajectory.

[0136] 3. Backend system development.

[0137] 1. Data processing module:

[0138] Audio processing: Develop audio processing algorithms to perform noise suppression, speech enhancement and other processing to ensure the high quality of uploaded audio data.

[0139] Positioning data processing: Integrate positioning data from various devices and update the wearer's location information in real time.

[0140] 2. Audio distribution module:

[0141] Distance calculation: Based on the location information uploaded by the device, the distance between devices is calculated using a geometric algorithm.

[0142] Audio routing: Based on the distance calculation results, an audio routing table is dynamically generated to determine which devices need to receive specific audio content.

[0143] Real-time distribution: The processed audio data is distributed to the corresponding devices in real time to ensure low latency and high reliability of audio transmission.

[0144] 4. Deployment and testing.

[0145] 1. Equipment distribution and installation: Distribute the designed equipment to employees and instruct them to wear and use the equipment correctly. Correct wearing of the equipment ensures good sound insulation, sound reception, playback and other effects.

[0146] 2. Network environment construction: Build a suitable wireless network based on the specific conditions of the production environment to ensure that the equipment can be stably connected to the background system.

[0147] 3. Positioning base station deployment:

[0148] Base station installation: Install the positioning base station at a fixed location in the workshop, and perform calibration and debugging.

[0149] Coordinate map drawing: Draw a coordinate map based on the base station location and workshop layout to provide accurate positioning reference for the background system.

[0150] 5. Use and maintenance.

[0151] System monitoring: Establish a background monitoring system to monitor the operating status of equipment and network connection in real time, and promptly identify and resolve problems.

[0152] Regular maintenance: Clean, inspect and repair the equipment regularly to ensure long-term and stable operation of the equipment.

[0153] User feedback: Establish a user feedback mechanism to collect employees’ experience and opinions on the system, and continuously optimize and improve system functions.

[0154] Figure 4 A schematic diagram of an implementation flow of an audio processing method provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the system background (equivalent to an electronic device) sets parameters such as the broadcast range and volume, and the wearable device is equivalent to an audio playback device, such as the first audio playback device, the second audio playback device, and the third audio playback device in the aforementioned embodiment.

[0155] Figure 5 A schematic diagram of the corresponding relationship between distance and volume provided in an embodiment of the present application is shown in FIG. Figure 5As shown, the distance is 0-10 meters, and the volume is 10-0. The farther the distance, the lower the volume. The employee wears the device and turns it on, and communicates with the system background normally. The device starts to send positioning information to the system background at a fixed period of 1s / time to calculate the location information. The device waits for the sound to be received. After the employee speaks, the device transmits the audio to the background. The system background corresponds to the device position based on the received audio and calculates the distance between other devices and the device. According to the audio list of the sending device, send the audio file to the device with a non-0 sending volume. After receiving the audio file, the device plays it. Achieve an environment that simulates real-person communication.

[0156] The method provided in the embodiment of the present application can create a wearable device with comprehensive functions and stable performance and its management system to meet the needs of efficient communication and management in a noisy environment.

[0157] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0158] According to the aforementioned embodiments, the embodiments of the present application provide an audio processing device, and the modules included in the device, as well as the units included in the modules, can be implemented by a processor in a computer device; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing) or a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0159] The present application provides an audio processing device. Figure 6 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the audio processing device 500 includes:

[0160] The location information acquisition module 501 acquires the location information of each audio playback device;

[0161] A first receiver 502 receives a first audio captured by a first audio player among the audio players, and generates a second audio based on the first audio;

[0162] The control module 503 determines the distances from each second audio playback device, except the first audio playback device, to the first audio playback device based on the location information of each audio playback device; and determines the volume of the second audio played by each second audio playback device based on the distances;

[0163] The first transmitter 504 sends the second audio and the volume of playing the second audio to each of the second audio playing devices, so that each of the second audio playing devices plays the second audio at its own volume of playing the second audio.

[0164] In some embodiments, determining the volume of each of the second audio playback devices playing the second audio includes:

[0165] Obtaining a pre-established correspondence table between distance and volume;

[0166] The volume of the second audio played by each second audio playback device is determined from the correspondence table according to the distance from each second audio playback device to the first audio playback device.

[0167] In some embodiments, the distance in the correspondence table is divided into an adjacent first distance interval and a second distance interval; the volume corresponding to the first distance interval decreases as the corresponding distance increases, the volume corresponding to the second distance interval remains unchanged, and the first distance interval is smaller than the second distance interval, and the volume corresponding to the first distance interval is greater than the volume corresponding to the second distance interval.

[0168] In some embodiments, obtaining the location information of each audio playback device includes:

[0169] Obtaining the location information of the current audio playback device uploaded by each of the audio playback devices;

[0170] The location information is determined by each of the audio playback devices performing UWB communication with at least three base stations at fixed locations and positioning the current audio playback device based on the strength of the UWB signals fed back by each of the base stations.

[0171] In some embodiments, positioning the current audio playback device according to the strength of the UWB signal fed back by each base station includes:

[0172] The strength of the UWB signal fed back by each base station for the current audio playback device is input into the positioning model, so that the positioning model determines the position information of the current audio playback device based on the three-point positioning principle.

[0173] In some embodiments, generating the second audio based on the first audio includes:

[0174] performing noise suppression processing on the first audio to obtain a noise-suppressed first audio;

[0175] Perform speech enhancement processing on the noise-suppressed first audio to obtain the second audio.

[0176] In some embodiments, the audio processing device 500 is further configured to:

[0177] The second audio is sent to the first audio playback device, so that the first audio playback device plays the second audio at a default volume, where the default volume is the same as the volume of the first audio.

[0178] The present application embodiment provides an audio playback device. Figure 7 A schematic diagram of the structure of an audio playback device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the audio playback device 700 includes:

[0179] The location information acquisition module 701 acquires its own location information;

[0180] The second transmitter 702 transmits its own position information to the audio processing device;

[0181] The audio collection device 703 is used to collect audio and send the collected audio as the first audio to the audio processing device;

[0182] The second receiver 704 receives the second audio and the volume of the second audio sent by the audio processing device, and plays the second audio at the volume corresponding to the volume of the second audio played by itself.

[0183] The second audio is obtained by the audio processing device performing audio processing on the first audio collected by the first audio playback device and sent to the audio processing device, and the volume of playing the second audio is determined by the audio processing device based on the distance between the first audio playback device and the second audio playback device as each receiver.

[0184] In some embodiments, the audio playback device obtains its own location information, including:

[0185] After performing UWB communication with at least three base stations at fixed locations, the current audio playback device is positioned by the strength of the UWB signals fed back by each of the base stations to determine the location information of the current audio playback device.

[0186] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0187] in addition, Figure 6 The audio processing device shown may be a software unit, a hardware unit, or a combination of software and hardware units built into an existing electronic device, or may be integrated into the electronic device as an independent accessory, or may exist as an independent terminal device.

[0188] Figure 7 The audio playback device shown may be a software unit, a hardware unit, or a combination of software and hardware units built into an existing electronic device, or may be integrated into the electronic device as an independent accessory, or may exist as an independent terminal device.

[0189] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0190] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 3 of this embodiment may include: at least one processor 30 ( Figure 8 Only one processor 30 is shown in the figure), a memory 31, and a computer program 32 stored in the memory 31 and executable on at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above-mentioned method embodiments are implemented. For example, Figure 1 Steps S101 to S105 in the embodiment shown or, Figure 2 Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of modules 501 to 504 or modules 701 to 704 are shown.

[0191] Exemplarily, the computer program 32 may be divided into one or more modules / units, one or more modules / units are stored in the memory 31, and are executed by the processor 30 to complete the present application. One or more modules / units may be a series of computer program 32 instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3.

[0192] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program 32. When the computer program 32 is executed by the processor 30, the steps in the above-mentioned method embodiments can be implemented.

[0193] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. According to this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program 32, and the computer program 32 can be stored in a computer-readable storage medium, and the computer program 32 can implement the steps of the above-mentioned various method embodiments when executed by the processor 30. Among them, the computer program 32 includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the terminal, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0195] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0196] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0197] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0198] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0199] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An audio processing method, applied to an audio processing device, characterized in that: include: Get the location information of each audio playback device; Receiving a first audio captured by a first audio playback device among the audio playback devices, and generating a second audio based on the first audio; Based on the position information of each of the audio playback devices, determining the distances from each of the second audio playback devices except the first audio playback device to the first audio playback device; Based on the distance, determining the volume of the second audio played by each of the second audio playback devices; The second audio and the volume at which the second audio is played are sent to each of the second audio playback devices, so that each of the second audio playback devices plays the second audio at the volume at which it plays the second audio.

2. The method according to claim 1, characterized in that The determining the volume of each second audio playback device playing the second audio includes: Obtaining a pre-established correspondence table between distance and volume; The volume of the second audio played by each second audio playback device is determined from the correspondence table according to the distance from each second audio playback device to the first audio playback device.

3. The method according to claim 2, characterized in that The distances in the correspondence table are divided into adjacent first distance intervals and second distance intervals; the volume corresponding to the first distance interval decreases as the corresponding distance increases, the volume corresponding to the second distance interval remains unchanged, and the first distance interval is smaller than the second distance interval, and the volume corresponding to the first distance interval is greater than the volume corresponding to the second distance interval.

4. The method according to claim 1, characterized in that The obtaining of the location information of each audio playback device includes: Obtaining the location information of the current audio playback device uploaded by each of the audio playback devices; The location information is determined by each of the audio playback devices performing UWB communication with at least three base stations at fixed locations and locating the current audio playback device based on the strength of the UWB signals fed back by each of the base stations.

5. The method according to claim 4, characterized in that Positioning the current audio playback device according to the strength of the UWB signal fed back by each base station includes: The strength of the UWB signal fed back by each base station for the current audio playback device is input into the positioning model, so that the positioning model determines the position information of the current audio playback device based on the three-point positioning principle.

6. The method according to claim 1, characterized in that The generating the second audio based on the first audio comprises: performing noise suppression processing on the first audio to obtain a noise-suppressed first audio; Perform speech enhancement processing on the noise-suppressed first audio to obtain the second audio.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The second audio is sent to the first audio playback device, so that the first audio playback device plays the second audio at a default volume, where the default volume is the same as the volume of the first audio.

8. An audio playing method, applied to an audio playing device, characterized in that: The audio playing method comprises: The audio playback device acquires its own position information and transmits it to the audio processing device, and when collecting the first audio, acts as the first audio playback device to send the first audio to the audio processing device; The audio playback device, as a second audio playback device, receives the second audio and the volume of the second audio sent by the audio processing device, and plays the second audio at the volume corresponding to the volume of the second audio played by itself. The second audio is obtained by the audio processing device performing audio processing on the first audio collected by the first audio playback device and sent to the audio processing device, and the volume of playing the second audio is determined by the audio processing device based on the distance between the first audio playback device and the second audio playback device as the receiver.

9. The method according to claim 8, characterized in that The audio playback device obtains its own location information, including: After performing UWB communication with at least three base stations at fixed locations, the current audio playback device is positioned by the strength of the UWB signals fed back by each of the base stations to determine the location information of the current audio playback device.

10. An audio processing device, characterized in that: include: A location information acquisition module is used to acquire the location information of each audio playback device; A first receiver receives a first audio captured by a first audio player among the audio players, and generates a second audio based on the first audio; a control module, based on the position information of each of the audio playback devices, determining the distances from each of the second audio playback devices except the first audio playback device to the first audio playback device; and based on the distances, determining the volume of the second audio played by each of the second audio playback devices; The first transmitter sends the second audio and the volume of playing the second audio to each of the second audio playback devices, so that each of the second audio playback devices plays the second audio at the volume of the second audio played by itself.

11. An audio playback device, characterized in that: include: A location information acquisition module is used to obtain its own location information; A second transmitter transmits its position information to the audio processing device; An audio collection device, used for collecting sound and sending the collected audio as the first audio to the audio processing device; a second receiver, receiving the second audio and the volume of playing the second audio sent by the audio processing device, and playing the second audio at the volume corresponding to the volume of playing the second audio by itself, The second audio is obtained by the audio processing device performing audio processing on the first audio collected by the first audio playback device and sent to the audio processing device, and the volume of playing the second audio is determined by the audio processing device based on the distance between the first audio playback device and the second audio playback device as each receiver.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images