Audio separation method, human voice extraction method, audio processing system, electronic device and computer program product

Abstract

The invention provides an audio separation method, a human voice extraction method, an audio processing system, an electronic device and a computer program product, and the method comprises the steps: obtaining to-be-processed data from an audio collection device, the to-be-processed data comprises multi-sound-source audio data recorded by the audio acquisition equipment and attitude data in the process of recording the multi-sound-source audio data by the audio acquisition equipment; and separating audio data of a target sound source from the multi-sound-source audio data, and adjusting a beam direction to point to or approach the target sound source according to the attitude data in the separation process. By dynamically adjusting the beam direction, the target sound source can be locked in time, so that the audio separation effect is improved.

Description

Technical Field

[0001] The present application relates to the technical field of audio separation, and in particular to an audio separation method, a human voice extraction method, an audio processing system, an electronic device and a computer program product. Background Art

[0002] Audio separation technology is an important branch in the field of audio signal processing. It aims to extract independent sound source signals from mixed audio signals. Traditional audio separation methods mainly rely on voiceprint recognition technology, that is, to distinguish and separate sounds by analyzing the spectral characteristics and temporal characteristics of different sound sources. This audio separation method faces many challenges in practical applications, such as complex processes and poor noise resistance.

[0003] At present, in order to overcome the limitations of voiceprint separation technology, researchers have proposed an audio separation solution based on spatial information. The core idea of ​​this solution is to use the spatial positioning capability of the microphone array, take the predetermined direction as the beam direction pointing to the sound source, and then separate the audio signal according to this direction. The advantage of the spatial separation solution is that its process is relatively simple and does not require complex voiceprint analysis.

[0004] However, although the spatial separation scheme has certain advantages in simplifying the audio separation process, it also has an obvious defect, namely, the stability of the separation effect is poor. Summary of the invention

[0005] Based on the above technical status, the present application proposes an audio separation method, a human voice extraction method, an audio processing system, an electronic device and a computer program product.

[0006] According to a first aspect of an embodiment of the present application, there is provided an audio separation method, the method comprising:

[0007] Acquire data to be processed from an audio acquisition device, wherein the data to be processed includes: multi-source audio data recorded by the audio acquisition device and posture data of the audio acquisition device during the process of recording the multi-source audio data; separate the audio data of a target sound source from the multi-source audio data, and adjust the beam direction to point to or approach the target sound source according to the posture data during the separation process.

[0008] According to a second aspect of an embodiment of the present application, a method for extracting a human voice is provided, the method comprising:

[0009] Acquire data to be processed from an audio acquisition device, wherein the data to be processed includes: multi-source audio data recorded by the audio acquisition device and posture data during the recording of the multi-source audio data by the audio acquisition device; separate the audio data of a target sound source from the multi-source audio data, and adjust the beam direction to point to or approach the target sound source according to the posture data during the separation process to obtain the audio data of the target sound source; perform sound processing on the audio data of the target sound source to obtain target human voice data, wherein the sound processing includes: human voice extraction and / or human voice enhancement.

[0010] According to a third aspect of an embodiment of the present application, there is provided an audio separation method, which is applied to an audio acquisition device, wherein the audio acquisition device comprises: an audio acquisition unit having a sound recording function, and a posture acquisition unit having a posture detection function, and the method comprises:

[0011] In response to a start-up operation, multi-source audio data is recorded by the audio acquisition unit, and gesture data during the recording of the multi-source audio data by the audio acquisition unit is obtained by the gesture acquisition unit; the multi-source audio data and the gesture data are sent to a target device, so that the target device separates the audio data of a target sound source from the multi-source audio data, and adjusts a beam direction to point to or approach the target sound source according to the gesture data during the separation process.

[0012] According to a fourth aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor; the memory is connected to the processor and is used to store programs; the processor is used to implement the method described in the first aspect, the second aspect, or the third aspect by running the program in the memory.

[0013] According to a fifth aspect of the embodiments of the present application, there is provided an electronic device, an audio processing system, comprising an audio acquisition device and a target device;

[0014] Wherein, the audio acquisition device is configured to perform the audio separation method as described in the third aspect;

[0015] and / or,

[0016] The target device is configured to execute the audio separation method as described in the first aspect, or to execute the human voice extraction method as described in the second aspect.

[0017] According to a sixth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first aspect, the second aspect, or the third aspect is implemented.

[0018] According to a seventh aspect of an embodiment of the present application, a computer program product is provided, comprising: a computer program, which, when executed by a processor, implements the method described in the first aspect, the second aspect, or the third aspect.

[0019] In an embodiment of the present application, the acquired data to be processed includes multi-source audio data recorded by an audio acquisition device and posture data during the process of recording multi-source audio data by the audio acquisition device. Therefore, the posture change of the audio acquisition device during the process of recording multi-source audio data can be determined through the posture data. Then, the audio data of the target sound source is separated from the multi-source audio data, and the beam direction is adjusted to point to or approach the target sound source according to the posture data during the separation process, so as to achieve dynamic locking of the target sound source. Therefore, during the process of recording multi-source audio data, even if the posture of the audio acquisition device changes, the audio data of the target sound source can be accurately separated, thereby improving the stability of the audio separation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0021] Figure 1 One of the flowcharts of an audio separation method provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a flow chart of a method for extracting human voice provided in an embodiment of the present application;

[0023] Figure 3 A second flowchart of an audio separation method provided in an embodiment of the present application;

[0024] Figure 4 One of the structural schematic diagrams of an audio separation device provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of a human voice extraction device provided in an embodiment of the present application;

[0026] Figure 6 A second structural diagram of an audio separation device provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the architecture of an audio processing system provided in an embodiment of the present application;

[0028] Figure 8A schematic diagram of a process of obtaining a human voice by a cloud device provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] Overview

[0032] As described in the background technology, the audio separation solution based on spatial information mainly relies on the multi-microphone array in the audio acquisition device. First, the difference in the time when multiple microphones receive the same sound source is determined, and then the closest one is selected from multiple fixed beam directions as the beam direction pointing to the sound source based on these differences. During the entire separation process, the fixed beam direction will continue to be selected as the direction of the sound source to achieve the location of the sound source. Finally, the audio signal is separated according to the location result of the sound source.

[0033] However, the spatial posture of the audio acquisition device that records audio may change during the recording process. For example, if the wearer of the audio acquisition device walks, sits up, etc. during the recording process, the spatial posture of the audio acquisition device may change significantly. At this time, if the audio separation is continued according to the fixed beam direction, the separation result will inevitably be affected.

[0034] In view of the above-mentioned technical status, the inventor proposes that the posture data of the audio acquisition device can be synchronously recorded during the process of recording audio data, and then in the process of separating the audio data, the beam direction can be dynamically adjusted using the posture data to achieve dynamic locking of the sound source. The embodiment of the present application proposes an audio separation method. The acquired data to be processed includes multi-source audio data recorded by the audio acquisition device and the posture data of the audio acquisition device during the process of recording the multi-source audio data. Therefore, the posture change of the audio acquisition device during the process of recording the multi-source audio data can be determined by the posture data. Then, the audio data of the target sound source is separated from the multi-source audio data, and the beam direction is adjusted to point to or approach the target sound source according to the posture data during the separation process to achieve dynamic locking of the target sound source. Therefore, in the process of recording multi-source audio data, even if the posture of the audio acquisition device changes, the audio data of the target sound source can be accurately separated, which improves the stability of the audio separation effect.

[0035] Exemplary Methods

[0036] See also Figure 1 In an exemplary embodiment, an audio separation method is provided, which can be applied to any electronic device with audio data processing capability, for example, the electronic device can be a server, a computer, a smart phone, but is not limited thereto. The electronic device can even be a dedicated recording device with audio collection function. The method can include:

[0037] S101: Acquire data to be processed from an audio acquisition device.

[0038] In this step, the audio acquisition device may be any electronic device with a recording function. In some embodiments, the audio acquisition device may be a wearable audio acquisition device. For example, the audio acquisition device may be a wearable smart ID card, which can be used as an ID card by employees and can also record audio during their work.

[0039] The data to be processed include: multi-source audio data recorded by an audio acquisition device. Among them, the multi-source audio data is the object that needs to be audio separated, and it contains sounds from multiple sources. The multiple sound sources include but are not limited to people and animals. For example, the multiple sound sources include multiple different people. Regarding the data format of the multi-source audio data, it can be any audio data format. For example, the multi-source audio data can be audio data formed by sub-band coding (SBC) technology. Regarding the data content or semantic content of the multi-source audio data, it is not limited here, and it can be any semantic content.

[0040] The data to be processed also includes: posture data of the audio acquisition device during recording of multi-source audio data. The posture data can represent the posture of the audio acquisition device in space. It can be understood that the posture data in the data to be processed is continuous posture data within a time period. The time period is the time period for recording multi-source audio data. Therefore, the posture data in the data to be processed can be used to determine the posture or posture change of the audio acquisition device during recording of multi-source audio data.

[0041] S102: Separating the audio data of the target sound source from the audio data of multiple sound sources, and adjusting the beam direction to point to or approach the target sound source according to the posture data during the separation process.

[0042] In this step, the target sound source may be any sound source in the multi-source audio data. In some embodiments, the user's input may be received, and then the sound source to be separated may be determined based on the user's input, and finally the sound source to be separated may be used as the target sound source. In some embodiments, the target sound source may be a target person, and the audio data of the separated target sound source is the voice of the separated target person.

[0043] When separating audio data, a spatial separation technique can be used. In some embodiments, the beam direction is first determined, and then the beam direction is regarded as indicating the direction of the target sound source to achieve the positioning of the target sound source. Finally, based on the positioning result / beam direction of the target sound source, the audio data of the target sound source is separated from the multi-source audio data. In this embodiment, after determining the beam direction, the fixed beam direction is not always used to separate the audio data, but the beam direction is dynamically adjusted using the posture data so that the beam direction can always accurately point to or approach the target sound source to achieve the locking of the target sound source. Finally, the audio data of the target sound source is separated while the target sound source is continuously locked. In this way, even if the posture of the audio acquisition device changes significantly during the recording of multi-source audio data, the target sound source can be locked in time, instead of using the original position of the target sound source to separate the audio. For example, in a scene where the audio acquisition device is used dynamically to record, when separating the audio of the target sound source, the target sound source can always be locked, thereby achieving a better separation effect.

[0044] In an embodiment of the present application, the acquired data to be processed includes multi-source audio data recorded by an audio acquisition device and posture data during the process of recording multi-source audio data by the audio acquisition device. Therefore, the posture change of the audio acquisition device during the process of recording multi-source audio data can be determined through the posture data. Then, the audio data of the target sound source is separated from the multi-source audio data, and the beam direction is adjusted to point to or approach the target sound source according to the posture data during the separation process, so as to achieve dynamic locking of the target sound source. Therefore, during the process of recording multi-source audio data, even if the posture of the audio acquisition device changes, the audio data of the target sound source can be accurately separated, thereby improving the stability of the audio separation effect.

[0045] In order to avoid frequent adjustment of the beam direction, which results in excessive hardware device overhead or increases the complexity of the separation process, in some embodiments of the present application, after obtaining the data to be processed from the audio acquisition device, the method further includes:

[0046] According to the posture data, the posture changes of the audio acquisition device in each recording period are determined; according to the posture changes in each recording period, the target recording period that needs to adjust the beam direction is determined.

[0047] During the separation process, the beam direction is adjusted to point to or approach the target sound source according to the attitude data, including:

[0048] During the separation process, the beam direction within the target recording period is adjusted to point to or approach the target sound source according to the posture data of the target recording period.

[0049] It should be noted that during the recording of multi-source audio data, the posture of the audio acquisition device may change frequently. For example, during the recording process of a wearable audio acquisition device, the user walks, sits up, etc., which causes the posture of the audio acquisition device to change frequently. If the beam direction needs to be adjusted every time a change occurs, it will inevitably increase the burden on the hardware device, increase the complexity of the separation process, and even affect the separation speed.

[0050] To this end, certain screening conditions can be set to screen the recording time periods that need to adjust the beam direction, that is, the target recording time periods. Then, the beam direction is adjusted only in the target recording time period. Among them, the number of target recording time periods is not limited, and it can be one or more. In some embodiments, when the number of target recording time periods is multiple, the beam direction between any two adjacent target recording time periods is consistent with the beam direction in the previous target recording time period. For example, the overall time period of the recording process starts from 0 minutes and 0 seconds and ends at 10 minutes and 20 seconds. If the audio acquisition device has undergone significant changes in the recording time period from 1 minute and 20 seconds to 1 minute and 30 seconds and the recording time period from 5 minutes and 10 seconds to 6 minutes and 20 seconds. Then the target recording time period includes a first target recording time period from 1 minute and 20 seconds to 1 minute and 30 seconds, and a second target recording time period from 5 minutes and 10 seconds to 6 minutes and 20 seconds. In the process of separating audio, the beam direction between 0 minutes and 0 seconds and 1 minute and 19 seconds can be determined based on the spatial separation technology. In the first target recording period, the beam direction is adjusted to the first beam direction, and the beam direction from 1 minute 31 seconds to 5 minutes 9 seconds is still the first beam direction. In the second target recording period, the beam direction is adjusted to the second beam direction, and the beam direction from 6 minutes 21 seconds to 10 minutes 20 seconds is still the second beam direction.

[0051] The posture change includes: the performance of the audio acquisition device in terms of change amplitude and change frequency during the recording period. In some embodiments, the filtering condition can be set based on the scene where the posture of the audio acquisition device changes. For example, the wearer sitting up will cause the posture of the audio acquisition device to change. The filtering condition set accordingly can be that the posture change indicates that the audio acquisition device has undergone a large change in a short period of time.

[0052] In some embodiments, multiple neural network models can be pre-trained for audio separation of multi-source audio data recorded in different postures. During the separation process, when the beam direction in the target recording period is adjusted to point to or approach the target sound source according to the posture data of the target recording period, the corresponding neural network model can be selected according to the posture data of the target recording period, and then the selected neural network model is used for audio separation.

[0053] In an embodiment of the present application, when the audio acquisition device changes frequently, the target recording period can be screened to adjust the beam direction, thereby avoiding frequent adjustment of the beam direction, which results in excessive hardware device overhead or increases the complexity of the separation process.

[0054] In some embodiments of the present application, the posture change includes: time domain features and / or frequency domain features of posture data; multi-source audio data is recorded when the target person wears an audio acquisition device;

[0055] According to the posture changes in each recording period, determine the target recording period that needs to adjust the beam direction, including:

[0056] According to the time domain characteristics and / or frequency domain characteristics of the posture data in each recording period, the behavior label corresponding to the target person in each recording period is determined; the recording period corresponding to the target behavior label is determined as the target recording period; wherein the target behavior label includes a behavior label that is the same as any dangerous behavior label in the behavior label library.

[0057] It should be noted that since the target person wears an audio acquisition device, the posture of the audio acquisition device will change differently when the target person performs different behaviors during the recording of multi-source audio data. Conversely, different posture changes of the audio acquisition device correspond to different behaviors of the target person. Therefore, the behavior of the target person is closely related to the posture of the audio acquisition device.

[0058] This embodiment uses the time domain features and / or frequency domain features of the posture data to reversely deduce the behavior of the target person. If continuing to separate the audio according to the original beam direction after implementing the behavior will cause the separation effect to deteriorate, then the behavior is defined as a dangerous behavior, and the recording period in which the behavior occurs is regarded as the target recording period. Here, a label library is pre-set, and the behavior labels of dangerous behaviors are stored in the label library. Therefore, after determining the behavior of the target person, the target recording period is determined based on the comparison result of the behavior label of the behavior with the label library. For example, the determined behavior includes behavior A, and the label corresponding to behavior A is a. If there is a behavior label a in the label library, then behavior A is regarded as a dangerous behavior, and the recording period in which it occurs is regarded as the target recording period.

[0059] In the embodiment of the present application, the behavior of the target person is derived with the help of the time domain characteristics and frequency domain characteristics of the posture data, and then the target recording period is determined based on the comparison results of the behavior label of the target person with the label library. The process is simple.

[0060] In some embodiments of the present application, determining the target recording period for which the beam direction needs to be adjusted according to the posture change in each recording period includes:

[0061] According to the posture changes in each recording period, the recording period with low-frequency and large-scale posture changes is determined; the recording period with low-frequency and large-scale posture changes is determined as the target recording period.

[0062] It should be noted that multi-source audio data can be recorded when the target person wears an audio acquisition device. Therefore, when the target person performs different behaviors during the recording of multi-source audio data, the posture of the audio acquisition device will change differently. For example, when the target person performs behaviors such as walking, going up and down stairs, the corresponding posture data shows a small-amplitude and high-frequency change in posture. Since this behavior has little effect on the accuracy of audio separation, the beam direction can be left unchanged during the occurrence of this behavior. When the target person changes his standing posture, sitting posture, wears the device improperly, falls, etc., the corresponding posture data shows a low-frequency and large-amplitude change in posture. Since this behavior has a great impact on the accuracy of audio separation, it is necessary to adjust the beam direction during the occurrence of this behavior, that is, to determine the period of this behavior as the target recording period.

[0063] In the embodiment of the present application, the beam direction can be adjusted when the posture changes at a low frequency and a large amplitude, thereby avoiding a significant impact on the audio separation accuracy.

[0064] In some embodiments of the present application, obtaining data to be processed from an audio acquisition device includes:

[0065] Receiving an encoded file from an audio acquisition device, wherein the encoded file is a single file formed by encoding multi-source audio data recorded by the audio acquisition device and posture data in a process of the audio acquisition device recording the multi-source audio data;

[0066] The encoded file is decoded to obtain multi-source audio data and posture data.

[0067] It should be noted that the device for audio separation and the audio acquisition device can be different electronic devices. In this way, the audio acquisition device is required to transmit relevant data to the device for audio separation. In some embodiments, the audio acquisition device is required to upload relevant data to the cloud, and the audio separation device downloads relevant data from the cloud. For the convenience of uploading and downloading, the audio acquisition device can encode multi-source audio data and posture data into a single file, and subsequently upload and download the single file. The audio separation device can decode the single file to obtain multi-source audio data and posture data. This application does not limit the encoding method.

[0068] In the embodiment of the present application, multi-source audio data and posture data are transmitted in the form of a single file, which improves the convenience of transmission.

[0069] In some embodiments of the present application, the step of obtaining posture data in the data to be processed includes:

[0070] Obtaining the original data output by the inertial measurement unit during the process of the audio acquisition device recording multi-source audio data;

[0071] The raw data is converted into posture data representing the posture of the audio acquisition device.

[0072] It should be noted that the inertial measurement unit can measure the three-axis attitude angle (or angular velocity) and acceleration of an object, which will not be described in detail here. The audio acquisition device is associated with an inertial measurement unit (IMU for short). For example, the audio acquisition device is provided with an IMU. In this embodiment, the process of converting the raw data output by the IMU into attitude data occurs in the audio separation device. In this way, the pressure on the audio acquisition device and the requirements for the audio acquisition device are reduced. For example, the audio acquisition device only needs to have a recording function and an IMU detection function.

[0073] In the embodiment of the present application, the device for audio separation converts the raw data output by the IMU into posture data, thereby alleviating the pressure on the audio acquisition device and reducing the requirements for the audio acquisition device.

[0074] To improve the quality of the posture data, in some embodiments of the present application, after converting the original data into posture data representing the posture of the audio acquisition device, the method further includes:

[0075] The posture data is filtered to remove data representing data noise and / or glitches.

[0076] It should be noted that the gesture data may contain data noise and glitches caused by various external factors (hardware factors of the audio acquisition device or collisions with other objects during the operation of the audio acquisition device, etc.). These data noise and glitches reduce the quality of the gesture data. For this reason, after obtaining the gesture data, data filtering is first performed to filter out the data representing data noise and / or glitches to obtain gesture data with better quality, which will be used for audio separation later.

[0077] In the embodiment of the present application, data filtering is performed on the posture data to eliminate the influence of data noise and burrs, thereby improving the quality of the posture data.

[0078] See also Figure 2 According to another aspect of the present application, in an exemplary embodiment, a method for extracting human voice is provided, the method comprising:

[0079] S201: Acquire data to be processed from an audio acquisition device, wherein the data to be processed includes: multi-source audio data recorded by the audio acquisition device and posture data during the process of the audio acquisition device recording the multi-source audio data;

[0080] S202: Separating the audio data of the target sound source from the audio data of multiple sound sources, and adjusting the beam direction to point to or approach the target sound source according to the posture data during the separation process to obtain the audio data of the target sound source;

[0081] S203: Performing sound processing on the audio data of the target sound source to obtain target human voice data, wherein the sound processing includes: human voice extraction and / or human voice enhancement.

[0082] It should be noted that, with respect to S201-S202, reference may be made to the similar contents of S101-S102 in the above-mentioned embodiment, which will not be described in detail here. In this embodiment, the target sound source is any designated person, and after the audio data of the target sound source is separated, the audio data usually contains interference such as noise. Therefore, through voice extraction and / or voice enhancement, the noise interference can be filtered out to obtain target voice data with better quality, further improving the audio separation effect.

[0083] In some embodiments, after obtaining the target voice data, the identity of the target person in the target voice data can be identified, and different business processes can be performed based on the identity recognition results. For example, if the identity recognition result is a staff member, the target voice data is used to perform work quality inspection to examine whether the staff member's work method is appropriate or in compliance with the regulations. If the identity recognition result is a customer, the target voice data is used to create a work persona profile, and corresponding experts are arranged to provide services based on the persona profile.

[0084] Optionally, after acquiring the data to be processed from the audio acquisition device, the method further includes:

[0085] According to the posture data, determine the posture change of the audio acquisition device in each recording period; according to the posture change of each recording period, determine the target recording period that needs to adjust the beam direction;

[0086] During the separation process, the beam direction is adjusted to point to or approach the target sound source according to the attitude data, including:

[0087] During the separation process, the beam direction within the target recording period is adjusted to point to or approach the target sound source according to the posture data of the target recording period.

[0088] Optionally, the posture change includes: time domain features and / or frequency domain features of posture data; multi-source audio data is recorded when the target person wears an audio acquisition device;

[0089] According to the posture changes in each recording period, determine the target recording period that needs to adjust the beam direction, including:

[0090] According to the time domain characteristics and / or frequency domain characteristics of the posture data in each recording period, the behavior label corresponding to the target person in each recording period is determined; the recording period corresponding to the target behavior label is determined as the target recording period; wherein the target behavior label includes a behavior label that is the same as any dangerous behavior label in the behavior label library.

[0091] Optionally, according to the posture change in each recording period, determining the target recording period in which the beam direction needs to be adjusted includes:

[0092] According to the posture changes in each recording period, the recording period with low-frequency and large-scale posture changes is determined; the recording period with low-frequency and large-scale posture changes is determined as the target recording period.

[0093] Optionally, obtaining data to be processed from an audio acquisition device includes:

[0094] Receive an encoded file from an audio acquisition device, wherein the encoded file is a single file formed by encoding multi-source audio data recorded by the audio acquisition device and posture data in a process of the audio acquisition device recording the multi-source audio data; decode the encoded file to obtain the multi-source audio data and posture data.

[0095] Optionally, the step of obtaining posture data in the data to be processed includes:

[0096] The original data output by the inertial measurement unit when the audio acquisition device records multi-source audio data is obtained; the original data is converted into posture data representing the posture of the audio acquisition device.

[0097] Optionally, after converting the original data into posture data representing the posture of the audio acquisition device, the method further includes:

[0098] The posture data is filtered to remove data representing data noise and / or glitches.

[0099] The human voice extraction method provided in this embodiment belongs to the same application concept as the audio separation method provided in the above embodiment of this application. For technical details not fully described in this embodiment, please refer to the specific processing content of the audio separation method provided in the above embodiment of this application, which will not be repeated here.

[0100] See also Figure 3 According to another aspect of the present application, in an exemplary embodiment, an audio separation method is provided, which is applied to an audio acquisition device, wherein the audio acquisition device includes: an audio acquisition unit with a sound recording function, a posture acquisition unit with a posture detection function,

[0101] S301: In response to a start operation, the audio data from multiple sound sources are recorded by an audio acquisition unit, and the posture data in the process of recording the audio data from multiple sound sources by the audio acquisition unit is obtained by a posture acquisition unit.

[0102] S302: Sending multi-source audio data and gesture data to a target device, so that the target device separates the audio data of a target sound source from the multi-source audio data, and adjusts the beam direction to point to or approach the target sound source according to the gesture data during the separation process.

[0103] It should be noted that the audio acquisition device may be a wearable audio acquisition device. For example, the audio acquisition device may be a wearable smart ID card, which can be worn by enterprise employees and used as an ID card and can also record during the employee's work. The posture acquisition unit includes but is not limited to an IMU. For example, the posture data may be the raw data output by the IMU.

[0104] Multi-source audio data is an object that needs to be separated from audio, and it contains sounds from multiple sources. The multiple sound sources include, but are not limited to, people and animals. For example, the multiple sound sources include multiple different people. Regarding the data format of the multi-source audio data, it can be any audio data format, for example, the multi-source audio data can be audio data formed by sub-band coding (SBC) technology. Regarding the data content or semantic content of the multi-source audio data, it is not limited here, and it can be any semantic content.

[0105] The posture data can represent the posture of the audio acquisition device in space. It can be understood that the posture data in the data to be processed is continuous posture data within a time period. The time period is the time period for recording multi-source audio data. Therefore, the posture data in the data to be processed can be used to determine the posture of the audio acquisition device or the posture change of the audio acquisition device when recording multi-source audio data.

[0106] In some embodiments, the audio acquisition unit and the posture acquisition unit can be controlled by the same start button, and can start and end work synchronously. Further, the audio acquisition unit and the posture acquisition unit use the same clock device for timing, or, after the audio acquisition unit and the posture acquisition unit finish working, the data generated by the two are time-aligned.

[0107] In an embodiment of the present application, the posture of the device during the audio recording process is taken into account, so that during the audio separation process, the beam direction is dynamically adjusted to point to or approach the sound source according to the posture data, so that the beam direction is locked to the sound source in real time during the separation process, and the separation effect is greatly enhanced.

[0108] In some embodiments of the present application, sending multi-source audio data and gesture data includes:

[0109] Encode multi-source audio data and gesture data into a single file and send the single file.

[0110] It should be noted that the target device for audio separation is a different electronic device from the audio acquisition device. In this way, the audio acquisition device is required to transmit the relevant data to the target device. In some embodiments, the audio acquisition device is required to upload the relevant data to the cloud, and the target device downloads the relevant data from the cloud. For the convenience of uploading and downloading, the audio acquisition device can encode the multi-source audio data and posture data into a single file, and subsequently upload and download the single file. The target device can decode the single file to obtain the multi-source audio data and posture data. This application does not limit the encoding method.

[0111] In the embodiment of the present application, multi-source audio data and posture data are transmitted in the form of a single file, which improves the convenience of transmission.

[0112] Exemplary Devices

[0113] Accordingly, the present application embodiment also provides an audio separation device, see Figure 4 As shown, the device comprises:

[0114] The data acquisition module 401 is used to acquire the data to be processed from the audio acquisition device, wherein the data to be processed includes: multi-source audio data recorded by the audio acquisition device and posture data in the process of the audio acquisition device recording the multi-source audio data.

[0115] The audio separation module 402 is used to separate the audio data of the target sound source from the audio data of multiple sound sources, and adjust the beam direction to point to or approach the target sound source according to the posture data during the separation process.

[0116] In some embodiments of the present application, the device further comprises:

[0117] The recording period screening module is used to determine the posture change of the audio acquisition device in each recording period according to the posture data; according to the posture change of each recording period, determine the target recording period that needs to adjust the beam direction;

[0118] The audio separation module 402 is specifically used to adjust the beam direction in the target recording period to point to or approach the target sound source according to the posture data of the target recording period during the separation process.

[0119] In some embodiments of the present application, the posture change includes: time domain features and / or frequency domain features of posture data; multi-source audio data is recorded when the target person wears an audio acquisition device;

[0120] The recording period screening module is specifically used to determine the behavior label corresponding to the target person in each recording period according to the time domain characteristics and / or frequency domain characteristics of the posture data in each recording period; the recording period corresponding to the target behavior label is determined as the target recording period; wherein the target behavior label includes a behavior label that is the same as any dangerous behavior label in the behavior label library.

[0121] In some embodiments of the present application, the recording period screening module is specifically used to determine the recording period with low-frequency and large-scale posture changes according to the posture changes in each recording period; and determine the recording period with low-frequency and large-scale posture changes as the target recording period.

[0122] In some embodiments of the present application, the data acquisition module 401 is specifically used to receive an encoded file from an audio acquisition device, wherein the encoded file is a single file formed by encoding multi-source audio data recorded by the audio acquisition device and posture data in the process of recording the multi-source audio data by the audio acquisition device; the encoded file is decoded to obtain the multi-source audio data and posture data.

[0123] In some embodiments of the present application, the step of the data acquisition module 401 acquiring the posture data in the data to be processed includes:

[0124] The original data output by the inertial measurement unit when the audio acquisition device records multi-source audio data is obtained; the original data is converted into posture data representing the posture of the audio acquisition device.

[0125] In some embodiments of the present application, the device further comprises:

[0126] The data filtering module is used to filter the posture data to remove data representing data noise and / or glitches.

[0127] The audio separation device provided in this embodiment belongs to the same application concept as the audio separation method provided in the above embodiments of this application, and can execute the audio separation method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the audio separation method provided in the above embodiments of this application, and will not be repeated here.

[0128] Accordingly, the present application embodiment also provides a human voice extraction device, see Figure 5 As shown, the device comprises:

[0129] The data acquisition module 401 is used to acquire the data to be processed from the audio acquisition device, wherein the data to be processed includes: multi-source audio data recorded by the audio acquisition device and posture data in the process of the audio acquisition device recording the multi-source audio data.

[0130] The audio separation module 402 is used to separate the audio data of the target sound source from the audio data of multiple sound sources, and adjust the beam direction to point to or approach the target sound source according to the posture data during the separation process.

[0131] The vocal processing module 403 is used to perform sound processing on the audio data of the target sound source to obtain target vocal data, wherein the sound processing includes: vocal extraction and / or vocal enhancement.

[0132] In some embodiments of the present application, the device further comprises:

[0133] The recording period screening module is used to determine the posture change of the audio acquisition device in each recording period according to the posture data; according to the posture change of each recording period, determine the target recording period that needs to adjust the beam direction;

[0134] The audio separation module 402 is specifically used to adjust the beam direction in the target recording period to point to or approach the target sound source according to the posture data of the target recording period during the separation process.

[0135] In some embodiments of the present application, the posture change includes: time domain features and / or frequency domain features of posture data; multi-source audio data is recorded when the target person wears an audio acquisition device;

[0136] The recording period screening module is specifically used to determine the behavior label corresponding to the target person in each recording period according to the time domain characteristics and / or frequency domain characteristics of the posture data in each recording period; the recording period corresponding to the target behavior label is determined as the target recording period; wherein the target behavior label includes a behavior label that is the same as any dangerous behavior label in the behavior label library.

[0137] In some embodiments of the present application, the recording period screening module is specifically used to determine the recording period with low-frequency and large-scale posture changes according to the posture changes in each recording period; and determine the recording period with low-frequency and large-scale posture changes as the target recording period.

[0138] In some embodiments of the present application, the data acquisition module 401 is specifically used to receive an encoded file from an audio acquisition device, wherein the encoded file is a single file formed by encoding multi-source audio data recorded by the audio acquisition device and posture data in the process of recording the multi-source audio data by the audio acquisition device; the encoded file is decoded to obtain the multi-source audio data and posture data.

[0139] In some embodiments of the present application, the step of the data acquisition module 401 acquiring the posture data in the data to be processed includes:

[0140] The original data output by the inertial measurement unit when the audio acquisition device records multi-source audio data is obtained; the original data is converted into posture data representing the posture of the audio acquisition device.

[0141] In some embodiments of the present application, the device further comprises:

[0142] The data filtering module is used to filter the posture data to remove data representing data noise and / or glitches.

[0143] The human voice extraction device provided in this embodiment belongs to the same application concept as the human voice extraction method provided in the above embodiments of this application, and can execute the human voice extraction method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the human voice extraction method provided in the above embodiments of this application, and will not be repeated here.

[0144] Accordingly, the embodiment of the present application further provides an audio separation device, which is applied to an audio acquisition device, wherein the audio acquisition device comprises: an audio acquisition unit with a sound recording function, a posture acquisition unit with a posture detection function, see Figure 6 As shown, the device comprises:

[0145] A response module 601 is used for, in response to the start operation, obtaining multi-source audio data through the audio acquisition unit, and obtaining, through the posture acquisition unit, posture data in the process of recording the multi-source audio data by the audio acquisition unit;

[0146] The sending module 602 is used to send multi-source audio data and gesture data to the target device, so that the target device separates the audio data of the target sound source from the multi-source audio data, and adjusts the beam direction to point to or approach the target sound source according to the gesture data during the separation process.

[0147] In some embodiments of the present application, the sending module 602 is specifically used to encode the multi-source audio data and gesture data into a single file, and send the single file.

[0148] The audio separation device provided in this embodiment belongs to the same application concept as the audio separation method provided in the above embodiments of this application, and can execute the audio separation method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the audio separation method provided in the above embodiments of this application, and will not be repeated here.

[0149] It should be understood that the modules in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the unit in the device can be implemented in the form of a hardware circuit, and the functions of some or all units can be realized by designing the hardware circuit. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of a hardware circuit, or in part by a processor calling software, and the remaining part is implemented in the form of a hardware circuit.

[0150] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0151] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0152] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0153] Exemplary Systems

[0154] Accordingly, an embodiment of the present application further provides an audio processing system, the audio processing system comprising: an audio acquisition device and a target device;

[0155] The audio acquisition device is configured to execute the audio separation method applied to the audio acquisition device provided in the above embodiments;

[0156] and / or,

[0157] The target device is configured to execute the audio separation method provided in the above embodiments, or to execute the human voice extraction method provided in the above embodiments.

[0158] The audio processing system provided in this embodiment belongs to the same application concept as the audio separation method and the human voice extraction method provided in the above embodiments of this application. For technical details not fully described in this embodiment, please refer to the specific processing contents of the audio separation method and the human voice extraction method provided in the above embodiments of this application, which will not be repeated here.

[0159] For ease of understanding, the audio processing system is described below by way of example. Figure 7 As shown, the audio processing system includes a terminal device 701 and a cloud device 702.

[0160] After receiving the user's start-up operation, the terminal device 701 simultaneously performs IMU data collection and recording, then encodes the IMU data and the audio data, and encodes the two into a single file, stores the single file locally and uploads it to the cloud device 702.

[0161] The cloud device 702 first parses a single file to obtain separate audio data and IMU data. The IMU data is solved and processed to extract behavioral features related to the wearer's behavior; the audio data is decoded by SBC to obtain decoded audio data. The behavioral data and the decoded audio data are used as two inputs, and the audio data of the target sound source is separated using a spatial separation algorithm. If the audio data recorded by the terminal device 701 contains sales audio and customer audio, the separated audio data will contain two copies, one containing only the sales audio of the sales sound source, and the other containing only the customer audio of the customer sound source. After the two copies of audio data are processed by voice extraction and enhancement, the sales voice and customer voice are obtained. Then they are transcribed separately for quality inspection and portrait generation.

[0162] The cloud device 702 calculates and processes the IMU data, separates the audio data, and obtains the human voice. Figure 8 As shown, including:

[0163] S801: Start.

[0164] S802: Data filtering, converting the IMU data from the raw data state into attitude data, and filtering out the data noise and burrs therein.

[0165] S803: extracting behavior features, extracting behavior features from the posture data obtained in S802, and determining the behavior features of each recording period, wherein the behavior features include: behavior features of posture with small amplitude and high frequency changes, and behavior features of posture with low frequency and large amplitude changes.

[0166] S804: Dynamic beam adjustment. In the process of separating the audio data, the target recording period in which the beam direction needs to be dynamically adjusted is determined based on the behavior characteristics, and the beam direction is adjusted to point to or approach the target sound source based on the posture data during the target recording period.

[0167] S805: vocal enhancement: vocal extraction and enhancement are performed on the audio data separated in S804 to obtain vocal data.

[0168] S806: End.

[0169] Exemplary Electronic Devices

[0170] The present application embodiment provides an electronic device, see Fig. 9 As shown, the device includes:

[0171] Memory 900 and processor 910;

[0172] The memory 900 is connected to the processor 910 and is used to store programs;

[0173] The processor 910 is used to implement the audio separation method or the human voice extraction method disclosed in any of the above embodiments by running the program stored in the memory 900.

[0174] Specifically, the electronic device may further include: a bus, a communication interface 920 , an input device 930 and an output device 940 .

[0175] The processor 910, the memory 900, the communication interface 920, the input device 930 and the output device 940 are connected to each other via a bus.

[0176] A bus may include a pathway that transfers information between components of a computer system.

[0177] The processor 910 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0178] The processor 910 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0179] The memory 900 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 900 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0180] The input device 930 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0181] Output device 940 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0182] The communication interface 920 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0183] The processor 910 executes the program stored in the memory 900 and calls other devices, which can be used to implement each step of any audio separation method or human voice extraction method provided in the above embodiments of the present application.

[0184] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the audio separation method or human voice extraction method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the embodiment introduction of the above-mentioned audio separation method or human voice extraction method.

[0185] Exemplary computer program products and storage media

[0186] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the audio separation method or human voice extraction method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0187] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0188] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the audio separation method or human voice extraction method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0189] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0190] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0191] The steps in the methods of each embodiment of the present application can be adjusted in sequence, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0192] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0193] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0194] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0195] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0196] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0197] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0198] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0199] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio separation method, characterized in that: The method comprises: Acquire data to be processed from an audio acquisition device, wherein the data to be processed includes: multi-source audio data recorded by the audio acquisition device and posture data in a process in which the audio acquisition device records the multi-source audio data; The audio data of the target sound source is separated from the multi-sound source audio data, and during the separation process, the beam direction is adjusted according to the posture data to point to or approach the target sound source.

2. The method according to claim 1, characterized in that After acquiring the data to be processed from the audio acquisition device, the method further includes: Determining, according to the posture data, a posture change of the audio acquisition device in each recording period; According to the posture changes in each recording period, determine the target recording period in which the beam direction needs to be adjusted; During the separation process, adjusting the beam direction to point to or approach the target sound source according to the posture data includes: During the separation process, the beam direction in the target recording period is adjusted to point to or approach the target sound source according to the posture data of the target recording period.

3. The method according to claim 2, characterized in that The posture change includes: the time domain characteristics and / or frequency domain characteristics of the posture data; the multi-source audio data is recorded when the target person wears the audio acquisition device; According to the posture changes in each recording period, determine the target recording period that needs to adjust the beam direction, including: Determine the behavior label corresponding to the target person in each recording period according to the time domain characteristics and / or frequency domain characteristics of the posture data in each recording period; The recording period corresponding to the target behavior tag is determined as the target recording period; wherein the target behavior tag includes a behavior tag that is the same as any dangerous behavior tag in the behavior tag library.

4. The method according to claim 2, characterized in that: According to the posture changes in each recording period, determine the target recording period that needs to adjust the beam direction, including: According to the posture changes in each recording period, determine the recording period with low-frequency and large-scale posture changes; The recording period in which the posture changes with low frequency and large amplitude is determined as the target recording period.

5. The method according to claim 1, characterized in that Obtain the data to be processed from the audio acquisition device, including: Receiving an encoded file from the audio acquisition device, wherein the encoded file is a single file formed by encoding multi-source audio data recorded by the audio acquisition device and gesture data in a process of the audio acquisition device recording the multi-source audio data; The encoded file is decoded to obtain the multi-source audio data and the posture data.

6. The method according to claim 1, characterized in that The step of obtaining the posture data in the data to be processed comprises: Acquire the original data output by the inertial measurement unit during the process of recording the multi-source audio data by the audio acquisition device; The original data is converted into posture data representing the posture of the audio acquisition device.

7. The method according to claim 6, characterized in that After converting the original data into posture data representing the posture of the audio acquisition device, the method further includes: The posture data is subjected to data filtering to filter out data representing data noise and / or glitches.

8. A method for extracting human voice, characterized in that: The method comprises: Acquire data to be processed from an audio acquisition device, wherein the data to be processed includes: multi-source audio data recorded by the audio acquisition device and posture data in a process in which the audio acquisition device records the multi-source audio data; Separating the audio data of the target sound source from the multi-sound source audio data, and adjusting the beam direction to point to or approach the target sound source according to the posture data during the separation process to obtain the audio data of the target sound source; Performing sound processing on the audio data of the target sound source to obtain target human voice data, wherein the sound processing includes: human voice extraction and / or human voice enhancement.

9. An audio separation method, characterized in that: Applied to an audio acquisition device, wherein the audio acquisition device comprises: an audio acquisition unit with a sound recording function, and a posture acquisition unit with a posture detection function, and the method comprises: In response to the start operation, the audio acquisition unit records the multi-source audio data, and the posture acquisition unit obtains the posture data of the multi-source audio data during the recording process of the audio acquisition unit; The multi-source audio data and the posture data are sent to a target device, so that the target device separates the audio data of a target sound source from the multi-source audio data, and adjusts the beam direction to point to or approach the target sound source according to the posture data during the separation process.

10. The method according to claim 9, characterized in that Sending the multi-source audio data and the gesture data includes: The multi-source audio data and the gesture data are encoded into a single file, and the single file is transmitted.

11. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 10 by running the program in the memory.

12. An audio processing system, characterized in that: Including audio acquisition devices and target devices; Wherein, the audio acquisition device is configured to perform the audio separation method according to claim 9 or 10; and / or, The target device is configured to execute the audio separation method according to any one of claims 1 to 7, or to execute the human voice extraction method according to claim 8.

13. A computer program product, characterized in that include: A computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images