Multi-person sound source separation method, device, equipment, medium and computer program product

Abstract

The invention provides a multi-person sound source separation method, device and equipment, a medium and a computer program product, and relates to the technical field of audio signal processing. The method comprises the following steps: acquiring a voice segment containing multi-person voices; based on the physical position of each microphone unit, using an ODAS algorithm to determine the azimuth angle of each sound source; determining an identifier of each sound source according to the azimuth angle of each sound source; based on the identifier of each sound source, reading a voice fragment segment corresponding to each identifier from the voice fragment; and segmenting the voice segments belonging to the same identifier as independent voice segments of the speaker corresponding to the same identifier. According to the method, an artificial intelligence model trained in advance is not needed, a large amount of voiceprint data does not need to be marked in advance, independent voice segments of different speakers can be separated according to azimuth angles of different sound sources collected by the microphone array, the accuracy of voice recognition is ensured, meanwhile, computing resources are reduced, and therefore the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of audio signal processing, and in particular to a method, device, equipment, medium and computer program product for separating multiple sound sources. Background Art

[0002] Voice interaction technology is widely used in smart homes, car-mounted equipment, conference systems, etc. With the continuous development of artificial intelligence technology, deep learning methods are also widely used in the field of voice interaction, such as recognizing speech and outputting it as text.

[0003] In complex environments, such as when multiple people are speaking at the same time or when the background noise is loud, it is necessary to separate the independent signal of each speaker from the mixed multi-speaker voice. Existing deep learning technologies for multi-speaker voice separation generally require a large amount of labeled data for training, which consumes a lot of computing resources. Summary of the invention

[0004] The present invention provides a method, device, equipment, medium and computer program product for separating multiple sound sources, which are used to solve the defect in the prior art that a large amount of computing resources are consumed in the process of using artificial intelligence models for speech separation, and realize speech recognition without using pre-trained artificial intelligence models, thereby saving computing resources.

[0005] The present invention provides a method for separating multiple sound sources, comprising the following steps.

[0006] Acquire a voice segment containing multiple voices; wherein the voice segment is obtained by simultaneously capturing multiple sound sources using a microphone array with calibrated physical positions; Based on the physical position of each microphone unit, the ODAS algorithm is used to determine the azimuth of each sound source; Determining an identifier of each sound source according to the azimuth of each sound source; Based on the identifiers of the sound sources, the speech segment segments corresponding to the identifiers are read from the speech segment; and the speech segment segments belonging to the same identifier are used as independent speech segments of the speaker corresponding to the same identifier.

[0007] According to a method for separating multiple sound sources provided by the present invention, the identifier includes a sound source ID, and the method of reading the speech segment corresponding to each identifier from the speech segment based on the identifiers of each sound source includes: According to the timestamps in the voice segment, the voice segment is divided into sub-segments corresponding to the timestamps one by one; In each sub-segment under each timestamp, detecting the sound source ID contained in the sub-segment; Determine whether the sound source activity corresponding to any sound source ID is greater than a preset activity threshold; If yes, then the timestamp when any sound source ID and its corresponding sound source activity is greater than a preset activity threshold is recorded in the voice segment segment exclusive to any sound source ID; After traversing all timestamps, the speech segment corresponding to each sound source ID is obtained.

[0008] According to a method for separating multiple sound sources provided by the present invention, after reading the speech segment corresponding to each identifier from the speech segment based on the identifiers of each sound source, the method further includes: Extracting acoustic features from the speech segment corresponding to each identifier; Calculating the similarity between the acoustic feature and each voiceprint in a preset voiceprint library; According to the similarity, the identity of the speaker in the speech segment is determined.

[0009] According to a method for separating multiple sound sources provided by the present invention, the azimuth of each sound source is determined by using an ODAS algorithm based on the physical position of each microphone unit, including: Calculating the arrival time differences of the multiple sound sources at the respective microphone units according to the propagation speed of the speech signal and the physical position of each microphone unit in the microphone array; Determine a direction estimate of each sound source based on the arrival time difference and the physical position of each microphone unit; The signals received by all microphone units in the microphone array from any sound source are regarded as a beam, and based on the direction estimation value of any sound source, the beam is beamformed to obtain an enhanced signal of any sound source; The azimuth of the enhanced signal of any sound source is calculated using the generalized cross-correlation method.

[0010] According to a method for separating multiple sound sources provided by the present invention, after traversing all timestamps, obtaining the speech segment corresponding to each sound source ID includes: If the number of timestamps contained in the duration in which the activity corresponding to any sound source ID is less than the preset activity threshold reaches the preset timestamp number threshold, the recording is terminated and the current voice segment is saved.

[0011] According to a method for separating multiple sound sources provided by the present invention, obtaining a speech segment containing multiple voices includes: Obtaining an original voice clip containing multiple voices; The original speech segment is subjected to noise filtering using the FRCRN speech noise reduction algorithm to obtain the speech segment.

[0012] The present invention also provides a multi-person sound source separation device, comprising the following modules: A voice acquisition module, used to acquire a voice segment containing multiple voices; wherein the voice segment is obtained by simultaneously capturing multiple sound sources using a microphone array with calibrated physical positions; An azimuth angle determination module, used to determine the azimuth angle of each sound source using an ODAS algorithm based on the physical position of each microphone unit; An identifier determination module, which determines an identifier of each sound source according to the azimuth of each sound source; The speech segment reading module is used to read the speech segment corresponding to each identifier from the speech segment based on the identifiers of the various sound sources; and to use the speech segment segments belonging to the same identifier as the independent speech segments of the speaker corresponding to the same identifier.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned methods for separating multiple sound sources is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for separating multiple sound sources as described above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the multi-person sound source separation method as described above is implemented.

[0016] The multi-person sound source separation method, device, equipment, medium and computer program product provided by the present invention obtain a voice segment containing multi-person voices; wherein the voice segment is obtained by simultaneously capturing multiple sound sources using a microphone array with a calibrated physical position; based on the physical position of each microphone unit, the azimuth of each sound source is determined using the ODAS algorithm; according to the azimuth of each sound source, the identifier of each sound source is determined; based on the identifier of each sound source, the voice segment segment corresponding to each identifier is read from the voice segment; the voice segment segment belonging to the same identifier is used as the independent voice segment of the speaker corresponding to the same identifier. This method does not require a pre-trained artificial intelligence model, does not require a large amount of voiceprint data to be pre-labeled, and can separate independent voice segments of different speakers according to the azimuths of different sound sources collected by the microphone array, while ensuring the accuracy of speech recognition, reducing computing resources, thereby reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the method for separating multiple sound sources provided by the present invention.

[0019] Figure 2 It is a flow chart of speech separation using sound source ID provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of a multi-person sound source separation device provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Combine the following Figure 1-Figure 4 Specific embodiments of the present invention are described.

[0024] Figure 1 is a flow chart of the method for separating multiple sound sources provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101, obtaining a voice segment containing multiple voices; wherein the voice segment is obtained by simultaneously capturing multiple sound sources using a microphone array with calibrated physical positions; Among them, the microphone array is a system composed of multiple microphones arranged in a certain geometry, which is used to simultaneously capture the sound signals in the sound field. Each microphone in the array is called a microphone unit. The microphones can be arranged in different geometric shapes. Common arrangements include: planar array, stereo array (i.e. distributed in three-dimensional space), circular array, etc. The core idea of ​​the microphone array is to use the time and phase differences of the sound signals captured by multiple microphones to achieve the positioning of the sound source and signal processing.

[0025] Specifically, multiple audio signals are captured simultaneously in the environment through a microphone array. The physical positions of the microphones need to be accurately calibrated so that the relative time delay between them can be used to locate the sound source during processing. After multi-channel acquisition, the original signal is first pre-processed by noise suppression, gain adjustment, and echo cancellation to ensure the clarity and quality of the input signal.

[0026] Optionally, the above step 101 includes: obtaining an original speech segment containing multiple voices; and filtering the original speech segment with noise using the FRCRN speech noise reduction algorithm to obtain the speech segment.

[0027] Specifically, in practical applications, audio is easily affected by noisy environments: in noisy environments, incorrect active source identification may occur, resulting in inaccurate segmentation of the sound source. Therefore, the original audio output by the microphone array can first be input into the FRCRN speech noise reduction algorithm to filter the noise of the original audio. The filtered noise is then subjected to the ODAS algorithm to obtain a more accurate sound source position and reduce the angle affected by noise. Secondly, in the segmented fragments, due to the influence of noise, fragments containing only meaningless noise will be segmented. In order to distinguish them from human voice fragments, MFCC (Mel-frequency cepstral coefficients) are used to screen the fragments. The above method can further improve the segmentation effect of speech segments.

[0028] Step 102: Based on the physical position of each microphone unit, the ODAS algorithm is used to determine the azimuth of each sound source.

[0029] Among them, the ODAS (Online Dynamic Acoustic Sensing) algorithm is an algorithm for real-time sound source localization, tracking and separation. It is specially designed to process multi-channel audio signals collected by microphone arrays and can locate and track multiple sound sources in real time in a dynamic environment.

[0030] Specifically, the ODAS algorithm can estimate the direction (azimuth and elevation) of the sound source in real time, including: using time delay estimation (TDOA) or beamforming technology to calculate the direction of the sound source; estimating the time difference of the signal arriving at different microphones through generalized cross correlation (GCC) or adaptive filtering methods; combining the geometric structure of the microphone array to calculate the azimuth and elevation of the sound source.

[0031] Step 103: Determine an identifier of each sound source according to the azimuth angle of each sound source.

[0032] Specifically, according to the azimuths of the above-mentioned sound sources, a unique identifier (ie, sound source ID) is determined for each sound source, which is used to identify the sound source at the azimuth.

[0033] Step 104: based on the identifiers of the sound sources, read the speech segment segments corresponding to the identifiers from the speech segment; and use the speech segment segments belonging to the same identifier as the independent speech segments of the speaker corresponding to the same identifier.

[0034] The identifier may be a sound source ID, which is used to uniquely identify a sound source at an azimuth.

[0035] Specifically, the speech of different sound source IDs (or different speakers) is separated according to the identifiable IDs at the same timestamp in the above voice segment. The processing process is based on the following two key concepts: activity and activity threshold (threshold_activity). When the activity of a sound source ID exceeds this threshold, the sound source ID is considered to be speaking. Silent state: If the activity of a sound source ID in multiple consecutive timestamps is lower than the threshold, the sound source is considered to be in a silent state. According to these rules, the algorithm will traverse the timestamps in the entire voice segment, analyze the changes in the sound source ID and its activity one by one, and generate independent voice segments corresponding to each sound source ID.

[0036] The above embodiment obtains a voice segment containing multiple voices; wherein the voice segment is obtained by simultaneously capturing multiple sound sources using a microphone array with calibrated physical positions; based on the physical position of each microphone unit, the azimuth of each sound source is determined using the ODAS algorithm; according to the azimuth of each sound source, the identifier of each sound source is determined; based on the identifier of each sound source, the voice segment corresponding to each identifier is read from the voice segment; the voice segment segments belonging to the same identifier are used as independent voice segments of the speaker corresponding to the same identifier. This method does not require a pre-trained artificial intelligence model, does not require a large amount of voiceprint data to be pre-labeled, and can separate independent voice segments of different speakers according to the azimuths of different sound sources collected by the microphone array, while ensuring the accuracy of speech recognition, reducing computing resources, thereby reducing costs.

[0037] In one embodiment, if Figure 2 As shown, Figure 2 The flowchart of separating speech using sound source ID is shown, that is, the above step 104 includes: Step 201, according to the timestamp in the voice segment, dividing the voice segment into sub-segments corresponding to the timestamp one by one; Among them, timestamp is information used to mark the time point or time period of a specific event or segment in a speech signal. It is usually expressed in time units (such as seconds or milliseconds).

[0038] Specifically, a timestamp is set for the voice segments, for example, a timestamp is marked every 5 seconds, and the voice segments are divided into sub-segments corresponding to the timestamps one by one according to the timestamps.

[0039] Step 202, in each sub-segment under a timestamp, detecting the sound source ID contained in the sub-segment; Specifically, first initialize important parameters, that is, set the activity threshold threshold_activity to determine whether the sound source is active; set the silent threshold silent_threshold to determine whether a sound source is considered silent in multiple consecutive timestamps. Create a list segments for storing speech segment segments, each speech segment segment will contain the start time, end time and corresponding sound source ID of the segment.

[0040] Traverse each timestamp, process the sound source information under each timestamp, and identify the sound source ID that appears in the sub-segment corresponding to each timestamp.

[0041] Optionally, the above identifier may also include the following characters: tag: identifies whether the sound source is active. Common values ​​include "dynamic" to indicate that the sound source is active, or an empty string to indicate that the sound source is inactive.

[0042] x, y, z: The three-dimensional coordinates of the sound source in space, indicating the specific location of the sound source (e.g. azimuth).

[0043] Activity: indicates the activity of the sound source. When the activity value is greater than a certain threshold, the sound source is considered to be making sound.

[0044] Step 203: determine whether the sound source activity corresponding to any sound source ID is greater than a preset activity threshold.

[0045] Specifically, when the algorithm detects any sound source ID, the following steps are performed: identifying active sound sources, that is, checking the activity of each sound source to determine whether it is greater than a preset activity threshold (threshold_activity).

[0046] Step 204: If yes, then the timestamp when any sound source ID and its corresponding sound source activity is greater than a preset activity threshold is recorded in the voice segment segment exclusive to any sound source ID.

[0047] Specifically, the activity of each sound source is checked. If the activity is greater than the preset activity threshold (threshold_activity), the sound source is considered active. At this time, the algorithm will add the sound source ID and the timestamp of the detection to the current active sound source list segments, and create a voice segment in segments to record the sound source ID and its timestamp information.

[0048] It is worth mentioning that if the sound source ID appears for the first time, it means that a new voice segment about the sound source ID has begun to be recorded. That is, when a new sound source ID appears for the first time at the current timestamp and its activity is greater than the threshold, the algorithm will create a new voice segment and record the ID and the timestamp of its utterance. If it is detected that the sound source ID in the current timestamp already exists in the previous adjacent timestamp and is still active (that is, the activity is greater than the threshold), the algorithm will continue to extend the time range of the current voice segment until the voice segment ends.

[0049] The sign of the end is: if the number of timestamps contained in the duration in which the activity corresponding to any sound source ID is less than the preset activity threshold reaches the preset timestamp number threshold, the recording is ended and the current voice segment is saved.

[0050] Specifically, if the activity of the sound source ID is lower than the preset activity threshold (threshold_activity) for multiple consecutive timestamps, the algorithm will determine whether to end the current voice segment by counting the number of timestamps (silent_count) that are continuously lower than the activity threshold (threshold_activity). Once the number of silences exceeds the set timestamp count threshold (silent_count_threshold), the current voice segment will be marked as ended and the segment will be saved in the segments list.

[0051] Step 205: After traversing all time stamps, the speech segment corresponding to each sound source ID is obtained.

[0052] Specifically, after traversing all the above time stamps, the voice segment corresponding to each sound source ID is obtained, and one sound source ID may correspond to one voice segment or may correspond to multiple voice segment.

[0053] Optionally, adjusting a preset activity threshold (threshold_activity) can change the sensitivity of the algorithm to active sound sources. It should be noted that a lower activity threshold may result in more noise being identified as active signals.

[0054] The timestamp count threshold (silent_count_threshold) determines how many timestamps of continuous silence are required before the recording of a voice segment ends. Increasing this value can avoid premature termination of segment recording due to short pauses.

[0055] In the above embodiment, by setting the activity threshold of the sound source and counting the active sound sources in each timestamp, the voice segments corresponding to different sound source IDs can be read. This method can achieve real-time and dynamic separation of overlapping voice segments. By recording the activity state of each sound source and the corresponding time interval, the algorithm can accurately separate overlapping voice signals (such as multiple people speaking) and finally obtain independent voice segments for each speaker.

[0056] In one embodiment, the above step 103 includes: extracting acoustic features from the speech segment corresponding to each identifier; calculating the similarity between the acoustic features and each voiceprint in a preset voiceprint library; and determining the identity of the speaker in the speech segment based on the similarity.

[0057] Specifically, for the above-mentioned separated speech segments containing only one person speaking, the speaker's identity is further identified. First, feature extraction methods such as i-vectors and d-vectors are used to extract acoustic features representing the speaker's identity from the separated speech signal. Secondly, a speaker voiceprint library is constructed, and the features are classified by calculating cosine similarity and other methods to identify the speaker identity of each speech segment.

[0058] Based on the processed sound source localization data, sound source separation results and speaker recognition information, it is helpful to further understand the various participants in the audio scene, speech content and direction, and further complete deeper issues such as speaker differentiation, speech summary, question summary, etc.

[0059] In the above embodiment, the speaker identity is obtained by identifying the similarity between the acoustic features and the voiceprints in the preset voiceprint library, which is conducive to recording and outputting the speaker and the corresponding voice information, without the need for manual recording, and improving the efficiency of voice recognition.

[0060] In one embodiment, the above-mentioned step 102 includes: calculating the arrival time difference of the multiple sound sources at each microphone unit respectively according to the propagation speed of the speech signal and the physical position of each microphone unit in the microphone array; determining the direction estimation value of each sound source according to the arrival time difference and the physical position of each microphone unit; treating the signal received by all microphone units in the microphone array from any sound source as a beam, and based on the direction estimation value of any sound source, beamforming the beam to obtain an enhanced signal of any sound source; and calculating the azimuth of the enhanced signal of any sound source using the generalized cross-correlation method.

[0061] Specifically, the time difference of arrival (TDOA) is first estimated: different microphone units in the microphone array receive the same sound signal at different times. Based on the propagation speed of the signal, the arrival time difference of the sound can be calculated. The direction of the sound source is determined by minimizing the matching degree of these time differences with the geometric layout of the microphone array.

[0062] Beamforming technology: The signal received by all microphone units in the microphone array from any sound source is taken as a microphone array signal (i.e., a beam), and the signal from the direction of the target sound source is enhanced by weighting and phase adjustment of the microphone array signal. Beamforming technology adjusts the weight of the received signal based on the angle information of the sound source positioning, enhances the signal in a specific direction, and thus suppresses noise and interference in other directions.

[0063] Direction of Arrival (DOA): Use efficient angle calculation algorithms, such as Generalized Cross Correlation (GCC) or Maximum Likelihood Estimation (MLE), to determine the azimuth of each sound source. The ODAS algorithm can optimize these calculation results by combining multi-channel data to obtain accurate direction estimates.

[0064] In the above embodiment, the beamforming technology in the ODAS algorithm is used to enhance the signal in a specific direction, which is conducive to locating the accurate azimuth of the sound source and providing an effective data basis for subsequent speech separation.

[0065] The multi-person sound source separation device provided by the present invention is described below. The multi-person sound source separation device described below and the multi-person sound source separation method described above can be referenced to each other.

[0066] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of a multi-person sound source separation device, which includes the following modules: The speech acquisition module 301 is used to acquire a speech segment containing multiple voices; wherein the speech segment is obtained by simultaneously capturing multiple sound sources using a microphone array with calibrated physical positions; An azimuth angle determination module 302 is used to determine the azimuth angle of each sound source using an ODAS algorithm based on the physical position of each microphone unit; An identifier determination module 303 determines an identifier of each sound source according to the azimuth of each sound source; The speech segment reading module 304 is used to read the speech segment corresponding to each identifier from the speech segment based on the identifiers of the various sound sources; and to use the speech segment segments belonging to the same identifier as the independent speech segments of the speaker corresponding to the same identifier.

[0067] In one embodiment, the identifier includes a sound source ID, and the voice segment segment reading module 304 is further used to: According to the timestamps in the voice segment, the voice segment is divided into sub-segments corresponding to the timestamps one by one; In each sub-segment under each timestamp, detecting the sound source ID contained in the sub-segment; Determine whether the sound source activity corresponding to any sound source ID is greater than a preset activity threshold; If yes, then the timestamp when any sound source ID and its corresponding sound source activity is greater than a preset activity threshold is recorded in the voice segment segment exclusive to any sound source ID; After traversing all timestamps, the speech segment corresponding to each sound source ID is obtained.

[0068] In one embodiment, the above-mentioned device also includes an identity recognition unit, which is used to: extract acoustic features from the speech segment corresponding to each identifier; calculate the similarity between the acoustic features and each voiceprint in a preset voiceprint library; and determine the identity of the speaker in the speech segment based on the similarity.

[0069] In one embodiment, the azimuth angle determination module 302 is further configured to: Calculating the arrival time differences of the multiple sound sources at the respective microphone units according to the propagation speed of the speech signal and the physical position of each microphone unit in the microphone array; Determine a direction estimate of each sound source based on the arrival time difference and the physical position of each microphone unit; The signals received by all microphone units in the microphone array from any sound source are regarded as a beam, and based on the direction estimation value of any sound source, the beam is beamformed to obtain an enhanced signal of any sound source; The azimuth of the enhanced signal of any sound source is calculated using the generalized cross-correlation method.

[0070] In one embodiment, the speech segment reading module 304 is further used for: If the number of timestamps contained in the duration in which the activity corresponding to any sound source ID is less than the preset activity threshold reaches the preset timestamp number threshold, the recording is terminated and the current voice segment is saved.

[0071] In one embodiment, the voice acquisition module 301 is further used for: An original speech segment containing multiple voices is obtained; and the original speech segment is subjected to noise filtering using an FRCRN speech noise reduction algorithm to obtain the speech segment.

[0072] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute a method for separating multiple sound sources, the method comprising: obtaining a speech segment containing multiple voices; wherein the speech segment is obtained by simultaneously capturing multiple sound sources using a microphone array with a calibrated physical position; based on the physical position of each microphone unit, determining the azimuth of each sound source using the ODAS algorithm; determining the identifier of each sound source according to the azimuth of each sound source; based on the identifier of each sound source, reading the speech segment segment corresponding to each identifier from the speech segment; and using the speech segment segment belonging to the same identifier as the independent speech segment of the speaker corresponding to the same identifier.

[0073] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0074] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-sound source separation method provided by the above-mentioned methods, which includes: obtaining a speech segment containing multi-person voices; wherein the speech segment is obtained by simultaneously capturing multiple sound sources using a microphone array with a calibrated physical position; based on the physical position of each microphone unit, determining the azimuth of each sound source using the ODAS algorithm; determining the identifier of each sound source according to the azimuth of each sound source; based on the identifier of each sound source, reading the speech segment segment corresponding to each identifier from the speech segment; and treating the speech segment segments belonging to the same identifier as independent speech segments of the speaker corresponding to the same identifier.

[0075] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the multi-sound source separation method provided by the above-mentioned methods, the method comprising: obtaining a speech segment containing multi-person voices; wherein the speech segment is obtained by simultaneously capturing multiple sound sources using a microphone array with a calibrated physical position; based on the physical position of each microphone unit, determining the azimuth of each sound source using the ODAS algorithm; determining an identifier of each sound source based on the azimuth of each sound source; based on the identifier of each sound source, reading the speech segment segment corresponding to each identifier from the speech segment; treating the speech segment segments belonging to the same identifier as independent speech segments of the speaker corresponding to the same identifier.

[0076] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for separating multiple sound sources, characterized in that: include: Acquire a voice segment containing multiple voices; wherein the voice segment is obtained by simultaneously capturing multiple sound sources using a microphone array with calibrated physical positions; Based on the physical position of each microphone unit, the ODAS algorithm is used to determine the azimuth of each sound source; Determining an identifier of each sound source according to the azimuth of each sound source; Based on the identifiers of the sound sources, the speech segment segments corresponding to the identifiers are read from the speech segment; and the speech segment segments belonging to the same identifier are used as independent speech segments of the speaker corresponding to the same identifier.

2. The method for separating multiple sound sources according to claim 1, characterized in that: The identifier includes a sound source ID, and the step of reading a speech segment corresponding to each identifier from the speech segment based on the identifiers of each sound source includes: According to the timestamps in the voice segment, the voice segment is divided into sub-segments corresponding to the timestamps one by one; In each sub-segment under each timestamp, detecting the sound source ID contained in the sub-segment; Determine whether the sound source activity corresponding to any sound source ID is greater than a preset activity threshold; If yes, then the timestamp when any sound source ID and its corresponding sound source activity is greater than a preset activity threshold is recorded in the voice segment segment exclusive to any sound source ID; After traversing all timestamps, the speech segment corresponding to each sound source ID is obtained.

3. The method for separating multiple sound sources according to claim 1, characterized in that: After reading the speech segment corresponding to each identifier from the speech segment based on the identifiers of each sound source, the method further includes: Extracting acoustic features from the speech segment corresponding to each identifier; Calculating the similarity between the acoustic feature and each voiceprint in a preset voiceprint library; According to the similarity, the identity of the speaker in the speech segment is determined.

4. The method for separating multiple sound sources according to claim 1, characterized in that: The method of determining the azimuth of each sound source by using the ODAS algorithm based on the physical position of each microphone unit includes: Calculating the arrival time differences of the multiple sound sources at the respective microphone units according to the propagation speed of the speech signal and the physical position of each microphone unit in the microphone array; Determine a direction estimate of each sound source based on the arrival time difference and the physical position of each microphone unit; The signals received by all microphone units in the microphone array from any sound source are regarded as a beam, and based on the direction estimation value of any sound source, the beam is beamformed to obtain an enhanced signal of any sound source; The azimuth of the enhanced signal of any sound source is calculated using the generalized cross-correlation method.

5. The method for separating multiple sound sources according to claim 2, characterized in that: After traversing all timestamps, the voice segment segments corresponding to each sound source ID are obtained, including: If the number of timestamps contained in the duration in which the activity corresponding to any sound source ID is less than the preset activity threshold reaches the preset timestamp number threshold, the recording is terminated and the current voice segment is saved.

6. The method for separating multiple sound sources according to any one of claims 1 to 5, characterized in that: The obtaining of a voice segment containing multiple voices includes: Obtaining an original voice clip containing multiple voices; The original speech segment is subjected to noise filtering using the FRCRN speech noise reduction algorithm to obtain the speech segment.

7. A multi-person sound source separation device, characterized in that: include: A voice acquisition module, used to acquire a voice segment containing multiple voices; wherein the voice segment is obtained by simultaneously capturing multiple sound sources using a microphone array with calibrated physical positions; An azimuth angle determination module, used to determine the azimuth angle of each sound source using an ODAS algorithm based on the physical position of each microphone unit; An identifier determination module, which determines an identifier of each sound source according to the azimuth of each sound source; The speech segment reading module is used to read the speech segment corresponding to each identifier from the speech segment based on the identifiers of the various sound sources; and to use the speech segment segments belonging to the same identifier as the independent speech segments of the speaker corresponding to the same identifier.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method for separating multiple sound sources according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for separating multiple sound sources according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for separating multiple sound sources according to any one of claims 1 to 6 is implemented.

Images