Streaming speaker log method and system

Abstract

The invention relates to a streaming speaker log method and system, and the method comprises the steps: obtaining a voice stream, and detecting the voice of a speaker in the voice stream through employing a VAD algorithm; slicing the voice of the speaker, and performing speaker jump detection based on the sliced voice to obtain a speaker jump point; the method comprises the following steps: splicing sliced voice streams before a jump point of a speaker, extracting voiceprint features of spliced voice segments, comparing the extracted voiceprint features in a pre-established temporary voiceprint library, and determining a speaker category corresponding to the voiceprint features after successful comparison. According to the invention, a streaming processing mode is adopted, continuous voice streams can be processed and analyzed in real time, waiting for the completion of processing of the whole voice file is not needed, and the processing efficiency and real-time performance are improved. According to the method, the number of speakers does not need to be preset in advance, the voiceprints are registered step by step in the execution process, the new speaker is found through the temporary voiceprint library, and unlimited processing of the number of speakers is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent speech processing, and in particular to a streaming speaker log method and system. Background Art

[0002] In daily life and various work scenarios, it is very common for multiple people to speak at the same time, such as conference discussions, debates, street interviews, etc. In order to effectively distinguish and identify different speakers in these scenarios, speaker log technology and voiceprint library retrieval technology have become key. The core task of speaker log technology is to segment the mixed voice stream into independent voice segments belonging to different speakers, while voiceprint library retrieval technology is responsible for matching these independent voice segments with the pre-established voiceprint library to determine the identity of the speaker. With the increasing richness and complexity of application scenarios, higher requirements are placed on the real-time, accuracy and scalability of speaker log technology.

[0003] Traditional speaker log technologies include the following two categories:

[0004] One is the speaker log technology based on feature clustering: This method first requires the entire speech to be sliced ​​into fixed-length slices. Then, the voiceprint features are extracted for each slice, and these features are grouped through a clustering algorithm, with each group corresponding to a speaker. Finally, the original speech is divided into speech segments belonging to different speakers based on the clustering results. This method is highly restrictive and usually requires the specification of the number of personnel categories and the input of the entire speech. It is not suitable for streaming processing. Since all speech segments need to be clustered, if the speech is long, it will result in high memory usage, large amount of calculation, and significant processing delay. It also has poor flexibility and is difficult to adapt to scenarios where the number of speakers changes dynamically or the length of the speech is uncertain.

[0005] The other is an end-to-end speaker log technology based on deep learning. This method uses a deep learning model to directly extract high-level voiceprint features from raw speech data. The model determines which speaker the current speech segment belongs to based on these features and historical context information. This method can automatically handle speaker jumps and achieve real-time speaker log generation. However, although this method can achieve real-time speaker logs, it requires a large amount of labeled training data to train the model, and the data acquisition and labeling costs are high. The model performance is easily affected by factors such as environmental noise and changes in voice quality, resulting in fluctuations in recognition accuracy. In addition, as the number of speakers increases, the model recognition accuracy decreases significantly, limiting its application effect in multi-speaking scenarios. Summary of the invention

[0006] In view of this, it is necessary to provide a streaming speaker log method and system to solve the above-mentioned defects of the prior art.

[0007] In order to solve the above problems, in a first aspect, an embodiment of the present invention provides a streaming speaker log method, comprising:

[0008] Acquire a speech stream and use a VAD algorithm to detect the speaker's voice in the speech stream;

[0009] Slicing the speaker's speech, performing speaker transition detection based on the sliced ​​speech, and obtaining a speaker transition point;

[0010] The sliced ​​speech streams before the speaker jump point are spliced, the voiceprint features of the spliced ​​speech segments are extracted, and the extracted voiceprint features are compared with the pre-established temporary voiceprint library. After the comparison is successful, the speaker category corresponding to the voiceprint features is determined.

[0011] Preferably, after comparing the extracted voiceprint features with a pre-established temporary voiceprint library to determine the speaker category corresponding to the voiceprint features, the method further includes:

[0012] The speaker's voiceprint features are compared with the known voiceprint features in the entity voiceprint library to identify the speaker's identity information.

[0013] Preferably, after using the VAD algorithm to detect the speaker's voice in the voice stream, the method further comprises:

[0014] Perform noise reduction processing on the speaker's speech.

[0015] Preferably, the speaker jump detection based on the sliced ​​speech specifically includes:

[0016] Extracting speech features of the sliced ​​speech; wherein the speech features include frequency, amplitude, zero-crossing rate, Mel cepstral coefficients and linear prediction cepstral coefficients;

[0017] Extract voiceprint features from speech features. After accumulating a certain number of voiceprint features, calculate the similarity of the voiceprint features to determine whether the voiceprint features come from the same speaker.

[0018] The calculated similarity values ​​are analyzed to determine the speaker change point.

[0019] Preferably, the voiceprint feature extraction is performed on the speech feature, and after a certain number of voiceprint features are accumulated, similarity calculation is performed on the voiceprint features, which specifically includes:

[0020] A certain number of sliced ​​voices are spliced ​​into a voice matrix, and the voice matrix is ​​input into the voiceprint model to extract the voiceprint features of each sliced ​​voice;

[0021] The similarity between each row of voiceprint features in the speech matrix and other rows of voiceprint features is calculated to obtain a similarity matrix. Each row of the similarity matrix is ​​searched according to the set similarity threshold to determine the jump point of this row. The speech before the jump point is identified as the speech of the same speaker.

[0022] Preferably, after comparing the extracted voiceprint features with the pre-established temporary voiceprint library, the method further comprises:

[0023] If the voiceprint feature comparison fails and the system cannot classify the current voiceprint feature into a known speaker category in the temporary voiceprint library, the voiceprint feature is registered as a new category in the temporary voiceprint library.

[0024] Preferably, after the comparison is successful, the method further comprises:

[0025] The voiceprint features of the concatenated speech segment are smoothed with the voiceprint features of the corresponding speaker class in the temporary voiceprint library.

[0026] In a second aspect, an embodiment of the present invention provides a streaming speaker log system, including:

[0027] VAD speech detection module, used to obtain speech stream and detect the speaker's speech in the speech stream using VAD algorithm;

[0028] A speaker transition detection module, used for slicing the speaker's speech, performing speaker transition detection based on the sliced ​​speech, and obtaining a speaker transition point;

[0029] The voiceprint comparison module is used to splice the voice stream slices before the speaker jump point, extract the voiceprint features of the spliced ​​voice segment, compare the extracted voiceprint features with the pre-established temporary voiceprint library, and determine the speaker category corresponding to the voiceprint features after the comparison is successful;

[0030] The temporary voiceprint library module is used to store the speaker category and its corresponding voiceprint features in the current voice stream.

[0031] In a third aspect, the present invention further provides an electronic device, including a memory and a processor, wherein:

[0032] The memory is used to store programs;

[0033] The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the streaming speaker log method as described in the embodiment of the first aspect of the present invention.

[0034] In a fourth aspect, the present invention further provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps in the streaming speaker log method as described in the embodiment of the first aspect of the present invention.

[0035] Compared with the prior art, the streaming speaker log method and system provided by the present invention have the following beneficial effects:

[0036] 1) Traditional speaker log technology based on feature clustering needs to process the entire speech, resulting in high memory usage, large amount of calculation, and significant processing delay. However, the present invention adopts a streaming processing method, which can process and analyze continuous speech streams in real time without waiting for the entire speech file to be processed, thereby significantly improving processing efficiency and real-time performance.

[0037] 2) The present invention divides the speech stream into fixed-length segments and extracts the voiceprint features of each segment for similarity comparison, thereby realizing rapid and accurate detection of the speaker's transition points. This method is not only simple and easy to operate, but also can effectively deal with sudden changes in speech, providing a reliable basis for subsequent processing. This efficient transition detection mechanism enables the present invention to maintain a high degree of accuracy and stability when processing continuous speech streams in real time.

[0038] 3) Traditional technologies usually require the number of personnel categories to be specified, which makes it difficult to adapt to scenarios where the number of speakers changes dynamically. However, the present invention does not require the number of speakers to be preset in advance, but gradually registers voiceprints during execution, discovers new speakers through a temporary voiceprint database, and realizes unlimited processing of the number of speakers. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flow chart of the streaming speaker log method provided by the present invention;

[0040] Figure 2 A schematic diagram of speaker categories stored in the temporary voiceprint library module provided by the present invention;

[0041] Figure 3 A schematic diagram of the structure of the streaming speaker log system provided by the present invention;

[0042] Figure 4 This is a structural block diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0043] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0044] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0045] Speaker log refers to a technical processing result that uses technical means to separate and classify the speech of different speakers in a scene where multiple people are speaking, and records the speech fragments of each speaker and their corresponding identity information.

[0046] At present, the traditional speaker log technology has the following defects: the feature clustering-based method requires the input of the entire speech, which is not suitable for streaming processing, and has large memory usage, high computational complexity, long processing delay, and is difficult to cope with scenarios where the number of speakers changes and the length of the speech is uncertain. The end-to-end method based on deep learning, although it can achieve real-time processing, relies on a large amount of labeled data, has high acquisition and labeling costs, is easily affected by environmental noise and speech quality, and has unstable recognition accuracy. In addition, as the number of speakers increases, the accuracy of this method decreases significantly, limiting its application in multi-person scenarios. In summary, the traditional solution has deficiencies in real-time, accuracy, and scalability, and it is difficult to meet the growing needs of speaker speech segmentation and recognition.

[0047] In view of this, the present invention provides a streaming speaker log method to solve the real-time and accuracy problems faced in the prior art when processing continuous speech streams. The following will be described and introduced through multiple embodiments.

[0048] Figure 1 The flow chart of the streaming speaker log method provided by the present invention is shown in FIG. Figure 1 The streaming speaker log method provided by the present invention at least comprises the following steps:

[0049] Step S1, obtain a voice stream, and use a VAD algorithm to detect the speaker's voice in the voice stream.

[0050] First, a continuous voice stream is obtained through an audio input device such as a microphone. The microphone captures the surrounding sound and converts it into an electrical signal, which then forms a continuous voice data stream. This voice stream contains multiple components such as the speaker's voice, background noise, and silent segments.

[0051] Next, the VAD algorithm is used to process the speech stream to detect the part of the speech stream that contains the speaker's speech and exclude the non-speech part. The VAD algorithm is a technology used to distinguish speech from non-speech (such as silence and noise). It determines whether the current segment contains valid speech information by analyzing the characteristics of the speech stream.

[0052] It is understandable that in some cases, due to environmental noise, equipment quality, etc., the acquired speaker's voice may contain more noise components, affecting the voice quality. In this case, the retained voice part can be subjected to noise reduction processing, and the noise reduction amount can be increased by adding a noise filtering module to ensure recognition performance.

[0053] As an optional implementation, a high-performance microphone array is installed in the device, and a multi-channel VAD algorithm is used to judge multiple audio sources by processing the signals of the microphone array. The algorithm is based on a deep learning model and can identify voice activities and distinguish background noise by dynamically learning different audio signals.

[0054] The real-time environmental noise analysis module is used to adaptively adjust the sensitivity of the VAD algorithm according to different noise environments. For example, in complex environments such as buses or airports, the system can improve the detection rate of speaker voices and reduce false positives and false negatives by continuously adjusting the noise model.

[0055] In this embodiment, a VAD algorithm based on deep learning, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), is used to train the model to adapt to various complex environments. The model learns the characteristics of different background noises, so that the VAD algorithm can not only accurately recognize speech in a quiet environment, but also perform effective voice activity detection in a high-noise background. In combination with adaptive filtering technology (such as Wiener filter or Kalman filter), real-time noise estimation and suppression are performed in the VAD stage. This will enable VAD to not only distinguish between speech and silence, but also enhance the quality of speech signals, further improving the accuracy of speaker recognition.

[0056] As an optional implementation, this embodiment can pre-process the retained speaker's voice, including voice slicing, sampling rate conversion, denoising, pre-emphasis and other operations, and extract voice features such as MFCC, FBANK, etc.

[0057] Step S2, slicing the speaker's speech, performing speaker transition detection based on the sliced ​​speech, and obtaining a speaker transition point.

[0058] The purpose of jump detection is to identify the change point of the speaker in a continuous speech stream, that is, the moment when the speaker switches from one speaker to another.

[0059] Specifically, voice slicing is first performed, and segments of the voice stream are intercepted regularly, for example, once every 200ms to 1s.

[0060] Next, the speech features of the sliced ​​speech are extracted; wherein the speech features include physical features such as frequency, amplitude, and zero-crossing rate, as well as acoustic features such as Mel-scale Frequency Cepstral Coefficients (MFCC) and linear prediction cepstral coefficients (fiterbank, FBANK). In this embodiment, a certain number (e.g., 10-20) of sliced ​​speech are spliced ​​into a speech matrix, and the speech matrix is ​​input into the voiceprint model to extract the voiceprint features of each sliced ​​speech.

[0061] The voiceprint feature is extracted from the speech feature. The voiceprint feature refers to a fixed-dimensional vector that can distinguish the target after processing the speech feature. It can be i-vector, d-vector, x-vector, etc. After accumulating a certain number of voiceprint features, the similarity of the voiceprint features is calculated to determine whether the voiceprint features come from the same speaker. The calculated similarity value is analyzed to determine the speaker jump point.

[0062] In this embodiment, the similarity between each row of voiceprint features and other rows of voiceprint features in the speech matrix is ​​calculated to obtain a similarity matrix. Each row of the similarity matrix is ​​searched according to the set similarity threshold, and the jump point of this row is determined. The speech before the jump point is identified as the speech of the same speaker. Starting from the determined jump point, the next jump point detection is continued according to the threshold. This process will continue until the speech stream ends or other stop conditions are met.

[0063] As an optional implementation, speaker change detection is crucial in speech streams because it marks the moment when the speaker switches. Traditional change detection usually relies on fixed-length speech segments for analysis, which may not accurately capture complex speaker switches. To overcome this limitation, this implementation introduces a dynamic change detection method based on deep learning:

[0064] When slicing the speech stream, it no longer relies on fixed time segments, but determines the timing of slicing each segment based on the feature changes calculated in real time. The LSTM network model dynamically adjusts the length of the slice to ensure that when switching between different speakers, the detection length can be automatically adjusted to adapt to changes in speech speed and intonation;

[0065] Compared with traditional voiceprint features, the x-vector technology based on deep neural network is used to extract more complex speaker features, making the jump point detection more sensitive and able to capture subtle changes in speech, thereby accurately identifying speaker jumps;

[0066] By using a long short-term memory network (LSTM) or a bidirectional LSTM (BiLSTM), the time window length of the jump detection is dynamically adjusted in combination with time series data. According to the degree of change of the voice characteristics, the duration of the jump point detection is automatically optimized to improve the accuracy of the speaker switching; when detecting the jump point, not only the traditional MFCC, FBANK and other features are considered, but also the advanced features in voiceprint recognition (such as d-vector or x-vector) can be introduced, and these features are modeled through the deep neural network (DNN) to automatically identify the subtle features of the speaker switching.

[0067] Step S3, splicing the sliced ​​speech streams before the speaker jump point, extracting the voiceprint features of the spliced ​​speech segment, comparing the extracted voiceprint features with the pre-established temporary voiceprint library, and determining the speaker category corresponding to the voiceprint features after successful comparison.

[0068] Specifically, after detecting the jump point, the system will splice the sliced ​​speech before the jump point (ie, the continuous speech of the previous speaker) to form a complete speech segment.

[0069] The system will extract the voiceprint features of the concatenated speech segment. The extracted voiceprint features will be compared with the stored voiceprint features in the temporary voiceprint library. The temporary voiceprint library is dynamically updated and stores the speaker categories and their corresponding voiceprint features in the current voice stream, such as Figure 2 shown.

[0070] If the voiceprint feature comparison is successful, that is, similar voiceprint features are found in the temporary voiceprint library, the speaker category corresponding to the voiceprint features of the spliced ​​speech segment can be determined. Then, the voiceprint features of the speech segment are smoothed with the voiceprint features successfully compared in the temporary voiceprint library. The smoothing method can be to fuse the two voiceprint features. It is understandable that some voiceprint models are more sensitive to environmental noise. If the recording time is long or the scene changes, the voiceprint features will change. Smoothing can help reduce the impact of these external factors on the voiceprint features, so that the voiceprint features of the same speaker remain relatively stable under different conditions.

[0071] If the voiceprint feature comparison fails, the system cannot classify the current voiceprint feature into the known speaker category in the temporary voiceprint library. At this time, the system believes that the current voice segment comes from a new speaker. In order to subsequently recognize and classify the voice of this new speaker, the voiceprint feature of the current voice segment needs to be registered as a new category in the temporary voiceprint library.

[0072] Step S4, comparing the speaker's voiceprint features with known voiceprint features in the entity voiceprint library to identify the speaker's identity information.

[0073] In the above steps, the voice stream is segmented into voice segments of different speakers through VAD voice detection, speaker jump detection and voiceprint comparison, and voiceprint features are extracted for each segment and stored in a temporary voiceprint library. Then, the speaker identity is confirmed in step S4.

[0074] Specifically, after the entire voice stream is processed, we obtain a temporary voiceprint library containing all detected speaker voiceprint features. For each voiceprint feature in the temporary voiceprint library, it is compared with the voiceprint feature in the physical voiceprint library. Among them, the physical voiceprint library usually contains a large number of voiceprint features of known persons. If a voiceprint feature in the temporary voiceprint library matches a voiceprint feature of a known person in the physical voiceprint library, the identity of the temporary speaker can be determined and its identity information is returned.

[0075] As an optional implementation method, in real-time scenarios, there may be a large number of unregistered speakers. In order to solve this problem, a multi-level voiceprint matching mechanism and a fast identity confirmation strategy are added:

[0076] When a speaker is detected, the most likely speaker is first selected from the temporary voiceprint database through a simple similarity calculation (such as Euclidean distance or cosine similarity). If the match exceeds a certain threshold, the identity confirmation phase is entered, and the speaker's identity is further confirmed through advanced matching algorithms.

[0077] Based on identity confirmation, combined with the user rights management system, permissions are dynamically assigned. For example, registered users can directly access the device; for unregistered users, further verification is required.

[0078] This implementation method designs a multi-level voiceprint comparison mechanism. When a speaker is detected for the first time, a rough similarity algorithm is first used for quick matching to screen out the most likely identity candidates, and then a detailed matching algorithm (such as support vector machine SVM or deep neural network DNN) is used to further confirm the identity. For each identified speaker, the system can combine its identity information to perform real-time permission allocation and log recording. For example, certain specific device controls can be automatically enabled based on the speaker's identity, or the content and behavior of their calls can be recorded.

[0079] Compared with the prior art, the streaming speaker log method provided by the present invention has the following beneficial effects:

[0080] 1) Traditional speaker log technology based on feature clustering needs to process the entire speech, resulting in high memory usage, large amount of calculation, and significant processing delay. However, the present invention adopts a streaming processing method, which can process and analyze continuous speech streams in real time without waiting for the entire speech file to be processed, thereby significantly improving processing efficiency and real-time performance.

[0081] 2) The present invention divides the speech stream into fixed-length segments and extracts the voiceprint features of each segment for similarity comparison, thereby realizing rapid and accurate detection of the speaker's transition points. This method is not only simple and easy to operate, but also can effectively deal with sudden changes in speech, providing a reliable basis for subsequent processing. This efficient transition detection mechanism enables the present invention to maintain a high degree of accuracy and stability when processing continuous speech streams in real time.

[0082] 3) Traditional technologies usually require the number of personnel categories to be specified, which makes it difficult to adapt to scenarios where the number of speakers changes dynamically. However, the present invention does not require the number of speakers to be preset in advance, but gradually registers voiceprints during execution, discovers new speakers through a temporary voiceprint database, and realizes unlimited processing of the number of speakers.

[0083] In a preferred embodiment of the present invention, an algorithm description of a streaming speaker log method is provided. Algorithm: Based on scalable streaming speaker log technology; Input: microphone voice stream or voice file, voiceprint comparison threshold, speaker jump detection segment duration, overlapping voice duration, VAD active voice detection segment duration, etc. Output: different speaker voice segments, speaker information.

[0084] In this embodiment, a computer program logic is used to describe the algorithm flow based on the scalable streaming speaker log technology. The specific implementation flow is as follows:

[0085] 1. Connect to the streaming speaker log interface and transmit the voice stream in segments:

[0086] This step refers to connecting the voice data source (such as microphone, voice file, etc.) with the voice processing system and transmitting the voice data to the system in segments in the form of streams. For the voice stream, a cut point is set at a fixed time interval, and the extracted voice is a fixed length before the cut point (for example: 200ms-1s).

[0087] 2. While (voice does not stop):

[0088] This is a loop, meaning that the system will continue to process the voice data as long as the voice stream continues.

[0089] 3. while (smaller voice clip):

[0090] The purpose of this inner loop is to buffer shorter speech segments until they reach a processable length. This is to ensure that there is enough speech data for subsequent analysis and processing, such as voice activity detection (VAD) and speaker recognition.

[0091] 4. Get active speech frames through VAD:

[0092] The voice activity detection (VAD) technology is used to extract valid voice frames (i.e., the part containing actual voice content) from the buffered voice data and remove silence or background noise.

[0093] 5. Duration of the for speaker jump detection segment and the duration of overlapping speech in active speech frames:

[0094] Each active speech frame is traversed and divided into certain time segments (such as 0.5 seconds) and overlapping durations (such as 0.25 seconds) so as to perform speaker change detection on these segments.

[0095] 6. Detect speaker jumps and record the jump position in the active speech frame:

[0096] A speaker change detection algorithm is executed on each segment to detect whether the speaker changes and record the position of the change.

[0097] 7. for voice segment in speaker jump detection position:

[0098] The speech segments before and after each detected speaker jump position are processed.

[0099] 8. Extract voiceprint features:

[0100] Voiceprint features are extracted from each speech segment. Voiceprint features are biometric features that uniquely represent the characteristics of the speaker's voice.

[0101] 9. If the voiceprint feature is not in the temporary voiceprint database:

[0102] Check whether the extracted voiceprint feature already exists in the temporary voiceprint database. If not, it means a new speaker is encountered.

[0103] 10. Register a new voiceprint and mark the temporary speaker information:

[0104] If a new speaker is encountered, the voiceprint feature is registered into the temporary voiceprint library and the relevant information (such as start time, duration, etc.) is marked.

[0105] 11. else returns the temporary speaker information:

[0106] If the voiceprint feature already exists in the temporary voiceprint library, it means that the current segment belongs to the identified speaker, and the information of the speaker is directly returned.

[0107] 12. for temporary speaker voice in temporary speaker library:

[0108] After processing all real-time voice data, traverse the records in the temporary voiceprint library.

[0109] 13. Compare the voiceprint with the physical voiceprint database and return the speaker information:

[0110] The voiceprint features of the temporary speaker are compared with the known voiceprint features in the entity voiceprint library to identify the exact identity of the speaker. If a match is successful, the speaker's identity information is returned.

[0111] The algorithm receives and processes voice streams in real time through streaming processing, and uses VAD, speaker jump detection, voiceprint extraction and comparison technologies to segment the voice stream into voice segments of different speakers and provide the speaker's identity information. The algorithm is scalable and can adjust parameters and modules according to different application scenarios and requirements to achieve optimal real-time performance and accuracy.

[0112] Figure 3 The structural diagram of the streaming speaker log system provided by the present invention is shown in FIG. Figure 3 , the streaming speaker log system includes:

[0113] VAD speech detection module, used to obtain speech stream and detect the speaker's speech in the speech stream using VAD algorithm;

[0114] A speaker transition detection module, used for slicing the speaker's speech, performing speaker transition detection based on the sliced ​​speech, and obtaining a speaker transition point;

[0115] The voiceprint comparison module is used to splice the voice stream slices before the speaker jump point, extract the voiceprint features of the spliced ​​voice segment, compare the extracted voiceprint features with the pre-established temporary voiceprint library, and determine the speaker category corresponding to the voiceprint features after the comparison is successful;

[0116] The temporary voiceprint library module is used to store the speaker category and its corresponding voiceprint features in the current voice stream.

[0117] The streaming speaker log system provided by the present invention executes the streaming speaker log methods provided by the aforementioned embodiments through the aforementioned modules. The streaming speaker log methods have been described in detail in the aforementioned embodiments, and will not be described in detail in this embodiment.

[0118] The present invention is scalable, and different modules can be replaced according to different application scenarios to make a better decision between accuracy and real-time performance.

[0119] Specifically, the present invention adopts a modular design concept, splitting the entire system into multiple independent but mutually cooperative modules, including a VAD voice detection module, a speaker jump detection module, a voiceprint comparison module, and an identity confirmation module. This modular design allows each module to be independently developed, tested, and optimized, thereby improving the overall maintainability and scalability of the system.

[0120] Flexible configuration capabilities:

[0121] Different modules can be selected for combination according to different application scenarios and requirements. For example, in scenarios with limited resources or high real-time requirements, smaller and faster modules can be selected; and in scenarios with higher accuracy requirements and more abundant resources, larger and more complex modules can be selected. This flexibility enables the present invention to adapt well to various application scenarios and requirements. By adjusting the configuration and parameters of different modules, the best balance between real-time and accuracy can be found. For example, by adjusting parameters such as the duration of the speaker jump detection segment, the duration of overlapping speech, and the duration of the VAD active speech detection segment, the real-time and accuracy of the system can be optimized.

[0122] The streaming speaker log system provided by the present invention has relatively simple model training for each module, and does not require a large amount of complex annotated speech and training process. This reduces the development cost and time of the system, allowing the system to be put into practical application more quickly. At the same time, the memory consumption of each module is small, which helps to reduce the resource occupation of the system and improve the operating efficiency of the system.

[0123] Figure 4 The structural block diagram of the electronic device provided by the present invention is as follows: Figure 4 As shown, the present invention also provides an electronic device, the electronic device 400 can be a computing device such as a mobile terminal, a desktop computer, a notebook, a palm computer, and a server. The electronic device 400 includes a processor 401 and a memory 402, wherein the memory 402 stores a streaming speaker log program 403.

[0124] In some embodiments, the memory 402 may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory 402 may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory 402 may also include both an internal storage unit of the computer device and an external storage device. The memory 402 is used to store application software and various types of data installed on the computer device, such as program codes installed on the computer device. The memory 402 may also be used to temporarily store data that has been output or is to be output. In one embodiment, when the streaming speaker log program 403 is executed by the processor 401, the following steps are implemented:

[0125] Acquire a speech stream and use a VAD algorithm to detect the speaker's voice in the speech stream;

[0126] Slicing the speaker's speech, performing speaker transition detection based on the sliced ​​speech, and obtaining a speaker transition point;

[0127] The sliced ​​speech streams before the speaker jump point are spliced, the voiceprint features of the spliced ​​speech segments are extracted, and the extracted voiceprint features are compared with the pre-established temporary voiceprint library. After the comparison is successful, the speaker category corresponding to the voiceprint features is determined.

[0128] In some embodiments, the processor 401 may be a central processing unit (CPU), a microprocessor or other data processing chip, and is used to run program codes stored in the memory 402 or process data, such as executing a streaming speaker log program.

[0129] This embodiment further provides a computer-readable storage medium, on which a streaming speaker log program is stored. When the streaming speaker log program is executed by a processor, the following steps are implemented:

[0130] Acquire a speech stream and use a VAD algorithm to detect the speaker's voice in the speech stream;

[0131] Slicing the speaker's speech, performing speaker transition detection based on the sliced ​​speech, and obtaining a speaker transition point;

[0132] The sliced ​​speech streams before the speaker jump point are spliced, the voiceprint features of the spliced ​​speech segments are extracted, and the extracted voiceprint features are compared with the pre-established temporary voiceprint library. After the comparison is successful, the speaker category corresponding to the voiceprint features is determined.

[0133] The above-described embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A streaming speaker log method, characterized in that: include: Acquire a speech stream and use a VAD algorithm to detect the speaker's voice in the speech stream; Slicing the speaker's speech, performing speaker transition detection based on the sliced ​​speech, and obtaining a speaker transition point; The sliced ​​speech streams before the speaker jump point are spliced, the voiceprint features of the spliced ​​speech segments are extracted, and the extracted voiceprint features are compared with the pre-established temporary voiceprint library. After the comparison is successful, the speaker category corresponding to the voiceprint features is determined.

2. The streaming speaker log method according to claim 1, characterized in that: After comparing the extracted voiceprint features with a pre-established temporary voiceprint library to determine the speaker category corresponding to the voiceprint features, the method further includes: The speaker's voiceprint features are compared with the known voiceprint features in the entity voiceprint library to identify the speaker's identity information.

3. The streaming speaker log method according to claim 1, characterized in that: After detecting the speaker's voice in the voice stream using the VAD algorithm, the method further includes: Perform noise reduction processing on the speaker's speech.

4. The streaming speaker log method according to claim 1, characterized in that: The speaker jump detection based on the sliced ​​speech specifically includes: Extracting speech features of the sliced ​​speech; wherein the speech features include frequency, amplitude, zero-crossing rate, Mel cepstral coefficients and linear prediction cepstral coefficients; Extract voiceprint features from speech features, and after accumulating a certain number of voiceprint features, calculate the similarity of the voiceprint features; The calculated similarity values ​​are analyzed to determine the speaker change point.

5. The streaming speaker log method according to claim 4, characterized in that: The voiceprint feature extraction is performed on the speech feature, and after a certain number of voiceprint features are accumulated, similarity calculation is performed on the voiceprint features, specifically including: A certain number of sliced ​​voices are spliced ​​into a voice matrix, and the voice matrix is ​​input into the voiceprint model to extract the voiceprint features of each sliced ​​voice; The similarity between each row of voiceprint features in the speech matrix and other rows of voiceprint features is calculated to obtain a similarity matrix. Each row of the similarity matrix is ​​searched according to the set similarity threshold to determine the jump point of this row. The speech before the jump point is identified as the speech of the same speaker.

6. The streaming speaker log method according to claim 1, characterized in that: After comparing the extracted voiceprint features with the pre-established temporary voiceprint library, the method further includes: If the voiceprint feature comparison fails and the system cannot classify the current voiceprint feature into a known speaker category in the temporary voiceprint library, the voiceprint feature is registered as a new category in the temporary voiceprint library.

7. The streaming speaker log method according to claim 1, characterized in that: After the comparison is successful, the method further comprises: The voiceprint features of the concatenated speech segment are smoothed with the voiceprint features of the corresponding speaker class in the temporary voiceprint library.

8. A streaming speaker log system applied to the streaming speaker log method according to any one of claims 1 to 7, characterized in that: include: VAD speech detection module, used to obtain speech stream and detect the speaker's speech in the speech stream using VAD algorithm; A speaker transition detection module, used for slicing the speaker's speech, performing speaker transition detection based on the sliced ​​speech, and obtaining a speaker transition point; The voiceprint comparison module is used to splice the voice stream slices before the speaker jump point, extract the voiceprint features of the spliced ​​voice segment, compare the extracted voiceprint features with the pre-established temporary voiceprint library, and determine the speaker category corresponding to the voiceprint features after the comparison is successful; The temporary voiceprint library module is used to store the speaker category and its corresponding voiceprint features in the current voice stream.

9. An electronic device, It is characterized in that comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the streaming speaker log method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the streaming speaker log method according to any one of claims 1 to 7.

Images