Speaker recognition method based on bipartite graph matching and electronic device

Abstract

The application discloses a speaker recognition method based on bipartite graph matching and an electronic device, and belongs to the technical field of audio recognition. The method comprises the following steps: acquiring an audio stream, and splitting the audio stream into continuous target time length audio segments in real time; determining a voiceprint embedding vector corresponding to each target time length audio segment; determining an embedding matrix based on the voiceprint embedding vector; determining a target voiceprint feature corresponding to each target time length audio segment according to a bipartite graph method; calculating the similarity between each voiceprint embedding vector; grouping each voiceprint embedding vector based on the similarity between each voiceprint embedding vector, clustering the voiceprint embedding vectors corresponding to the same speaker, obtaining a clustering result, and generating a target recognition report. The application can reduce the calculation complexity while improving the recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of audio recognition, and particularly relates to a speaker recognition method based on bipartite graph matching and an electronic device. BACKGROUND

[0002] Speaker recognition technology is a technology that realizes the identification and distinction of the identity of the speaker by analyzing the unique physiological and behavioral characteristics in the speech signal. This field has made significant progress in recent years and has shown important value in applications such as security verification, intelligent assistants, and speech transcription. However, existing technologies mostly extract and compare voiceprint features of the entire audio to identify the identity of the speaker. This method usually performs well in single speaker or short audio segments, but in complex application scenarios such as conference recordings, telephone calls, and video speech recognition, it faces challenges such as insufficient real-time performance, high complexity of long audio processing, difficulty in distinguishing multiple speakers, poor adaptability to dynamic change scenarios, and the like.

[0003] To address the above problems, the speaker recognition method based on bipartite graph matching and the electronic device in the present application are proposed. SUMMARY

[0004] To solve the deficiencies of the existing technology, the present application provides a speaker recognition method based on bipartite graph matching and an electronic device to solve the problems and challenges of insufficient real-time performance, high complexity of long audio processing, difficulty in distinguishing multiple speakers, poor adaptability to dynamic change scenarios, and the like in the existing technology.

[0005] The technical effects achieved by the present application are realized through the following solutions:

[0006] In a first aspect, the present application provides a speaker recognition method based on bipartite graph matching, which comprises:

[0007] obtaining an audio stream, and real-time dividing the audio stream into continuous target duration audio segments, the audio stream being an audio for multiple people speaking;

[0008] determining a voiceprint embedding vector corresponding to each target duration audio segment based on each target duration audio segment;

[0009] determining an embedding matrix based on the voiceprint embedding vector, wherein the embedding matrix is a structured representation of the voiceprint embedding vector;

[0010] determining a target voiceprint feature corresponding to each target duration audio segment according to a bipartite graph method;

[0011] calculating the similarity between each voiceprint embedding vector based on the embedding matrix and the target voiceprint feature;

[0012] grouping the voiceprint embedding vectors based on similarity between the voiceprint embedding vectors, clustering the voiceprint embedding vectors corresponding to the same speaker, and obtaining a clustering result;

[0013] generating a speaking time period and a duration of each person based on the clustering result and each of the target duration audio segments, and generating a target recognition report.

[0014] In some embodiments, the determining, according to a bipartite graph method, a target voiceprint feature corresponding to each of the target duration audio segments comprises:

[0015] constructing a corresponding bipartite graph structure by taking each of the target duration audio segments and a known voiceprint feature as two nodes of the bipartite graph, respectively; wherein the known voiceprint feature is dynamically changed in real time, and a new voiceprint feature acquired in real time is added to the known voiceprint feature in the identification process;

[0016] finding a first voiceprint feature that best matches each of the target duration audio segments in the bipartite graph structure, and determining the first voiceprint feature as the target voiceprint feature corresponding to each of the target duration audio segments; that is, the first voiceprint feature is determined from the known voiceprint feature.

[0017] In some embodiments, the similarity between the voiceprint embedding vectors is calculated according to a cosine similarity method.

[0018] In some embodiments, the voiceprint embedding vectors corresponding to the same speaker are clustered by using a K-means method or a hierarchical clustering method.

[0019] In some embodiments, the determining, based on each of the target duration audio segments, a voiceprint embedding vector corresponding to each of the target duration audio segments comprises:

[0020] determining, by using a deep learning model, the voiceprint embedding vector corresponding to each of the target duration audio segments from each of the target duration audio segments.

[0021] In some embodiments, the deep learning model comprises a convolutional neural network or a long short-term memory network.

[0022] In some embodiments, the target recognition report is used to present dynamic features of interactions between the speakers.

[0023] In some embodiments, the target duration audio segment is 25 seconds, 30 seconds, or 35 seconds.

[0024] In a second aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the speaker recognition method based on bipartite graph matching according to any one of the preceding aspects when executing the computer program.

[0025] In a third aspect, the present application provides a computer readable storage medium storing one or more programs, wherein the one or more programs are executable by one or more processors to implement the speaker recognition method based on bipartite graph matching according to any one of the preceding aspects.

[0026] By the speaker recognition method based on bipartite graph matching and the electronic device provided by the present application, in a multi-speaker scenario, the method uses a real-time updating and dynamic adaptation method, extracts high-frequency feature segments through voiceprint sampling, accelerates the feature comparison process by using the dichotomy matching, and optimizes the allocation of computing resources by combining the Top-K strategy, so as to significantly reduce the computing complexity while improving the recognition accuracy, and is especially suitable for complex long-time speech scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the prior art technical solutions, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0028] Figure 1 The flowchart of the speaker recognition method based on bipartite graph matching in an embodiment of the present application;

[0029] Figure 2 The schematic block diagram of the electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely in combination with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0031] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in one or more embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0032] Related audio recognition methods face the following core issues and challenges:

[0033] 1. Insufficient real-time performance

[0034] Traditional speaker recognition methods typically require collecting and processing complete audio data before analysis and output. This post-processing approach introduces significant time delays into the entire recognition process, making it difficult to meet real-time requirements. In dynamic scenarios, real-time speaker identification and updating are particularly critical. For example, in video conferencing, real-time speaker identification is fundamental for accurate captioning and immediate responses, a requirement that traditional methods cannot fulfill.

[0035] 2. Long audio processing is highly complex.

[0036] In processing long-duration audio data, fully analyzing all speech segments requires significant computational resources. This is especially true in multi-speaker scenarios, where the complexity of feature extraction and matching increases exponentially. The length and complexity of audio data significantly increase the computational load, placing higher demands on hardware resources and algorithm efficiency. For resource-constrained devices (such as mobile terminals), this high computational burden is particularly unbearable.

[0037] 3. Difficulty distinguishing between multiple speakers

[0038] In scenarios where multiple speakers take turns speaking, background noise, speech overlap, and similarity of speech features can all significantly affect the accuracy of speaker recognition. For example, in a conference room or telephone call, background noise may mask the speaker's speech features, and traditional methods struggle to accurately distinguish speakers when their voiceprints are highly similar. This is especially true when multiple speakers frequently take turns speaking, where traditional methods are prone to misidentification or underidentification.

[0039] 4. Poor adaptability to dynamic changing scenarios

[0040] In long-duration speech scenarios, speakers can dynamically change over time, such as new speakers joining or existing speakers' speech characteristics changing (e.g., speech rate or tone changing). Traditional global feature extraction methods often struggle to quickly adapt to new speech data due to their reliance on pre-extracted fixed patterns. This limitation can easily lead to recognition results lagging or distortion in real-world applications.

[0041] 5. Limitations of deep learning techniques

[0042] In recent years, deep learning techniques have achieved excellent performance in processing short audio segments. For example, Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) have shown high accuracy in speech feature extraction and classification. However, these models still face many challenges in processing long-duration audio and real-time recognition. Deep learning models typically require large amounts of training data and high computational power, and in multi-speaker scenarios, how to dynamically update the model and efficiently process overlapping speech remains an unsolved problem.

[0043] To address the above challenges, the present application attempts to overcome them by performing real-time speaker analysis every 30 seconds, combining binary matching algorithm, Top-K strategy, and speech sampling technology. This includes using binary matching to quickly narrow down the range of possible speech candidates, and using Top-K strategy to prioritize the most distinctive speech features for analysis. Speech sampling technology captures local audio features in a short time, enabling real-time updating and dynamic adaptation. For example, in multi-speaker scenarios, speech sampling extracts high-frequency feature segments, binary matching accelerates the feature comparison process, and Top-K strategy optimizes the allocation of computing resources, which can significantly reduce computational complexity while improving recognition accuracy, especially in complex long-duration speech scenarios.

[0044] The binary graph matching-based speaker recognition method of the present application has the following advantages:

[0045] 1. Using binary matching algorithm to improve processing efficiency

[0046] The present application combines a binary matching algorithm that can quickly narrow down the range of speech candidates, significantly improving the efficiency of speech feature matching. In the face of complex long-duration, multi-speaker audio data, the binary matching algorithm significantly reduces computational complexity by reducing the number of candidates that need to be compared. This technique is particularly important in dynamic, real-time application scenarios, as it can identify and update the identity of speakers in a short time, addressing the shortcomings of traditional methods in real-time performance.

[0047] 2. Top-K strategy optimizes feature selection

[0048] To address the diversity of voiceprint features in a multi-speaker scenario, the present application introduces a Top-K strategy, which prioritizes the most discriminative voiceprint features for analysis during each voiceprint comparison. In this way, the present application not only improves recognition accuracy but also optimizes the allocation of computing resources, avoiding excessive computation of low-discriminative features. In particular, in complex scenarios, the Top-K strategy effectively reduces computational burden and enhances the real-time response capability of the speaker recognition method based on bipartite graph matching of the present application.

[0049] 3. Voiceprint sampling technology enables dynamic adaptation

[0050] The present application introduces voiceprint sampling technology, which captures local audio features within a short time, enabling the speaker recognition method based on bipartite graph matching of the present application to dynamically update and adapt to new speech data. This is particularly important in scenarios where multiple speakers alternate speaking, allowing real-time differentiation of different speakers and avoiding recognition errors caused by overlapping speech or changes in speech rate in traditional methods. Voiceprint sampling technology enhances the adaptability of the speaker recognition method based on bipartite graph matching of the present application in long-term, variable environments, addressing the limitations of traditional global feature extraction methods that cannot quickly adapt to dynamic changes.

[0051] 4. Optimizing allocation of computing resources

[0052] When faced with long-term audio data, traditional methods often require a large amount of computing resources. The present application effectively reduces the amount of computation during each feature extraction and comparison by combining voiceprint sampling technology with bipartite matching and the Top-K strategy, thereby reducing the hardware resource requirements of the speaker recognition method based on bipartite graph matching of the present application. This optimization is particularly prominent on devices with limited computing resources, such as mobile terminals, allowing complex speaker recognition tasks to run efficiently in low-resource environments.

[0053] 5. Real-time and multi-speaker adaptability

[0054] In application scenarios such as meetings or phone calls where multiple speakers frequently alternate, the technical solution of the present application can update speaker identities in real time and accurately distinguish overlapping speech. Combined with bipartite matching and the Top-K strategy, the speaker recognition method based on bipartite graph matching of the present application can quickly adapt to different speaker changes, effectively addressing the poor adaptability and false recognition problems of existing technologies in dynamic and complex scenarios.

[0055] The various non-limiting embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0056] First, refer to Figure 1The method for speaker recognition based on bipartite graph matching of the present application is described in detail.

[0057] The present application provides a method for speaker recognition based on bipartite graph matching, which comprises:

[0058] S1: obtaining an audio stream, and real-time segmenting the audio stream into continuous target duration audio segments, wherein the audio stream is audio for multiple people speaking;

[0059] S2: determining, based on each target duration audio segment, a voiceprint embedding vector corresponding to each target duration audio segment;

[0060] S3: determining an embedding matrix based on the voiceprint embedding vectors, wherein the embedding matrix is a structured representation of the voiceprint embedding vectors;

[0061] S4: determining, according to a bipartite graph method, a target voiceprint feature corresponding to each target duration audio segment;

[0062] S5: calculating, based on the embedding matrix and the target voiceprint feature, a similarity between each voiceprint embedding vector;

[0063] S6: grouping each voiceprint embedding vector based on the similarity between each voiceprint embedding vector, clustering voiceprint embedding vectors corresponding to the same speaker, and obtaining a clustering result;

[0064] S7: generating, based on the clustering result and each target duration audio segment, a speaking time period and a duration for each person, and generating a target recognition report.

[0065] The technical solution of the present application not only makes up for the deficiencies of the prior art in real-time performance, computational complexity and multi-speaker adaptability, but also improves the accuracy and efficiency of speaker recognition. The method of the present application will provide more accurate and efficient technical support for intelligent meetings, real-time translation, speech transcription and other applications, and promote the wide application of speaker recognition technology in complex application scenarios.

[0066] Exemplarily, the continuous target duration audio segments can be obtained by a voiceprint embedding extraction module, specifically including:

[0067] A high-precision time axis segmentation technology is used to segment the audio stream into continuous target duration audio segments, such as 30-second segments, to ensure that each segment contains sufficient speech information for subsequent processing. In some embodiments, the target duration audio segment is 25 seconds, 30 seconds or 35 seconds.

[0068] In some embodiments, the target voiceprint feature corresponding to each target duration audio segment is determined according to a bipartite graph method, which comprises:

[0069] Each of the target duration audio segments and the known voiceprint features are taken as two nodes of a bipartite graph, respectively, to construct a corresponding bipartite graph structure; wherein the known voiceprint features are dynamically changed in real time, and a new voiceprint feature acquired in real time is added to the known voiceprint features in the identification process;

[0070] The first voiceprint feature that best matches each of the target duration audio segments is found in the bipartite graph structure, and the first voiceprint feature is determined as a target voiceprint feature corresponding to each of the target duration audio segments.

[0071] In some embodiments, the similarity between each of the voiceprint embedding vectors is calculated according to a cosine similarity method.

[0072] In some embodiments, the voiceprint embedding vectors corresponding to the same speaker are clustered by using a K-means method or a hierarchical clustering method to achieve accurate identification in a multi-speaker environment.

[0073] In some embodiments, the voiceprint embedding vector corresponding to each of the target duration audio segments is determined based on each of the target duration audio segments, including:

[0074] A deep learning model is used to determine the voiceprint embedding vector corresponding to each of the target duration audio segments from each of the target duration audio segments. The voiceprint embedding vector can effectively represent the unique acoustic features of the speaker. The extracted voiceprint embedding vector can be normalized to unify the numerical range and eliminate the scale difference between different audio segments, ensuring the stability and accuracy of subsequent algorithms.

[0075] In some embodiments, the deep learning model includes a convolutional neural network or a long short-term memory network.

[0076] After determining the voiceprint embedding vector corresponding to each of the target duration audio segments, based on the timestamp information of each target duration audio segment, the corresponding voiceprint embedding vector and its potential label (the label is used to identify the speaker identity or category of each audio segment) can be generated, ensuring that the label and the time corresponding relationship of each target duration audio segment are accurate.

[0077] Exemplarily, an optimization method such as the Hungarian algorithm can be used to find the best matching pair in the bipartite graph structure, to achieve accurate matching of each audio segment with the most possible voiceprint feature, and to improve the accuracy and efficiency of identification.

[0078] In some embodiments, the target identification report is used to present dynamic features of interactions between each speaker.

[0079] Exemplarily, time series analysis is performed on the clustering results of the target duration audio segments, to generate the speech time period and its duration of each speaker, and to comprehensively present the dynamic characteristics of multi-speaker interaction. The time series analysis results are integrated to output a detailed multi-speaker recognition report (i.e., target recognition report), which supports subsequent application scenarios such as conference recording, customer service analysis, etc., and improves the practicality and intelligent level of the speaker recognition method based on bipartite graph matching of the present application.

[0080] The present application brings a number of beneficial effects in the field of voiceprint recognition and audio analysis in multi-speaker environments, and has the following significant advantages over the prior art:

[0081] 1. Improved accuracy and stability of voiceprint recognition

[0082] By using advanced deep learning models (such as CNN and LSTM) to extract high-dimensional voiceprint embedding vectors, and combining with feature standardization processing, the present application significantly improves the accuracy and consistency of voiceprint recognition. The normalized embedding vectors effectively eliminate the numerical differences between different audio segments, ensuring the stable operation of subsequent matching and clustering algorithms.

[0083] 2. Efficient recognition ability in multi-speaker environment

[0084] Using bipartite graph matching analysis module and optimization matching algorithm (such as Hungarian algorithm), the present application can accurately distinguish and recognize the voiceprint features of different speakers in complex multi-speaker environment. By constructing an embedding matrix to describe the similarity relationship between audio segments, the speaker recognition method based on bipartite graph matching of the present application can efficiently process a large amount of audio data and adapt to the scenario of multiple speakers speaking at the same time.

[0085] 3. Enhanced similarity measurement and clustering analysis

[0086] Using cosine similarity measurement and advanced clustering algorithms (such as K-means and hierarchical clustering), the present application can accurately quantify the similarity between voiceprint embedding vectors, and effectively classify audio segments of the same speaker. This not only improves the accuracy of clustering, but also enhances the performance of the speaker recognition method based on bipartite graph matching of the present application in handling high-dimensional voiceprint data.

[0087] 4. Comprehensive time series analysis and comprehensive recognition

[0088] By performing time series analysis on the matching results of all audio segments, the scheme can generate the speaking time period and its duration of each speaker, providing detailed multi-speaker interaction dynamic information. This comprehensive recognition capability makes the speaker recognition method based on bipartite graph matching of the present application have higher practicality and intelligent level in actual applications, such as conference recording, customer service analysis, etc.

[0089] 5. Modular design, flexible and scalable

[0090] The present application adopts a modular technical architecture, and each module (voiceprint embedding extraction, dynamic label and embedding matrix construction, bipartite graph matching analysis, similarity measurement and clustering, multi-speaker comprehensive recognition) is independent and can be flexibly combined, facilitating system upgrading and function expansion. Users can flexibly adjust and optimize each module according to specific needs, improving the adaptability and scalability of the speaker recognition method based on bipartite graph matching of the present application.

[0091] 6. Optimized processing efficiency and resource utilization

[0092] Through accurate audio segmentation and efficient feature extraction method, the present application greatly improves the processing efficiency while ensuring the recognition accuracy. The optimized design of feature standardization and embedding matrix construction reduces the consumption of computing resources and improves the performance of the speaker recognition method based on bipartite graph matching of the present application in large-scale audio data processing.

[0093] The present application significantly improves the accuracy, efficiency and adaptability of voiceprint recognition by combining advanced voiceprint recognition technology and optimized algorithm in multi-speaker environment. These beneficial effects not only enhance the practical value of voiceprint recognition in practical applications, but also demonstrate the technical innovation and broad application prospects of the present application in the field of audio analysis and multi-speaker recognition.

[0094] It should be noted that the method of one or more embodiments of the present application can be performed by a single device, such as a computer or a server, etc. The method of the present embodiment can also be applied in a distributed scenario, and completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only perform one or more steps in the method of one or more embodiments of the present application, and the multiple devices will interact with each other to complete the method.

[0095] It is to be understood that the above- described embodiments are illustrative of only a few of the many permutations and combinations that are possible within the scope of the application. Other embodiments will be apparent to those of ordinary skill in the art upon encountering the disclosure herein. In some instances, the acts or steps described in the claims can be performed in an order other than as described in the embodiments, and still achieve desirable results. Additionally, the processes depicted in the attached figures do not necessarily require the particular order shown, or sequential order to achieve desirable results. In certain implementations, multitasking and parallel processing can be advantageous.

[0096] According to the same inventive concept, the application also discloses an electronic device corresponding to the method of any of the above embodiments.

[0097] Specifically, Figure 2 A hardware structure schematic diagram of an electronic device provided by the speaker recognition method based on bipartite graph matching is shown. The device can include a processor 410, a memory 420, an input / output interface 430, a communication interface 440, and a bus 450. The processor 410, the memory 420, the input / output interface 430, and the communication interface 440 are connected to each other through the bus 450 for internal communication.

[0098] The processor 410 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the application.

[0099] The memory 420 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 420 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the application are implemented by software or firmware, the related program codes are stored in the memory 420 and executed by the processor 410.

[0100] The input / output interface 430 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0101] The communication interface 440 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through wired mode (for example, USB, network cable, etc.), or can realize communication through wireless mode (for example, mobile network, WIFI, Bluetooth, etc.).

[0102] The bus 450 includes a path for transmitting information between various components (for example, the processor 410, the memory 420, the input / output interface 430, and the communication interface 440) of the device.

[0103] It should be noted that although the above device only shows the processor 410, the memory 420, the input / output interface 430, the communication interface 440, and the bus 450, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary for the implementation of the embodiments of the present application, and does not have to contain all the components shown in the figure.

[0104] The electronic device of the above embodiment is used to realize the speaker recognition method based on bipartite graph matching in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here.

[0105] Based on the same inventive concept, corresponding to the method of any of the above embodiments, one or more embodiments of the present application also provide a computer readable storage medium, which stores computer instructions for causing the computer to execute the speaker recognition method based on bipartite graph matching as described in any of the above embodiments.

[0106] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0107] The storage medium of the above embodiments stores computer instructions for causing the computer to execute the speaker recognition method based on bipartite graph matching as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here again.

[0108] It should be understood by those of ordinary skill in the art that the above discussion of any of the embodiments is merely exemplary and is not intended to suggest the scope of the application (including the claims) is limited to these examples; the embodiments above or technical features among different embodiments can also be combined, steps can be implemented in any order, and there are many other changes of different aspects of the one or more embodiments of the application as described above, which are not provided in detail for the sake of brevity.

[0109] In addition, in order to simplify the description and discussion, and so as not to make the one or more embodiments of the application difficult to understand, the well-known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, the apparatus can be shown in the form of a block diagram to avoid making the one or more embodiments of the application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram apparatus are highly dependent on the platform to be implemented by the one or more embodiments of the application (i.e., these details should be fully within the understanding of those skilled in the art). Where specific details (e.g., circuitry) are set forth in order to describe an exemplary embodiment of the application, it will be apparent to those skilled in the art that the one or more embodiments of the application can be practiced without these specific details or with variations on these specific details. Therefore, these descriptions should be considered as illustrative rather than limiting.

[0110] Although the application has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0111] The one or more embodiments of the application are intended to cover all such alternatives, modifications and variations as falling within the broad scope of the appended claims. Accordingly, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the one or more embodiments of the application should be included in the protection scope of the application.

Claims

1. A method for speaker recognition based on bipartite graph matching, characterized in that, The method comprises: acquiring an audio stream, and splitting the audio stream into continuous target time length audio segments in real time, the audio stream being audio of multiple people speaking; determining, based on each target time length audio segment, a voiceprint embedding vector corresponding to each target time length audio segment; determining an embedding matrix based on the voiceprint embedding vectors, wherein the embedding matrix is a structured representation of the voiceprint embedding vectors; determining, according to a bipartite graph method, a target voiceprint feature corresponding to each target time length audio segment; calculating, based on the embedding matrix and the target voiceprint feature, a similarity between each voiceprint embedding vector; grouping each voiceprint embedding vector based on the similarity between each voiceprint embedding vector, clustering voiceprint embedding vectors corresponding to the same speaker, and obtaining a clustering result; generating, based on the clustering result and each target time length audio segment, a speaking time period and a duration of each person, and generating a target recognition report; The method of determining, according to a bipartite graph method, a target voiceprint feature corresponding to each target time length audio segment comprises: constructing a corresponding bipartite graph structure by taking each target time length audio segment and a known voiceprint feature as two nodes of the bipartite graph, respectively; wherein the known voiceprint feature is dynamically changed in real time, and a new voiceprint feature acquired in real time is added to the known voiceprint feature during the recognition process; finding, in the bipartite graph structure, a first voiceprint feature that best matches each target time length audio segment, and determining the first voiceprint feature as a target voiceprint feature corresponding to each target time length audio segment.

2. The bipartite graph matching based speaker recognition method according to claim 1, characterized in that, The similarity between each voiceprint embedding vector is calculated according to a cosine similarity method.

3. The bipartite graph matching based speaker recognition method according to claim 2, characterized in that, The voiceprint embedding vectors corresponding to the same speaker are clustered by using a K-means method or a hierarchical clustering method.

4. The bipartite graph matching based speaker recognition method according to claim 3, characterized in that, The method of determining, based on each target time length audio segment, a voiceprint embedding vector corresponding to each target time length audio segment comprises: determining, by using a deep learning model, a voiceprint embedding vector corresponding to each target time length audio segment from each target time length audio segment.

5. The method of claim 4, wherein, The deep learning model comprises a convolutional neural network or a long short-term memory network.

6. The bipartite graph matching based speaker recognition method according to claim 1, characterized in that, The target recognition report is used to present dynamic characteristics of interactions between each speaker.

7. The bipartite graph matching based speaker recognition method according to claim 6, characterized in that, A target time length audio segment is 25 seconds, 30 seconds, or 35 seconds.

8. An electronic device, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the speaker recognition method based on bipartite graph matching according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores one or more programs executable by one or more processors to implement the speaker recognition method based on bipartite graph matching according to any one of claims 1 to 7.

Images