A Speaker Label Alignment Method, Device, Electronic Device, and Computer Readable Storage Medium

Abstract

Embodiments of the present invention provide a speaker label alignment method, apparatus, electronic device, and computer-readable storage medium. The method includes: obtaining N speaker logs of N channels, clustering each speaker log respectively to obtain N sets of clustered speaker labels, using the first target set of speaker labels as a reference label set, and performing alignment processing on the N - 1 second target sets of speaker labels other than the first target set of speaker labels based on the reference label set. It is possible to align the speaker labels corresponding to the multi-channel speaker logs, so that the sets of speaker labels corresponding to the speaker logs of multiple channels are no longer relative labels but absolute labels, further improving the accuracy of the speaker logs.

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition, and more particularly, to a method, apparatus, electronic device, and computer-readable storage medium for speaker label alignment. Background Art

[0002] With the continuous development of deep learning technology, the accuracy of speech recognition technology has been continuously improved, and the application has become more and more extensive. For the speech recognition scenario of a single person in the near field, speech recognition has been able to achieve a relatively high accuracy. However, for the speech recognition scenario of multiple people in the far field, it is still a difficult problem. One of the important problems lies in the speaker diarization technology, that is, it is first necessary to identify the speaking time of each speaker, and then the speech recognition technology can be used for speech recognition.

[0003] For the speech scenario of multiple people in the far field, we often use a multi-microphone device to obtain speech files of multiple channels. Before speech recognition, first use the speaker diarization technology to split the speech file into multiple single-speaker segments. For multiple channels, multiple segmentation results can be obtained. In the speaker diarization technology, a clustering algorithm is often used to obtain the segmentation result, but the clustering algorithm can only obtain relative labels and cannot obtain the absolute values of the labels. How to fuse the labels of multiple speaker diarizations to improve the accuracy of the speaker diarization algorithm is a difficult problem. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, apparatus, electronic device, and computer-readable storage medium for speaker label alignment, which can solve the problem of fusing the speaker diarization labels of multiple channels and improve the accuracy of speaker diarization.

[0005] In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for speaker label alignment, the method comprising:

[0007] Obtain N speaker diarizations of N channels, where each channel corresponds to one speaker diarization, and N is a positive integer;

[0008] Cluster each of the speaker diarizations to obtain N clustered speaker label sets;

[0009] Use a first target speaker label set as a reference label set, where the first target speaker label set is any one of the N speaker label sets;

[0010] Based on the reference label set, perform alignment processing on the N-1 second target speaker label sets other than the first target speaker label set.

[0011] In an alternative embodiment, the step of aligning the N - 1 second target speaker tag sets other than the first target speaker tag set based on the reference tag set includes:

[0012] Encoding the reference tag set and the N - 1 second target speaker tag sets respectively to obtain a first matrix and N - 1 second matrices, where the first matrix is the matrix obtained by encoding the reference tag set, and the second matrix is the matrix obtained by encoding the second target speaker tag set;

[0013] For each of the second matrices, calculating a distance matrix between the second matrix and the first matrix;

[0014] Based on the distance matrix, using a dynamic programming algorithm to calculate the shortest distance between the first matrix and the second matrix;

[0015] Determining the corresponding relationship between the reference tag set and the second target speaker tag set corresponding to the shortest distance;

[0016] Based on the corresponding relationship, aligning each tag in the second target speaker tag set with each tag in the reference tag set, where each tag in the second target speaker tag set corresponds one - to - one with each tag in the reference tag set.

[0017] In an alternative embodiment, the dynamic programming algorithm includes:

[0018]

[0019] Shortest distance = D(1,1), if Row(D) = 1;

[0020] where D is the distance matrix, i is the number of columns of the distance matrix, M 1,i is the matrix minor obtained by removing the first row and the i - th column of the distance matrix, F is the element value of the matrix minor, Row(D) is the number of rows of the distance matrix, and M2 is the dimension of the distance matrix.

[0021] In an alternative embodiment, the step of encoding the reference tag set and the N - 1 second target speaker tag sets respectively to obtain a first matrix and N - 1 second matrices includes:

[0022] Encoding the reference tag and the N - 1 second target speaker tag sets respectively using one - hot encoding to obtain a first matrix and N - 1 second matrices.

[0023] In an alternative embodiment, the method further includes:

[0024] Based on the reference speaker tag set, after aligning the N - 1 second target speaker tag sets other than the first target speaker tag set, N - 1 third target speaker tag sets are obtained;

[0025] When there are tags in the fourth target speaker tag set that are different from the N - m third target speaker tag sets, based on the N - m third target speaker tag sets, the abnormal tags in the fourth target speaker tag set are fused so that each tag in the fourth target speaker tag set is consistent with the N - m third target speaker tag sets, where m is a positive integer greater than or equal to 2, and the fourth target speaker tag set belongs to at least one of the N - 1 third target speaker tag sets.

[0026] In an alternative embodiment, the step of, when there are tags in the fourth target speaker tag set that are different from the N - m third target speaker tag sets, based on the N - m third target speaker tag sets, fusing the abnormal tags in the fourth target speaker tag set so that each tag in the fourth target speaker tag set is consistent with the N - m third target speaker tag sets includes:

[0027] Determine the positions of the abnormal tags in the fourth target speaker tag set;

[0028] Determine N - m target tags in the N - m third target speaker tag sets corresponding to the positions of the abnormal tags;

[0029] Determine the N - m speakers represented by the N - m target tags;

[0030] When the number of speakers represented by the N - m target tags being the same is greater than a preset number, change the abnormal tags in the fourth target speaker tag set to the tags corresponding to the same speaker.

[0031] In a second aspect, an embodiment of the present application provides a speaker tag alignment device, the device includes:

[0032] An acquisition module, configured to acquire N speaker logs of N channels, where each channel corresponds to one speaker log, and N is a positive integer;

[0033] A clustering module, configured to cluster each of the speaker logs respectively to obtain N clustered speaker tag sets;

[0034] A determination module, configured to use the first target speaker tag set as a reference tag set, where the first target speaker tag set is any one of the N speaker tag sets;

[0035] An alignment processing module, configured to perform alignment processing on N-1 second target speaker tag sets other than the first target speaker tag set based on the reference tag set.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the speaker tag alignment method are implemented.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the speaker tag alignment method are implemented.

[0038] The present application has the following beneficial effects:

[0039] The present application obtains N speaker logs of N channels, clusters each speaker log respectively to obtain N clustered speaker tag sets, uses the first target speaker tag set as the reference tag set, and based on the reference tag set, performs alignment processing on N-1 second target speaker tag sets other than the first target speaker tag set. It can align the speaker tags corresponding to the multi-channel speaker logs, so that the speaker tag sets corresponding to the speaker logs of multiple channels are no longer relative tags but absolute tags, thereby further improving the accuracy of the speaker logs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 It is a block diagram of the electronic device provided by the embodiment of the present invention;

[0042] Figure 2 It is one of the step flowcharts of the speaker tag alignment method provided by the embodiment of the present invention;

[0043] Figure 3 It is another step flowchart of the speaker tag alignment method provided by the embodiment of the present invention;

[0044] Figure 4 It is a third step flowchart of the speaker tag alignment method provided by the embodiment of the present invention;

[0045] Figure 5Structural block diagram of a speaker label alignment method provided by an embodiment of the present invention. Detailed implementation manners

[0046] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Components of the embodiments of the present invention usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0047] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0048] It should be noted that like reference numerals and letters denote like items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0049] In the description of the present invention, it should be noted that if terms such as "upper", "lower", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the present invention is usually placed during use, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation to the present invention.

[0050] In addition, terms such as "first" and "second" are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0051] In the description of the present application, it should also be noted that unless otherwise clearly defined and limited, the terms "set", "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific situations.

[0052] After extensive research by the inventors, it is found that for the voice scenario of multiple people in the far field, we often use multi-microphone devices to obtain voice files of multiple channels. Before speech recognition, the speaker diarization technique is first used to split the voice file into multiple single-speaker segments. For multiple channels, multiple segmentation results can be obtained. In the speaker diarization technique, clustering algorithms are often used to obtain the segmentation results. However, the clustering algorithms can only obtain relative labels and cannot obtain the absolute values of the labels. Therefore, the fusion of multiple relative labels is a difficult problem.

[0053] In view of the discovery of the above problems, this embodiment provides a method, apparatus, electronic device, and computer-readable storage medium for aligning speaker labels, which can align the speaker labels corresponding to the multi-channel speaker diarization, so that the set of speaker labels corresponding to the multi-channel speaker diarization is no longer relative labels but absolute labels, thereby further improving the accuracy of the speaker diarization. The solution provided in this embodiment will be elaborated in detail below.

[0054] This embodiment provides an electronic device that can align speaker labels. In a possible implementation, the electronic device may be a user terminal. For example, the electronic device may be, but is not limited to, a server, a smart phone, a personal computer (PC), a tablet computer, a personal digital assistant (PDA), a mobile internet device (MID), etc.

[0055] Please refer to Figure 1 , Figure 1 which is a schematic structural diagram of the electronic device 100 provided in the embodiment of the present application. The electronic device 100 may further include more or fewer components than those shown in Figure 1 or have a different configuration from that shown in Figure 1 . Figure 1 Each component shown in

[0056] The electronic device 100 includes a speaker label alignment device 110, a memory 120, and a processor 130.

[0057] The memory 120 and processor 130 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected via one or more communication buses or signal lines. The speaker label alignment device 110 includes at least one software functional module that can be stored in the memory 120 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 100. The processor 130 is configured to execute executable modules stored in the memory 120, such as the software functional modules and computer programs included in the speaker label alignment device 110.

[0058] The memory 120 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction.

[0059] Please refer to Figure 2 , Figure 2 For application Figure 1 A flowchart of a speaker label alignment method of the electronic device 100 is provided, and the method including each step is described in detail below.

[0060] Step 201: Obtain N speaker logs of N channels.

[0061] Each channel corresponds to a speaker log, and N is a positive integer.

[0062] Step 202: Cluster each speaker's log to obtain N clustered speaker tag sets.

[0063] Step 203: Use the first target speaker tag set as a reference tag set.

[0064] The first target speaker label set is any one of the N speaker label sets.

[0065] Step 204: Based on the reference speaker tag set, perform alignment processing on the N - 1 second target speaker tag sets other than the first target speaker tag set.

[0066] In a speech recognition scenario, multiple microphone devices are usually used to obtain audio files of multiple channels. Each channel corresponds to a speaker log, which can be understood as segmenting different speaker segments from a continuous multi - person conversation speech segment and determining which speaker each speech segment belongs to. For example: Channel 1 corresponds to Speaker A from 0 - 5s and Speaker B from 6 - 10s, and Channel 2 corresponds to Speaker B from 0 - 5s and Speaker A from 6 - 10s.

[0067] Cluster the speaker logs of each channel to obtain the clustered speaker tag set. For example: Channel 1 corresponds to Speaker A from 0 - 5s and Speaker B from 6 - 10s. After clustering the speaker log of this Channel 1, a speaker tag set of (1, 2) is obtained, where 1 is Speaker A and 2 is Speaker B. Channel 2 corresponds to Speaker B from 0 - 5s and Speaker A from 6 - 10s. After clustering the speaker log of Channel 2, a speaker tag set of (2, 1) is obtained.

[0068] Exemplarily, for the audio file of Channel 1, several voice segments containing human voices are generated via the VAD module. Using the voiceprint model, speaker features e1, e2, …, e are extracted based on a certain frame length / frame shift. N Using a clustering algorithm, such as Kmeans, spectral clustering, etc., assign speaker features to speaker tags to obtain a speaker tag set such as 11…2.

[0069] Select any one of the N speaker tag sets as the reference speaker tag set, and perform alignment processing on the N - 1 speaker tag sets other than the reference speaker tag set according to the reference speaker tag set, so that the final N - 1 second target speaker tag sets other than the reference speaker tag set are all aligned with the reference speaker tag set, improving the accuracy of the speaker log. Moreover, based on the aligned N speaker tag sets, speech recognition can also improve the accuracy of speech recognition.

[0070] This application obtains the N speaker logs of N channels, clusters each speaker log respectively to obtain the N clustered speaker tag sets, uses the first target speaker tag set as the reference speaker tag set, and based on the reference speaker tag set, performs alignment processing on the N - 1 second target speaker tag sets other than the first target speaker tag set. It can align the speaker tags corresponding to the multi - channel speaker logs, so that the speaker tag sets corresponding to the multi - channel speaker logs are no longer relative tags but absolute tags, thereby further improving the accuracy of the speaker log.

[0071] To perform comparison processing on N-1 second target speaker tag sets other than the first target speaker tag set, for step 204 above, as Figure 3 shown, it is a schematic flowchart of a speaker tag alignment method provided by an embodiment of the present application, which specifically includes the following steps:

[0072] Step 204-1: Encode the reference tag set and the N-1 second target speaker tag sets respectively to obtain a first matrix and N-1 second matrices.

[0073] Among them, the first matrix is the matrix obtained by encoding the reference tag set, and the second matrix is the matrix obtained by encoding the second target speaker tag set.

[0074] Step 20: Calculate the distance matrix between the second matrix and the first matrix for each second matrix.

[0075] Step 204-3: Based on the distance matrix, use the dynamic programming algorithm to calculate the shortest distance between the first matrix and the second matrix.

[0076] Step 204-4: Determine the corresponding relationship between the reference tag set and the second target speaker tag set corresponding to the shortest distance.

[0077] Step 204-5: Based on the corresponding relationship, align each tag in the second target speaker tag set with each tag in the reference tag set.

[0078] Among them, each tag in the second target speaker tag set corresponds one-to-one with each tag in the reference tag set.

[0079] Exemplarily, each speaker tag set includes a speaker tag set x1 and a speaker tag set x2. The speaker tag set x1 of channel 1 is selected as the reference tag set, and the speaker tag set x2 of channel 2 is selected as the tag set to be aligned, that is, the second target speaker tag set. Among them, both x1 and x2 are M-dimensional natural number vectors, and M is the number of speaker features.

[0080] Assume that H is the maximum value in the tag sets x1 and x2. For example, if the speaker tag set x1 contains three speakers and the speaker tag set x2 contains two speakers, then 3 is the maximum value in the tag sets x1 and x2. Convert the speaker tag sets x1 and x2 into one-hot encoding matrices E1 and E2, where both E1 and E2 are matrices with H rows and M columns. E1 is the first matrix, and E2 is the second matrix.

[0081] For example: when x1 = (1, 1, 2, 2, 1),

[0082] When x2 = (2, 2, 1, 1, 1),

[0083] Calculate the distance matrix D between the first matrix and the second matrix of channel 1 and channel 2, where the element D of this matrix ij is the distance between the i-th row of E1 and the j-th row of E2.

[0084] For the specific method of calculating the distance matrix between the first matrix and the second matrix, methods such as Euclidean distance and information content can be used to calculate the distance matrix.

[0085] Exemplarily, using the Euclidean distance calculation method, the distance matrix between the first matrix and the second matrix can be obtained:

[0086] Based on the distance matrix, using the dynamic programming algorithm, by calculating the shortest distance between channel 1 and channel 2, the shortest distance represents the optimal alignment method. The specific formula of the dynamic programming algorithm is as follows:

[0087]

[0088] Shortest distance = D(1,1), if Row(D) = 1;

[0089] where D is the distance matrix, i is the number of columns of the distance matrix, M 1,i is the matrix minor after removing the first row and the i-th column of the distance matrix, F is the element value of the matrix minor, Row(D) is the number of rows of the distance matrix, and M2 is the dimension of the distance matrix.

[0090] After the calculation is completed, the alignment method with the shortest distance between the speaker label of channel 2 and the speaker label set of channel 1 can be obtained. According to the above example, it can be concluded that label 1 of speaker label set x1 corresponds to label 2 of speaker label set x2, and label 2 of speaker label set x1 corresponds to label 1 of speaker label set x2. Thus, after aligning the second target speaker label set x2, it becomes x2 = (1, 1, 2, 2, 2).

[0091] When there are multiple second target speaker label sets, traverse each second target speaker label set, and convert each second target speaker label set into the alignment method with the smallest distance from the first target speaker label set x1.

[0092] Regarding how to fuse the label sets based on the aligned second target speaker label sets, as Figure 4 shown, it is a schematic flowchart of a speaker label alignment method provided by an embodiment of the present application, which specifically includes the following steps:

[0093] Step 301: Based on the reference tag set, after aligning the N - 1 second target speaker tag sets other than the first target speaker tag set, N - 1 third target speaker tag sets are obtained.

[0094] Step 302: When there are tags in the fourth target speaker tag set that are different from the N - m third target speaker tag sets, based on the N - m third target speaker tag sets, perform fusion processing on the abnormal tags in the fourth target speaker tag set so that each tag in the fourth target speaker tag set is consistent with the N - m third target speaker tag sets.

[0095] Where m is a positive integer greater than or equal to 2, and the fourth target speaker tag set belongs to at least one of the N - 1 third target speaker tag sets.

[0096] For the case where there are tags in the fourth target speaker tag set that are different from the N - m third target speaker tag sets, the method of performing fusion processing on the abnormal tags in the fourth target speaker tag set based on the N - m third target speaker tag sets can be as follows:

[0097] Determine the positions of the abnormal tags in the fourth target speaker tag set; determine the N - m target tags corresponding to the positions of the abnormal tags in the N - m third target speaker tag sets; determine the N - m speakers represented by the N - m target tags; when the number of the same speakers represented by the N - m target tags is greater than the preset number, change the abnormal tags in the fourth target speaker tag set to the tags corresponding to the same speakers.

[0098] To fuse all the aligned third target speaker tag sets, a simple voting average method can be selected. Determine whether there is a fourth target speaker tag set in the aligned third target speaker tag sets. The fourth target speaker tag set is the tag set with tags different from the N - m third speaker tag sets. Determine the tags in the fourth target speaker tag set that are different from the other third speaker tag sets, that is, the abnormal tags. Based on the N - m third target speaker tag sets, perform fusion processing on the abnormal tags in the fourth target speaker tag set so that each tag in the fourth target speaker tag set is consistent with the N - m third target speaker tag sets.

[0099] Exemplary: The third target speaker tag set includes: x2(1,1,2,2,1), x3(1,1,2,1,1), x4(1,1,2,2,1), x5(1,1,2,2,1). At this time, it is determined that there are different tags for the third target speaker tag set x3 compared to the other third target speaker tag sets x2, x4, and x5. Then x3 is determined as the fourth target speaker tag set, and the position of the abnormal tag in the fourth target speaker tag set x3 is determined, that is, the abnormal tag is the fourth element in the set.

[0100] Determine the fourth element in the sets of the third target speaker tag sets x2, x4, and x5, that is, the target tags are 2, 2, and 2 respectively. And the three target tags all represent the same speaker, and the number of target tags is 3. When the preset number is set to 2, the number of target cousins is greater than the preset number. Then change the abnormal tag 1 in the fourth target speaker tag set x3 to 2.

[0101] It should be noted that the preset number can be 2, 3, 5, etc., and the embodiments of the present application do not make specific limitations on this.

[0102] Please refer to Figure 5 , the embodiments of the present application also provide a speaker tag alignment device 110 applied to Figure 1 the electronic device 100. The speaker tag alignment device 110 includes:

[0103] An acquisition module 111, configured to acquire N speaker logs of N channels, where each channel corresponds to one speaker log, and N is a positive integer;

[0104] A clustering module 112, configured to cluster each of the speaker logs respectively to obtain N clustered speaker tag sets;

[0105] A determination module 113, configured to use the first target speaker tag set as a reference tag set, where the first target speaker tag set is any one of the N speaker tag sets;

[0106] An alignment processing module 114, configured to perform alignment processing on the N - 1 second target speaker tag sets except the first target speaker tag set based on the reference tag set.

[0107] Optionally, the alignment processing module 114 is specifically configured to:

[0108] The step of performing alignment processing on the N - 1 second target speaker tag sets except the first target speaker tag set based on the reference tag set includes:

[0109] Encode the reference tag set and the N-1 second target speaker tag sets respectively to obtain a first matrix and N-1 second matrices, where the first matrix is the matrix obtained by encoding the reference tag set, and the second matrix is the matrix obtained by encoding the second target speaker tag set;

[0110] For each of the second matrices, calculate the distance matrix between the second matrix and the first matrix;

[0111] Based on the distance matrix, use the dynamic programming algorithm to calculate the shortest distance between the first matrix and the second matrix;

[0112] Determine the correspondence between the reference tag set and the second target speaker tag set corresponding to the shortest distance;

[0113] Based on the correspondence, align each tag in the second target speaker tag set with each tag in the reference tag set, where each tag in the second target speaker tag set corresponds to each tag in the reference tag set one by one.

[0114] Optionally, the dynamic programming algorithm includes:

[0115]

[0116] Shortest distance = D(1,1), if Row(D) = 1;

[0117] where D is the distance matrix, i is the number of columns of the distance matrix, M 1,i is the matrix minor obtained by removing the first row and the i-th column of the distance matrix, F is the element value of the matrix minor, Row(D) is the number of rows of the distance matrix, and M2 is the dimension of the distance matrix.

[0118] Optionally, the alignment processing module 114 is further configured to:

[0119] Encode the reference tag and the N-1 second target speaker tag sets respectively using one-hot encoding to obtain a first matrix and N-1 second matrices.

[0120] Optionally, the apparatus further includes:

[0121] A fusion module 115, configured to, based on the reference tag set, align N - 1 second target speaker tag sets other than the first target speaker tag set, and obtain N - 1 third target speaker tag sets; when there are tags in the fourth target speaker tag set that are different from the N - m third target speaker tag sets, based on the N - m third target speaker tag sets, perform a fusion process on the abnormal tags in the fourth target speaker tag set, so that each tag in the fourth target speaker tag set is consistent with the N - m third target speaker tag sets, where m is a positive integer greater than or equal to 2, and the fourth target speaker tag set belongs to at least one of the N - 1 third target speaker tag sets.

[0122] Optionally, the fusion module 115 is further configured to:

[0123] Determine the positions of the abnormal tags in the fourth target speaker tag set;

[0124] Determine N - m target tags corresponding to the positions of the abnormal tags in the N - m third target speaker tag sets;

[0125] Determine N - m speakers represented by the N - m target tags;

[0126] When the number of speakers represented by the N - m target tags is greater than a preset number, change the abnormal tags in the fourth target speaker tag set to the tags corresponding to the same speaker.

[0127] In summary, this application obtains N speaker logs of N channels, performs clustering on each speaker log respectively to obtain N clustered speaker tag sets, uses the first target speaker tag set as the reference tag set, and based on the reference tag set, aligns N - 1 second target speaker tag sets other than the first target speaker tag set. It can align the speaker tags corresponding to the multi - channel speaker logs, so that the speaker tag sets corresponding to the multi - channel speaker logs are no longer relative tags but absolute tags, further improving the accuracy of the speaker logs.

[0128] This application further provides an electronic device 100, which includes a processor 130 and a memory 120. The memory 120 stores computer - executable instructions, and when the computer - executable instructions are executed by the processor 130, the speaker tag alignment method is implemented.

[0129] This application embodiment further provides a computer - readable storage medium, which stores a computer program. When the computer program is executed by the processor 130, the speaker tag alignment method is implemented.

[0130] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the part of the module, program segment, or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0131] In addition, the functional modules in each embodiment of this application can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part. If the functions are implemented in the form of software functional modules and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0132] It should be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0133] As described above, the above are only various embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A speaker label alignment method, characterized in that, The method includes: Obtaining N speaker logs of N channels, where each channel corresponds to a speaker log, and N is a positive integer; Performing clustering on each of the speaker logs respectively to obtain N clustered speaker label sets; Taking the first target speaker label set as the reference label set, where the first target speaker label set is any one of the N speaker label sets; Based on the reference label set, performing alignment processing on the N - 1 second target speaker label sets other than the first target speaker label set; the step of performing alignment processing on the N - 1 second target speaker label sets other than the first target speaker label set based on the reference label set includes: Encoding the reference label set and the N - 1 second target speaker label sets respectively to obtain a first matrix and N - 1 second matrices, where the first matrix is the matrix obtained by encoding the reference label set, and the second matrix is the matrix obtained by encoding the second target speaker label set; For each of the second matrices, calculating the distance matrix between the second matrix and the first matrix; Based on the distance matrix, using the dynamic programming algorithm to calculate the shortest distance between the first matrix and the second matrix; Determining the corresponding relationship between the reference label set and the second target speaker label set corresponding to the shortest distance; Based on the corresponding relationship, aligning each label in the second target speaker label set with each label in the reference label set, where each label in the second target speaker label set corresponds one-to-one with each label in the reference label set.

2. The method according to claim 1, characterized in that, The dynamic programming algorithm includes: Shortest distance = D(1,1), if Row(D) = 1; where D is the distance matrix, i is the number of columns of the distance matrix, M 1,i is the matrix minor obtained by removing the first row and the i-th column from the distance matrix, F is the element value of the matrix minor, Row(D) is the number of rows of the distance matrix, and M2 is the dimension of the distance matrix.

3. The method according to claim 1, wherein The step of encoding the reference label set and the N - 1 second target speaker label sets respectively to obtain a first matrix and N - 1 second matrices includes: Encoding the reference label and the N - 1 second target speaker label sets respectively using one - hot encoding to obtain a first matrix and N - 1 second matrices.

4. The method according to claim 1, wherein The method further includes: After performing alignment processing on the N - 1 second target speaker label sets other than the first target speaker label set based on the reference label set, obtaining N - 1 third target speaker label sets; When there are labels in the fourth target speaker label set that are different from the N - m third target speaker label sets, based on the N - m third target speaker label sets, performing fusion processing on the abnormal labels in the fourth target speaker label set so that each label in the fourth target speaker label set is consistent with the N - m third target speaker label sets, where m is a positive integer greater than or equal to 2, and the fourth target speaker label set belongs to at least one of the N - 1 third target speaker label sets.

5. The method according to claim 4, characterized in that, When there are tags in the fourth target speaker tag set that are different from the N - m third target speaker tag sets, the step of performing fusion processing on the abnormal tags in the fourth target speaker tag set based on the N - m third target speaker tag sets to make the tags in the fourth target speaker tag set consistent with the N - m third target speaker tag sets includes: Determine the positions of the abnormal tags in the fourth target speaker tag set; Determine N - m target tags corresponding to the positions of the abnormal tags in the N - m third target speaker tag sets; Determine the N - m speakers represented by the N - m target tags; When the number of speakers represented by the N - m target tags is greater than a preset number, change the abnormal tags in the fourth target speaker tag set to the tags corresponding to the same speaker.

6. A speaker label alignment device, characterized in that, The device includes: An acquisition module, configured to acquire N speaker logs of N channels, where each channel corresponds to one speaker log, and N is a positive integer; A clustering module, configured to perform clustering on each of the speaker logs respectively to obtain N clustered speaker tag sets; A determination module, configured to use the first target speaker tag set as a reference tag set, where the first target speaker tag set is any one of the N speaker tag sets; An alignment processing module, configured to perform alignment processing on the N - 1 second target speaker tag sets except the first target speaker tag set based on the reference tag set; specifically, the alignment processing module is configured to: The step of performing alignment processing on the N - 1 second target speaker tag sets except the first target speaker tag set based on the reference tag set includes: Encode the reference tag set and the N - 1 second target speaker tag sets respectively to obtain a first matrix and N - 1 second matrices, where the first matrix is the matrix obtained by encoding the reference tag set, and the second matrix is the matrix obtained by encoding the second target speaker tag set; For each of the second matrices, calculate the distance matrix between the second matrix and the first matrix; Based on the distance matrix, use the dynamic programming algorithm to calculate the shortest distance between the first matrix and the second matrix; Determine the corresponding relationship between the reference tag set and the second target speaker tag set corresponding to the shortest distance; Based on the corresponding relationship, align the tags in the second target speaker tag set with the tags in the reference tag set, where the tags in the second target speaker tag set and the tags in the reference tag set are in one - to - one correspondence.

7. An electronic device, characterized in that, It includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method according to any one of claims 1 - 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 - 5.

Images