Method and apparatus for generating real-time speaker logs based on speaker registration information

Abstract

This invention discloses a method and apparatus for generating real-time speaker logs based on speaker registration information. The method involves acquiring speaker registration information, including a set of registered speaker states and their corresponding pre-registered voiceprint feature vector sequences; acquiring real-time speech data and segmenting it into frames; extracting voiceprint feature vectors from each frame using a voiceprint model to obtain a real-time voiceprint feature vector sequence; inputting the real-time voiceprint feature vector sequence into an improved Hidden Markov Model (HMM) to decode and update the state set in real-time, and determining the real-time speaker state set based on the registered speaker state set; generating all possible paths based on the real-time speaker state set; calculating the probability of each speaker path based on the pre-registered voiceprint feature vector sequence; and outputting the speaker path with the highest probability as the optimal speaker path. By utilizing speaker registration information and considering contextual relationships, accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and specifically to a method and apparatus for generating real-time speaker logs based on speaker registration information. Background Technology

[0002] Speaker logging is a technology that marks the start and end times of each person's speech in a segment of audio. Real-time speaker logging requires continuously determining the speaker corresponding to the current audio segment as audio is collected. Offline speaker logging, on the other hand, processes a pre-collected audio segment.

[0003] Current real-time speaker logging technology requires pre-registration of the voiceprints of potential speakers in the speech, followed by frame-by-frame acquisition of voiceprint features from the collected speech. The voiceprint features of each frame are compared with the registered voiceprints, and the frame is assigned to the most similar speaker. The drawbacks of this method are its simple allocation mechanism, lack of consideration for contextual relationships, generally poor performance, and low accuracy. Summary of the Invention

[0004] In view of the aforementioned technical problems, the purpose of the embodiments of this application is to propose a method and apparatus for generating real-time speaker logs based on speaker registration information, so as to solve the technical problems mentioned in the background section above.

[0005] In a first aspect, the present invention provides a method for generating real-time speaker logs based on speaker registration information, comprising the following steps:

[0006] S1, obtain speaker registration information, which includes a set of registered speaker statuses and their corresponding pre-registered voiceprint feature vector sequences;

[0007] S2, acquire real-time speech data, divide the real-time speech data into frames, extract the voiceprint feature vector for each frame of real-time speech through the voiceprint model, and obtain the real-time voiceprint feature vector sequence.

[0008] S3, input the real-time voiceprint feature vector sequence into the improved hidden Markov model, set the initial real-time speaker state set to an empty set, decode and update the state set in real time, and determine the real-time speaker state set based on the registered speaker state set.

[0009] S4 generates all possible paths based on the real-time speaker state set, calculates the probability of each speaker path based on the pre-registered voiceprint feature vector sequence, and outputs the speaker path with the highest probability as the optimal speaker path.

[0010] Preferably, the probability of each speaker path is calculated based on the pre-registered voiceprint feature vector sequence, specifically including:

[0011] The real-time speaker state set is Y = {y1, y2, y3, ..., y...} t}, y t ∈Q, where Q is the set of registered speaker states, Q = {1, 2, 3, ..., n}, and n represents the nth registered speaker;

[0012] If the probability of the current path is P, then the probability of it transitioning to the next state is P. new = P * transition probability * generation probability;

[0013] The transition probability p(y) is determined based on whether the speaker's real-time state at time t is equal to the speaker's real-time state at time t-1. t |y t-1 ), y t y represents the real-time speaker state at time t. t-1 The speaker's real-time state at time t-1.

[0014] Based on the number of voiceprint feature vectors belonging to the sets of m real-time speakers in the newly generated voiceprint feature vector at time i, the generation probability is calculated as p(x). t |s m The real-time voiceprint feature vector sequence is X = {x1, x2, x3, ..., x...}. t}, where x t Let s be the newly generated voiceprint feature vector at time t. m Let be the voiceprint feature vector of the m-th real-time speaker.

[0015] As a preferred option, the transition probability p(y) t |y t-1 The calculation process is as follows:

[0016] When y t =y t-1 At that time, p(y) t |y t-1 ) = loopProb; when y t ≠y t-1 At that time, p(y) t |y t-1 ) = 1 - loopProb; where loopProb is the self-loop probability, and its value range is (0, 1).

[0017] As a preferred option, the generation probability is p(x) t |s m The calculation process is as follows:

[0018] At time t, when hour, m∈[1,n], where F a and F cr is a hyperparameter with a value range of (0, ∞). m This represents the pre-registered voiceprint feature vector of the m-th registered speaker;

[0019] At time t, when hour, m∈[1,n], where F b and F c This is a hyperparameter, with a value range of (-∞, ∞).

[0020] Preferably, the decoding in step S3 uses the Viterbi algorithm.

[0021] Preferably, the pre-registered voiceprint feature vector sequence is a set consisting of pre-registered voiceprint feature vectors generated from the pre-collected speech data of n registered speakers, R = {r1, r2, ..., r...} n};S m S represents the set of voiceprint feature vectors of m real-time speakers generated and updated during the dialogue. m ={s1, s2, ..., s m}, m∈[1,n].

[0022] Preferably, at time t, the voiceprint feature vector of the m-th real-time speaker calculated from the previous t-1 frames is:

[0023]

[0024] Where α is a hyperparameter.

[0025] Secondly, the present invention provides a real-time speaker log generation device based on speaker registration information, comprising:

[0026] The registration information acquisition module is configured to acquire speaker registration information, which includes a set of registered speaker statuses and their corresponding pre-registered voiceprint feature vector sequences.

[0027] The real-time voiceprint feature acquisition module is configured to acquire real-time speech data, divide the real-time speech data into frames, extract voiceprint feature vectors from each frame of real-time speech through a voiceprint model, and obtain a real-time voiceprint feature vector sequence.

[0028] The real-time speaker state set acquisition module is configured to input the real-time voiceprint feature vector sequence into the improved hidden Markov model, set the initial real-time speaker state set to an empty set, decode and update the state set in real time, and determine the real-time speaker state set based on the registered speaker state set.

[0029] The path output module is configured to generate all possible paths based on the real-time speaker state set, calculate the probability of each speaker path based on the pre-registered voiceprint feature vector sequence, and output the speaker path with the highest probability as the optimal speaker path.

[0030] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0031] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] (1) The real-time speaker log generation method based on speaker registration information proposed in this invention determines different generation probability calculation methods based on the statistical results of the voiceprint feature vectors in the first t-1 time period belonging to the voiceprint feature vector set of m real-time speakers. Speaker registration information is considered during allocation, making the calculation more accurate.

[0034] (2) The set of voiceprint feature vectors of m real-time speakers in the real-time speaker log generation method based on speaker registration information proposed in this invention is generated and updated in real time during the dialogue process. When the voiceprint feature vectors in the first t-1 time interval belong to the set of voiceprint feature vectors of m real-time speakers, the calculation formula of the generation probability needs to consider the voiceprint feature vector of the m-th real-time speaker and the voiceprint feature vector of the m-th registered speaker at the same time. The allocation result can be optimized by combining context information.

[0035] (3) The real-time speaker log generation method based on speaker registration information proposed in this invention utilizes existing speaker registration information and considers contextual relationships to improve the accuracy of allocation and avoid outputting speakers other than registered speakers in the optimal speaker path, resulting in better performance. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1This is an exemplary device architecture diagram in which an embodiment of this application can be applied;

[0038] Figure 2 This is a flowchart illustrating a real-time speaker log generation method based on speaker registration information, as an embodiment of this application.

[0039] Figure 3 This is a schematic diagram of the decoding process of a real-time speaker log generation method based on speaker registration information, as an embodiment of this application.

[0040] Figure 4 This is a schematic diagram of a real-time speaker log generation device based on speaker registration information, as an embodiment of this application.

[0041] Figure 5 This is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the present application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0043] Figure 1 An exemplary device architecture 100 is shown, which can be applied to the real-time speaker log generation method or the real-time speaker log generation apparatus based on speaker registration information according to the embodiments of this application.

[0044] like Figure 1 As shown, the device architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0045] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications, such as data processing applications and file processing applications, can be installed on terminal devices 101, 102, and 103.

[0046] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., software programs or software modules used to provide distributed services) or as a single software program or software module. No specific limitations are imposed here.

[0047] Server 105 can be a server that provides various services, such as a background data processing server that processes files or data uploaded by terminal devices 101, 102, and 103. The background data processing server can process the acquired files or data and generate processing results.

[0048] It should be noted that the real-time speaker log generation method based on speaker registration information provided in this application embodiment can be executed by server 105 or by terminal devices 101, 102, and 103. Correspondingly, the real-time speaker log generation device based on speaker registration information can be set in server 105 or in terminal devices 101, 102, and 103.

[0049] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Any number of terminal devices, networks, and servers can be included depending on implementation needs. If the data being processed does not need to be retrieved remotely, the above architecture may not include a network, requiring only servers or terminal devices.

[0050] Figure 2 An embodiment of this application illustrates a method for generating real-time speaker logs based on speaker registration information, comprising the following steps:

[0051] S1, obtain speaker registration information, which includes a set of registered speaker statuses and their corresponding pre-registered voiceprint feature vector sequences.

[0052] In a specific embodiment, this method is based on an improved Hidden Markov Model (HMM). The pre-registered voiceprint feature vector sequence R is a set of pre-registered voiceprint feature vectors generated from the pre-collected speech data of n registered speakers, i.e., R = {r1, r2, ..., r...} n};S m S represents the set of voiceprint feature vectors of m real-time speakers generated and updated during the dialogue. m ={s1, s2, ..., s m}, m∈[1,n].

[0053] In a specific embodiment, at time t, the voiceprint feature vector of the m-th real-time speaker calculated from the previous t-1 frames is:

[0054]

[0055] Where α is a hyperparameter.

[0056] Specifically, given the registration information of n speakers, the set of registered speaker states is Q = {1, 2, 3, ..., n}, where n represents the nth registered speaker. Correspondingly, the pre-registered voiceprint feature vector of the nth registered speaker is r. n The final pre-registered voiceprint feature vector sequence is R = {r1, r2, ..., r...} n}

[0057] S2, acquire real-time speech data, divide the real-time speech data into frames, extract the voiceprint feature vector for each frame of real-time speech through the voiceprint model, and obtain the real-time voiceprint feature vector sequence.

[0058] Specifically, the input sequence (observation sequence) of the improved Hidden Markov Model is X = {x1, x2, x3, ..., x...} t} where t is the current time. X is the process of dividing a segment of real-time speech data into frames. Each frame of real-time speech is processed by extracting a voiceprint feature vector through a voiceprint model, forming a real-time voiceprint feature vector sequence. The window length and window shift for framing are set according to requirements, and are generally set to 1.5s and 0.25s respectively. The voiceprint model can be d-vector, x-vector, ResNet, etc.

[0059] S3. Input the real-time voiceprint feature vector sequence into the improved hidden Markov model, set the initial real-time speaker state set to an empty set, decode and update the state set in real time, and determine the real-time speaker state set based on the registered speaker state set.

[0060] In a specific embodiment, the decoding in step S3 uses the Viterbi algorithm.

[0061] Specifically, the initial real-time speaker state set is empty. As the input sequence is processed, new speaker labels are found from the registered speaker state set and added to the real-time speaker state set. Simultaneously, the set of real-time speaker voiceprint feature vectors S... m It will also be updated accordingly, that is to say, S mIt is generated and updated during the dialogue. The initial state probability π = {1}, because the initial real-time speaker state set is empty, so this value is 1. The real-time speaker state set is determined based on the registered speaker state set. The real-time speaker state set corresponds to the real-time speaker label. The values ​​in the real-time speaker state set belong to the registered speaker state set. That is, if there are n registered speakers in the dialogue, the values ​​in the real-time speaker state set can only be selected from these n registered speakers. For example, there will not be a situation where there are only 2 people in the dialogue, but the result is assigned to 3 people. Therefore, the registered speaker state set is used to restrict the values ​​in the real-time speaker state set. Given the registered speaker information, considering the context information, the calculation results will be more accurate.

[0062] S4 generates all possible paths based on the real-time speaker state set, calculates the probability of each speaker path based on the pre-registered voiceprint feature vector sequence, and outputs the speaker path with the highest probability as the optimal speaker path.

[0063] In a specific embodiment, the probability of each speaker path is calculated based on the pre-registered voiceprint feature vector sequence, specifically including:

[0064] The real-time speaker state set is Y = {y1, y2, y3, ..., y...} t}, y t ∈Q, where Q is the set of registered speaker states, Q = {1, 2, 3, ..., n}, and n represents the nth registered speaker;

[0065] Assuming the probability of the current path is P, then the probability of it transitioning to the next state (new speaker) is also P. new = P * transition probability * generation probability;

[0066] The transition probability p(y) is determined based on whether the speaker's real-time state at time t is equal to the speaker's real-time state at time t-1. t |y t-1 ), y t y represents the real-time speaker state at time t. t-1 The speaker's real-time state at time t-1.

[0067] Based on the number of voiceprint feature vectors belonging to the sets of m real-time speakers in the newly generated voiceprint feature vector at time i, the generation probability is calculated as p(x). t |s m The real-time voiceprint feature vector sequence is X = {x1, x2, x3, ..., x...}. t}, where x t Let s be the newly generated voiceprint feature vector at time t. m Let be the voiceprint feature vector of the m-th real-time speaker.

[0068] In a specific embodiment, the transition probability p(y) t |y t-1 The calculation process is as follows:

[0069] When y t =y t-1 At that time, p(y) t |y t-1 ) = loopProb; when y t ≠y t-1 At that time, p(y) t |y t-1 ) = 1 - loopProb; where loopProb is the self-loop probability, and its value range is (0, 1).

[0070] Specifically, if the real-time speaker state at time t is equal to the real-time speaker state at time t-1, then p(y) t |y t-1 ) = loopProb; If the real-time speaker state at time t is not equal to the real-time speaker state at time t-1, then p(y) = loopProb; t |y t-1 ) = 1 - loopProb, where loopProb is 0 to 1.

[0071] In a specific embodiment, the generation probability is p(x) t |s m The calculation process is as follows:

[0072] At time t, when hour, m∈[1,n], where F a and F c r is a hyperparameter with a value range of (0, ∞). m This represents the pre-registered voiceprint feature vector of the m-th registered speaker;

[0073] At time t, when hour, m∈[1,n], where F b and F c This is a hyperparameter, with a value range of (-∞, ∞).

[0074] Specifically, the calculation of the generation probability requires determining how many frames of real-time speech in the first t-1 frames belong to the set of voiceprint feature vectors of m real-time speakers. If no frame of real-time speech has a corresponding real-time voiceprint feature vector belonging to the set of voiceprint feature vectors of m real-time speakers, then the probability of generation is determined. m When it is a zero vector, the generation probability is... m∈[1,n]; when there is a frame of real-time speech whose real-time voiceprint feature vector belongs to the set S of voiceprint feature vectors of m real-time speakers. m When, the generation probability is m∈[1,n], and the speaker registration information is taken into account during the allocation calculation, making the calculation more accurate. The number of all possible paths generated by the real-time speaker state set is m. t At frame t, the path with the highest probability is used as the decoding result.

[0075] Hyperparameters loopProb, F a F b F c The α optimization method uses grid search, which involves preparing a set of labeled test data and seeking the optimal solution on it. Specific Implementation

[0077] Suppose we have a dialogue with 3 frames of audio data, and the corresponding real-time voiceprint feature vector sequence is {x1, x2, x3}, where the actual speaker order is {1, 2, 1} (this value is unknown). There are two people in the dialogue. We pre-extract the registered voices of the two people and generate a pre-registered voiceprint feature vector sequence R = {r1, r2}. In addition, the set of registered speaker states is Q = {1, 2}.

[0078] like Figure 3 The diagram shows the decoding flowchart, where each column corresponds to the input of each frame, and each circle represents the speaker to which the corresponding frame jumps. There are two speakers in the decoding process, corresponding to {1, 2}.

[0079] 1. Input x1. Y has two possible paths: {1} and {2}. Calculate the probability of each path and update S accordingly. m Output the optimal speaker path: Y = {1}. At this point, S... m ={s1}.

[0080] 2. Input x2. The possible paths of Y are {1, 1}, {1, 2}, {2, 1}, and {2, 2}, a total of 4. Calculate the probability of each path and update S. m Output the optimal speaker path: Y = {1, 2}. A new speaker is then generated, and S... m = {s1, s2}.

[0081] 3. Input x3. The possible paths of Y are {1, 1, 1}, {1, 1, 2}, {1, 2, 1}, {1, 2, 2}, {2, 1, 1}, {2, 1, 2}, {2, 2, 1}, {2, 2, 2}, a total of 8 paths. Calculate the probability of each path and update S. m Output the optimal speaker path: Y = {1, 2, 1}. At this point, S... m= {s1, s2}.

[0082] Specifically, each new frame of real-time speech data is decoded to generate all possible paths. The probability of each path is calculated, and the path with the highest probability is output as the optimal speaker path. In actual calculations, Hidden Markov Models (HMMs) are used, and S is calculated based on the allocation. m Therefore, it will affect the allocation of the current real-time voiceprint feature vector x, S m The real-time speaker log changes as new real-time speech corresponding to the real-time voiceprint feature vector is input during the real-time speaker log generation process, enabling the combination of contextual information.

[0083] Further reference Figure 4 As an implementation of the methods shown in the above figures, this application provides an embodiment of a real-time speaker log generation device based on speaker registration information. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0084] This application provides a real-time speaker log generation device based on speaker registration information, including:

[0085] The registration information acquisition module 1 is configured to acquire speaker registration information, which includes a set of registered speaker statuses and their corresponding pre-registered voiceprint feature vector sequences.

[0086] The real-time voiceprint feature acquisition module 2 is configured to acquire real-time speech data, divide the real-time speech data into frames, extract voiceprint feature vectors for each frame of real-time speech through a voiceprint model, and obtain a real-time voiceprint feature vector sequence.

[0087] The real-time speaker state set acquisition module 3 is configured to input the real-time voiceprint feature vector sequence into the improved hidden Markov model, set the initial real-time speaker state set to an empty set, decode and update the state set in real time, and determine the real-time speaker state set based on the registered speaker state set.

[0088] The path output module 4 is configured to generate all possible paths based on the real-time speaker state set, calculate the probability of each speaker path based on the pre-registered voiceprint feature vector sequence, and output the speaker path with the highest probability as the optimal speaker path.

[0089] The following is for reference. Figure 5 It illustrates an electronic device suitable for implementing embodiments of this application (e.g., Figure 1 The diagram shows the structure of a computer device 500 (a server or terminal device). Figure 5The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0090] like Figure 5 As shown, the computer device 500 includes a central processing unit (CPU) 501 and a graphics processing unit (GPU) 502, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 503 or programs loaded from storage section 509 into random access memory (RAM) 504. The RAM 504 also stores various programs and data required for the operation of the device 500. The CPU 501, GPU 502, ROM 503, and RAM 504 are interconnected via a bus 505. An input / output (I / O) interface 506 is also connected to the bus 505.

[0091] The following components are connected to I / O interface 506: an input section 507 including a keyboard, mouse, etc.; an output section 508 including an LCD, speakers, etc.; a storage section 509 including a hard disk, etc.; and a communication section 510 including a network interface card, such as a LAN card or modem. The communication section 510 performs communication processing via a network such as the Internet. A drive 511 may also be connected to I / O interface 506 as needed. A removable medium 512, such as a hard disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 511 as needed so that computer programs read from it can be installed into storage section 509 as needed.

[0092] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 510, and / or installed from removable medium 512. When the computer program is executed by central processing unit (CPU) 501 and graphics processing unit (GPU) 502, the functions defined in the methods of this application are performed.

[0093] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any combination thereof. More specific examples of a computer-readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than a computer-readable medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution device, apparatus, or apparatus. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0094] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using dedicated hardware-based means to perform the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] The modules described in the embodiments of this application can be implemented in software or hardware. These modules can also be located within a processor.

[0097] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire speaker registration information, including a set of registered speaker states and their corresponding pre-registered voiceprint feature vector sequences; acquire real-time speech data, and divide the real-time speech data into frames, extracting voiceprint feature vectors from each frame of real-time speech using a voiceprint model to obtain a real-time voiceprint feature vector sequence; input the real-time voiceprint feature vector sequence into an improved hidden Markov model, setting the initial real-time speaker state set to an empty set, decoding and updating the state set in real-time, and determining the real-time speaker state set based on the registered speaker state set; generate all possible paths based on the real-time speaker state set, calculate the probability of each speaker path based on the pre-registered voiceprint feature vector sequence, and output the current speaker path with the highest probability as the optimal speaker path.

[0098] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for generating real-time speaker logs based on speaker registration information, characterized in that, Includes the following steps: S1, Obtain speaker registration information, the speaker registration information including a set of registered speaker statuses and their corresponding pre-registered voiceprint feature vector sequences; S2, acquire real-time speech data, divide the real-time speech data into frames, extract the voiceprint feature vector for each frame of real-time speech through the voiceprint model, and obtain the real-time voiceprint feature vector sequence. S3, input the real-time voiceprint feature vector sequence into the improved hidden Markov model, set the initial real-time speaker state set to an empty set, decode and update the real-time speaker state set in real time, and determine the real-time speaker state set according to the registered speaker state set. S4, generate all possible paths based on the real-time speaker state set, and calculate the probability of each speaker path based on the pre-registered voiceprint feature vector sequence, specifically including: The real-time speaker state set is Y = {y1, y2, y3, ..., y...} t }, Q is the set of registered speaker states, Q={1, 2, 3, ..., n}, where n represents the nth registered speaker; If the probability of the current path is P, then the probability of it transitioning to the next state is P. new =P * Transition probability * Generation probability; The transition probability p(y) is determined based on whether the speaker's real-time state at time t is equal to the speaker's real-time state at time t-1. t |y t-1 ), y t y represents the real-time speaker state at time t. t-1 The speaker's real-time state at time t-1. Based on the newly generated voiceprint feature vector at time i The set of voiceprint feature vectors belonging to m real-time speakers The number of [elements], and the generation probability is calculated as [the number of elements]. The generation probability is The calculation process is as follows: At time t, when hour, , of which F a and F c This is a hyperparameter, and its value range is (0,∞). This represents the pre-registered voiceprint feature vector of the m-th registered speaker; S represents the set of voiceprint feature vectors of m real-time speakers generated and updated during the dialogue. m ={s1, s2, ..., s} m }, ; At time t, when hour, , of which F b and F c The hyperparameter has a value range of (-∞, ∞); the real-time voiceprint feature vector sequence is X = {x1, x2, x3, ..., x...} t },in, Let be the newly generated voiceprint feature vector at time t. Let m be the voiceprint feature vector of the m-th real-time speaker; and output the optimal speaker path as the speaker path with the highest probability at present.

2. The real-time speaker log generation method based on speaker registration information according to claim 1, characterized in that, The transition probability p(y) t |y t-1 The calculation process is as follows: When y t =y t-1 At that time, p(y) t |y t-1 )=loopProb; when y t ≠y t-1 At that time, p(y) t |y t-1 ) = 1 - loopProb; where loopProb is the self-loop probability, and its value range is (0, 1).

3. The real-time speaker log generation method based on speaker registration information according to claim 1, characterized in that, The decoding in step S3 uses the Viterbi algorithm.

4. The real-time speaker log generation method based on speaker registration information according to claim 1, characterized in that, The pre-registered voiceprint feature vector sequence is a set of pre-registered voiceprint feature vectors generated from the pre-collected speech data of n registered speakers, R={r1, r2, ..., r...} n } 5. The real-time speaker log generation method based on speaker registration information according to claim 1, characterized in that, At time t, the voiceprint feature vector of the m-th real-time speaker, calculated from the previous t-1 frames, is: ； Where α is a hyperparameter.

6. A real-time speaker log generation device based on speaker registration information, characterized in that, include: The registration information acquisition module is configured to acquire speaker registration information, which includes a set of registered speaker statuses and their corresponding pre-registered voiceprint feature vector sequences. The real-time voiceprint feature acquisition module is configured to acquire real-time speech data, divide the real-time speech data into frames, extract voiceprint feature vectors from each frame of real-time speech through a voiceprint model, and obtain a real-time voiceprint feature vector sequence. The real-time speaker state set acquisition module is configured to input the real-time voiceprint feature vector sequence into the improved hidden Markov model, set the initial real-time speaker state set to an empty set, decode and update the real-time speaker state set in real time, and determine the real-time speaker state set based on the registered speaker state set. The path output module is configured to generate all possible paths based on the real-time speaker state set, and calculate the probability of each speaker path based on the pre-registered voiceprint feature vector sequence, specifically including: The real-time speaker state set is Y = {y1, y2, y3, ..., y...} t }, Q is the set of registered speaker states, Q={1, 2, 3, ..., n}, where n represents the nth registered speaker; If the probability of the current path is P, then the probability of it transitioning to the next state is P. new =P * Transition probability * Generation probability; The transition probability p(y) is determined based on whether the speaker's real-time state at time t is equal to the speaker's real-time state at time t-1. t |y t-1 ), y t y represents the real-time speaker state at time t. t-1 The speaker's real-time state at time t-1. Based on the newly generated voiceprint feature vector at time i The set of voiceprint feature vectors belonging to m real-time speakers The number of [elements], and the generation probability is calculated as [the number of elements]. The generation probability is The calculation process is as follows: At time t, when hour, , of which F a and F c This is a hyperparameter, and its value range is (0,∞). This represents the pre-registered voiceprint feature vector of the m-th registered speaker; S represents the set of voiceprint feature vectors of m real-time speakers generated and updated during the dialogue. m ={s1, s2, ..., s} m }, ; At time t, when hour, , of which F b and F c The hyperparameter has a value range of (-∞, ∞); the real-time voiceprint feature vector sequence is X = {x1, x2, x3, ..., x...} t },in, Let be the newly generated voiceprint feature vector at time t. Let m be the voiceprint feature vector of the m-th real-time speaker; and output the optimal speaker path as the speaker path with the highest probability at present.

7. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

Images