Voice data processing method and device, computer readable medium and computer equipment

Abstract

Embodiments of the present application provide a voice data processing method, device, computer readable medium and computer device. The method comprises: segmenting first voice data according to at least two segmentation manners to obtain at least two voice segment sets; extracting a voiceprint feature vector corresponding to each of two voice segments included in each voice segment set; calculating a similarity between the voiceprint feature vectors corresponding to the two voice segments respectively included in each voice segment set; selecting a lowest similarity from similarities corresponding to the plurality of voice segment sets; and determining a segmentation time point between the two voice segments in the voice segment set corresponding to the lowest similarity to determine a speaker change time point in the first voice data, if the lowest similarity is lower than or equal to a set threshold. The embodiments of the present application can improve the reliability and accuracy of speaker change detection in voice data, and effectively reduce false detection caused by noise or other interference factors.

Description

Technical Field

[0001] This application relates to the fields of computer and communication technology, and more specifically, to a voice data processing method, apparatus, computer-readable medium, and computer equipment. Background Technology

[0002] Speaker embedding is a mathematical representation used to represent and identify the characteristics of an individual's voiceprint. It is a high-dimensional vector constructed by extracting the speaker's acoustic features (such as pitch, timbre, and speech rate), facilitating similarity comparison and matching in databases to achieve identity recognition. In applications such as online meetings and telephone customer service, multiple people may express their opinions in a certain order or randomly. Therefore, recognizing the voices of different speakers is particularly important. However, the technical solutions proposed in related technologies cannot accurately identify whether the speaker in the voice data has changed. Summary of the Invention

[0003] The embodiments of this application provide a speech data processing method, apparatus, computer-readable medium, and computer device, which can improve the reliability and accuracy of speaker change detection in speech data and effectively reduce false detections caused by noise or other interference factors.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part by practice of this application.

[0005] According to one aspect of the embodiments of this application, a speech data processing method is provided, comprising: segmenting first speech data according to at least two segmentation methods to obtain at least two speech segment sets, wherein each speech segment set corresponds to one of the at least two segmentation methods, and each speech segment set contains two speech segments; for each speech segment set in the at least two speech segment sets, extracting voiceprint feature vectors corresponding to the two speech segments respectively; for each speech segment set in the at least two speech segment sets, calculating the similarity between the voiceprint feature vectors corresponding to the two speech segments respectively, obtaining the similarity corresponding to each speech segment set, and selecting the lowest similarity from the similarities corresponding to the plurality of speech segment sets respectively; if the lowest similarity is lower than or equal to a set threshold, determining the segmentation time point between the two speech segments in the speech segment set corresponding to the lowest similarity, so as to determine the speech data change time points of different speakers in the first speech data.

[0006] According to one aspect of the embodiments of this application, a voice data processing apparatus is provided, comprising: a segmentation unit configured to segment first voice data according to at least two segmentation methods to obtain at least two voice segment sets, wherein each voice segment set corresponds to one of the at least two segmentation methods, and each voice segment set contains two voice segments; an extraction unit configured to extract voiceprint feature vectors corresponding to the two voice segments for each voice segment set in the at least two voice segment sets; a calculation unit configured to calculate the similarity between the voiceprint feature vectors corresponding to the two voice segments for each voice segment set in the at least two voice segment sets, to obtain the similarity corresponding to each voice segment set, and to select the lowest similarity from the similarities corresponding to the plurality of voice segment sets; and a processing unit configured to determine the segmentation time point between the two voice segments in the voice segment set corresponding to the lowest similarity if the lowest similarity is lower than or equal to a set threshold, so as to determine the time point of voice data change of different speakers in the first voice data.

[0007] In some embodiments of this application, based on the foregoing scheme, the segmentation unit is configured to: segment the first speech data according to a first segmentation duration to obtain a speech segment set corresponding to the first segmentation duration; gradually increase the first segmentation duration according to a set step size to obtain at least one second segmentation duration, and segment the first speech data according to the at least one second segmentation duration to obtain a speech segment set corresponding to the at least one second segmentation duration.

[0008] In some embodiments of this application, based on the foregoing scheme, the partitioning unit is further configured to: divide the second speech data into at least two speech segments; the extraction unit is further configured to: extract the voiceprint feature vector corresponding to each of the at least two speech segments; the processing unit is further configured to: perform clustering processing on the at least two speech segments according to the voiceprint feature vectors corresponding to the at least two speech segments respectively, to obtain multiple speech segment clusters, and take the speech segment cluster containing the most speech segments in the multiple speech segment clusters as the speaker speech segment cluster in the second speech data.

[0009] In some embodiments of this application, based on the foregoing scheme, the processing unit is further configured to: generate a speaker voiceprint feature vector corresponding to the second speech data according to the voiceprint feature vector corresponding to the speech segment in the speaker speech segment cluster.

[0010] In some embodiments of this application, based on the foregoing scheme, the segmentation unit is configured to: divide the second speech data according to a set speech segment length to obtain the at least two speech segments.

[0011] In some embodiments of this application, based on the foregoing scheme, the partitioning unit is further configured to: if it is necessary to extract the voiceprint feature vector corresponding to a specified speech segment, divide the specified speech segment into multiple speech signal frames; the extraction unit is further configured to: extract the audio features corresponding to each of the multiple speech signal frames; input the audio features corresponding to each speech signal frame into a network model to obtain the frame-level voiceprint feature vector corresponding to each speech signal frame output by the network model; and generate the voiceprint feature vector corresponding to the specified speech segment based on the frame-level voiceprint feature vectors corresponding to the multiple speech signal frames respectively.

[0012] In some embodiments of this application, based on the foregoing scheme, the designated speech segment includes speech segments in each speech segment set and speech segments obtained by dividing the second speech data.

[0013] In some embodiments of this application, based on the aforementioned scheme, the segmentation unit is configured to: divide the specified speech segment according to a set frame length, and perform correction processing on the multiple speech signals obtained by the segmentation through a set window function to obtain the multiple speech signal frames.

[0014] In some embodiments of this application, based on the foregoing scheme, the extraction unit is configured to: convert each of the plurality of speech signal frames into a corresponding frequency domain representation; convert the frequency domain representation corresponding to each speech signal frame into an energy output using a Mel filter bank; and generate an audio feature corresponding to each speech signal frame based on the energy value corresponding to each speech signal frame output by the Mel filter bank.

[0015] In some embodiments of this application, based on the aforementioned scheme, the extraction unit is configured to perform average pooling processing on the frame-level voiceprint feature vectors corresponding to the plurality of speech signal frames respectively to obtain the voiceprint feature vector corresponding to the specified speech segment.

[0016] In some embodiments of this application, based on the foregoing scheme, the network model includes a convolutional neural network, which contains residual network blocks, and the residual network blocks contain convolutional layers. During the convolution process, the convolutional layers pad data on the left side of the input data and refuse to pad data on the right side of the input data.

[0017] In some embodiments of this application, based on the foregoing scheme, the segmentation unit is configured to: if the specified speech segment is a speech segment recorded by the speaker, then during the process of receiving the specified speech segment recorded by the speaker, each received speech signal of a set length is determined as a segmented speech signal frame.

[0018] According to one aspect of the embodiments of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the voice data processing method as described in the above embodiments.

[0019] According to one aspect of the embodiments of this application, a computer device is provided, including: one or more processors; and a storage device for storing one or more computer programs, wherein when the one or more computer programs are executed by the one or more processors, the computer device enables the voice data processing method as described in the above embodiments.

[0020] According to one aspect of the embodiments of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads from the computer-readable storage medium and executes the computer program, causing the computer device to perform the voice data processing methods provided in the various alternative embodiments described above.

[0021] In some embodiments of this application, the first speech data can be segmented according to at least two segmentation methods to obtain multiple speech segment sets. Then, the voiceprint feature vectors corresponding to the two speech segments in each speech segment set are extracted, and the similarity between the voiceprint feature vectors corresponding to the two speech segments in each speech segment set is calculated as the similarity for each speech segment set. Subsequently, the lowest similarity is selected from the similarities corresponding to the multiple speech segment sets. When the lowest similarity is lower than or equal to a set threshold, the segmentation time point between the two speech segments in the speech segment set corresponding to the lowest similarity is determined as the speech data change time point of different speakers in the first speech data. Therefore, the technical solution of this application, by segmenting the first speech data according to at least two segmentation methods to obtain multiple speech segment sets, can ensure that speech changes at different time scales (different time scales correspond to different segmentation methods) can be captured, thereby improving the accuracy and comprehensiveness of subsequent detection of speech data change time points of different speakers. By extracting the voiceprint feature vectors corresponding to two speech segments in each speech segment set and calculating their similarity, the method of comparing speech segments in each set can more precisely capture subtle differences between speakers. Since the lowest similarity score usually reflects the true speaker variation, by selecting the lowest similarity score from the similarity scores corresponding to multiple speech segment sets, a speaker variation is considered to exist when the lowest similarity score is lower than or equal to a set threshold. This combination of the lowest similarity score and the set threshold effectively reduces false detections caused by noise or other interference factors, ensuring the reliability of the detection results.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0023] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0024] Figure 2 The diagram illustrates an application scenario where the technical solutions of the embodiments of this application can be applied.

[0025] Figure 3 A flowchart of a voice data processing method according to an embodiment of this application is shown;

[0026] Figure 4A A schematic diagram of speech data segmentation according to an embodiment of this application is shown;

[0027] Figure 4B A schematic diagram of speech data segmentation according to an embodiment of this application is shown;

[0028] Figure 5A A schematic diagram of the structure of a CNN model according to an embodiment of this application is shown;

[0029] Figure 5B A schematic diagram of the structure of a residual network block according to an embodiment of this application is shown;

[0030] Figure 6 A flowchart of a voice data processing method according to an embodiment of this application is shown;

[0031] Figure 7 A flowchart illustrating the extraction of frame-level acoustic signature feature vectors according to an embodiment of this application is shown;

[0032] Figure 8 A flowchart illustrating the speaker voiceprint feature vector recognition process according to an embodiment of this application is shown.

[0033] Figure 9 A block diagram of a voice data processing apparatus according to an embodiment of this application is shown;

[0034] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the computer device of the present application is shown. Detailed Implementation

[0035] Exemplary embodiments will now be described in a more comprehensive manner with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to these examples; rather, these embodiments are provided so that this application will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0036] Furthermore, the features, structures, or characteristics described in this application can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to provide a full understanding of the embodiments of this application. However, those skilled in the art will recognize that when implementing the technical solutions of this application, not all the detailed features in the embodiments may be used, one or more specific details may be omitted, or other methods, elements, devices, steps, etc., may be employed.

[0037] In this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0040] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0041] It is understood that this application may display a prompt interface or pop-up window before and during the collection of user-related data (such as voice data, voiceprint features, etc.). This prompt interface or pop-up window is used to inform the user that their relevant data is being collected. This ensures that the application only begins the steps for collecting user-related data after receiving confirmation from the user regarding the prompt interface or pop-up window; otherwise (i.e., without receiving confirmation from the user), the steps for collecting user-related data end, meaning no user-related data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of relevant user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0042] With the increasing prevalence of applications such as speech recognition, voiceprint recognition, meeting recording, and telephone customer service, efficient and accurate processing of voice data has become crucial. Especially in multi-speaker scenarios, automatically detecting the speaker's voiceprint feature vector and, based on this, automatically detecting speaker transition points (i.e., the switching points between different speakers) is fundamental to implementing many advanced functions, such as speaker separation, dialogue management, and content annotation. However, traditional speaker transition detection methods often suffer from insufficient accuracy, high false positive rates, and poor real-time performance, making it difficult to meet the needs of practical applications.

[0043] In order to overcome the problems existing in the related technologies, this application proposes a new speech data processing scheme, which can improve the reliability and accuracy of speaker change detection in speech data and effectively reduce false detections caused by noise or other interference factors.

[0044] Specifically, such as Figure 1 As shown, in a system architecture 100 applying the technical solution of this application embodiment, it may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include smartphones, tablets, laptops, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc. The server 130 may be a server providing various services; it may be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The network 120 may be a medium providing a communication connection between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.

[0045] Depending on the implementation requirements, the system architecture in this embodiment can have any number of terminal devices 110, network 120, and server 130. For example, server 130 can be a server group composed of multiple server devices.

[0046] In one embodiment of this application, a user can use terminal device 110 to record voice data and then send the recorded voice data to server 130 via network 120. For example, in an online meeting application scenario, terminal device 110 runs an online meeting client, and server 130 runs an online meeting server. The user can then express their views in an online meeting through the online meeting client on terminal device 110, and terminal device 110 can send the user's voice data to server 130 via the network. Simultaneously, other users can also express their views in online meetings through online meeting clients on other terminal devices 110 and send their voice data to server 130 via the network.

[0047] After receiving the voice data sent by the terminal device 110, the server 130 can process the voice data to identify the speaker change time points, and then perform processing such as speaker separation, dialogue management, and content annotation. Optionally, the server 130 can also return the identification results of the speaker change time points, or the results of other processing based on the speaker change time points, to the terminal device 110 via the network 120.

[0048] Specifically, after receiving the voice data sent by the terminal device 110, the server 130 can segment the voice data according to at least two segmentation methods to obtain at least two voice segment sets. Each segmentation method corresponds to one voice segment set, and each voice segment set contains two voice segments. Then, the server 130 can extract the voiceprint feature vectors corresponding to the two voice segments in each voice segment set, and calculate the similarity between the voiceprint feature vectors corresponding to the two voice segments in each voice segment set as the similarity for each voice segment set. The server then selects the lowest similarity from the similarity values ​​of multiple voice segment sets. If the selected lowest similarity is lower than or equal to a set threshold, the segmentation time point between the two voice segments in the voice segment set corresponding to the lowest similarity is used as the time point of voice data change for different speakers in the first voice data.

[0049] In a specific application scenario, such as Figure 2As shown, terminal device 110 is connected to a speaker device with a microphone (optionally, terminal device 110 may also have a speaker device installed on it without needing to connect to an external speaker device), which can collect the user's voice data. In one example, if a user needs to participate in an online meeting, an online meeting client can run on terminal device 110, and the user can record their speech during the meeting through the speaker device. Then, terminal device 110 can send the user's voice data to server 130. After receiving the voice data sent by terminal device 110, server 130 can process the voice data to identify speaker change time points, and then perform processing such as speaker separation, dialogue management, and content annotation. Then, server 130 can return the identification result of speaker change time points, or the result of other processing based on speaker change time points, to terminal device 110.

[0050] As can be seen, the technical solution of this application embodiment can improve the reliability and accuracy of speaker change detection in speech data and effectively reduce false detections caused by noise or other interference factors.

[0051] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0052] Figure 3 A flowchart illustrating a voice data processing method according to an embodiment of this application is shown. This voice data processing method can be executed by a computer device, which may be a server or other device. (Refer to...) Figure 3 As shown, the voice data processing method includes at least steps S310 to S340, which are described in detail below:

[0053] In step S310, the first speech data is segmented according to at least two segmentation methods to obtain at least two speech segment sets, wherein each speech segment set corresponds to one of the at least two segmentation methods, and each speech segment set contains two speech segments.

[0054] In some alternative embodiments, the first voice data is voice data that needs to identify the speaker's change points, such as voice data from an online meeting, voice data from a telephone customer service representative communicating with a user, voice recording data from a video, etc.

[0055] In some optional embodiments, segmenting the first speech data according to at least two segmentation methods means dividing the first speech data into two speech segments using at least two segmentation methods. The lengths of the speech segments obtained by segmenting using different methods are different; that is, one segmentation method corresponds to one set of speech segments. For example, refer to... Figure 4AAs shown, for the first segmentation method of the first speech data, it can be segmented at 1 / 5 of the first speech data to obtain two speech segments, which serve as the first speech segment set; for the second segmentation method of the first speech data, it can be segmented at 2 / 5 of the first speech data to obtain two speech segments, which serve as the second speech segment set; for the third segmentation method of the first speech data, it can be segmented at 3 / 5 of the first speech data to obtain two speech segments, which serve as the third speech segment set; for the fourth segmentation method of the first speech data, it can be segmented at 4 / 5 of the first speech data to obtain two speech segments, which serve as the fourth speech segment set. Thus, the first speech data is segmented according to four different segmentation methods, resulting in four speech segment sets.

[0056] In some optional embodiments, when the first speech data is segmented according to at least two segmentation methods to obtain at least two speech segment sets, the first speech data can be segmented according to a first segmentation duration to obtain a speech segment set corresponding to the first segmentation duration; then, the first segmentation duration is gradually increased according to a set step size to obtain at least one second segmentation duration, and the first speech data is segmented according to at least one second segmentation duration to obtain a speech segment set corresponding to at least one second segmentation duration. This embodiment's technical solution, by employing a method of gradually increasing the segmentation duration (i.e., starting from the first segmentation duration, gradually increasing according to a set step size to obtain at least one second segmentation duration, and segmenting the speech data according to these intervals), can capture changes in the speech data at different time scales. That is, a smaller segmentation duration can capture short-term, minute changes, while a larger segmentation duration helps to identify longer, global changes. This multi-scale analysis enables a more comprehensive understanding of the speech data, thereby improving the accuracy of speaker change point detection.

[0057] Specifically, such as Figure 4B As shown, assuming the first voice data is 2 seconds long, the first segment duration is 0.1 seconds, and the step size is set to 0.1 seconds, then... Figure 4B As shown, the speech segments can be divided at 0.1 seconds, 0.2 seconds, 0.3 seconds, ..., 1.7 seconds, 1.8 seconds, and 1.9 seconds respectively (i.e., as the segmentation time points). Accordingly, the resulting speech segment set is (0.1 seconds, 1.9 seconds), (0.2 seconds, 1.8 seconds), (0.3 seconds, 1.7 seconds), ... (1.7 seconds, 0.3 seconds), (1.8 seconds, 0.2 seconds), (1.9 seconds, 0.1 seconds).

[0058] In step S320, for each speech segment set in at least two speech segment sets, the voiceprint feature vectors corresponding to the two speech segments are extracted respectively.

[0059] In some optional embodiments, the voiceprint feature vector is a mathematical representation used to represent and identify the voiceprint features of an individual. It is a high-dimensional vector constructed by extracting the speaker's acoustic features (such as pitch, timbre, speech rate, etc.). When extracting the voiceprint feature vectors corresponding to the two speech segments contained in each speech segment set, the voiceprint feature vector extraction process can be performed separately for each speech segment.

[0060] In some optional embodiments, when extracting the voiceprint feature vector of a speech segment, the speech segment can be divided into multiple speech signal frames, and then the audio features corresponding to each of these multiple speech signal frames can be extracted. Then, the audio features corresponding to each speech signal frame can be input into the network model to obtain the frame-level voiceprint feature vector corresponding to each speech signal frame output by the network model. Then, the voiceprint feature vector corresponding to the speech segment can be generated based on the frame-level voiceprint feature vectors corresponding to the multiple speech signal frames.

[0061] In some optional embodiments, if it is necessary to extract voiceprint feature vectors from the speaker's recorded speech segments, then during the process of receiving the speaker's recorded speech segments, each received speech signal of a set length can be determined as a segmented speech signal frame. That is, the technical solution of this application embodiment can automatically and in real-time perform speech signal frame truncation processing during the speaker's recorded speech segments. For example, if the set length (i.e., the set frame length) is 20ms, then each time a 20ms speech signal is received, it can be used as a segmented speech signal frame.

[0062] In some optional embodiments, when dividing a speech segment into multiple speech signal frames, the speech segment can be divided according to a set frame length, and the resulting multiple speech signal segments can be corrected using a set window function to obtain multiple speech signal frames.

[0063] Optionally, the set frame length can be between 20ms and 40ms, specifically 20ms, 25ms, etc. When dividing a speech segment according to the set frame length, you can start from the beginning of the speech segment, extract the first frame according to the set frame length, then move to the next position according to the frame length, extract another frame, and repeat this process until all speech data has been processed. Optionally, if the last frame is less than a complete frame length, it can be padded or discarded as needed.

[0064] It should be noted that the correction processing of the segmented speech signal using a set window function mainly involves windowing the segmented speech signal, that is, applying a window function to the segmented speech signal to reduce spectral leakage caused by signal truncation. The window function is a mathematical function that modifies the sample values ​​within each frame, making them gradually approach zero at the beginning and end of the frame, smoothing the transition between frames, thereby improving the quality of subsequent spectral analysis and feature extraction. Optionally, the set window function can be a Hanning window, Hamming window, Blackman window, Gaussian window, etc.

[0065] In some optional embodiments, when extracting the audio features corresponding to each of the multiple speech signal frames, each of the multiple speech signal frames can be converted into a corresponding frequency domain representation. Then, the frequency domain representation corresponding to each speech signal frame can be converted into energy output through a Mel filter bank. Subsequently, the audio features corresponding to each speech signal frame can be generated based on the energy value corresponding to each speech signal frame output by the Mel filter bank.

[0066] Optionally, converting each speech signal frame into its corresponding frequency domain representation can be done by using Fast Fourier Transform (FFT) to obtain FFT frequency domain features. Then, a Mel filter bank is used to process the FFT frequency domain features to extract the energy of a specific frequency band. Finally, the energy output of each filter is calculated to obtain the Fbank feature, which is then used as the audio feature corresponding to the speech signal frame.

[0067] In some optional embodiments, the network model used to extract the frame-level voiceprint feature vector corresponding to each speech signal frame can be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), or a long short-term memory network (LSTM), etc.

[0068] Optionally, in embodiments of this application, a CNN can be used to extract frame-level voiceprint feature vectors. For example, referring to... Figure 5AAs shown, a CNN model structure in one example may include a ResNet block that is sequentially connected (i.e., the output of the previous network layer is the input of the next network layer), a frame-level embedding layer, a fully connected layer, a mean-pooling layer, and a fully connected layer.

[0069] Alternatively, the structure of the residual network (ResNet) block can be as follows: Figure 5B As shown, the network comprises multiple stacked convolutional layers that implement residual learning through the introduction of "shortcut connections." The working principle of the residual network block can be represented as: Output = F(x) + x, where F(x) represents the convolutional operations and transformations such as the ReLU activation function within the residual network block, and x represents the input feature map. This structure makes it easier for the network to learn the identity mapping during training, i.e., when the residual is 0, the output equals the input, thus avoiding the degradation problem of deep networks. For the frame-level voiceprint feature vector extraction in this application, the convolutional layers can pad data only on the left side of the input data during convolution processing, without padding on the right side. This maintains the temporal alignment of the CNN model's output, ensuring that the output at each time step depends only on the current and past time steps, not on future information. This allows the CNN model to respond immediately upon receiving new data, reducing processing latency.

[0070] In some optional embodiments, the process of generating a voiceprint feature vector corresponding to a speech segment based on the frame-level voiceprint feature vectors corresponding to multiple speech signal frames can be achieved by averaging the frame-level voiceprint feature vectors corresponding to the multiple speech signal frames to obtain the voiceprint feature vector corresponding to the speech segment. Optionally, when averaging the frame-level voiceprint feature vectors corresponding to the multiple speech signal frames, the frame-level voiceprint feature vectors corresponding to the multiple speech signal frames can be averaged using pooling to obtain the voiceprint feature vector corresponding to the speech segment.

[0071] Continue to refer to Figure 3 As shown, in step S330, for each speech segment set in at least two speech segment sets, the similarity between the voiceprint feature vectors corresponding to the two speech segments is calculated to obtain the similarity corresponding to each speech segment set, and the lowest similarity is selected from the similarity corresponding to the multiple speech segment sets.

[0072] In some optional embodiments, calculating the similarity between the voiceprint feature vectors corresponding to two speech segments is equivalent to calculating the similarity between the voiceprint feature vectors corresponding to two speech segments contained in each speech segment set. Calculating the similarity between the voiceprint feature vectors corresponding to two speech segments contained in each speech segment set can be achieved by calculating the cosine similarity between the voiceprint feature vectors corresponding to the two speech segments, where the cosine similarity is used to measure the magnitude of the difference between the two voiceprint feature vectors using the cosine of the angle between two vectors in the vector space.

[0073] It should be noted that if the similarity between the voiceprint feature vectors of two speech segments is small, it means that the two speech segments may come from different speakers. The minimum similarity usually reflects the actual speaker change better. Therefore, by selecting the minimum similarity from the similarity corresponding to multiple speech segment sets, and using the minimum similarity to determine whether there has been a change in the speaker, false detections caused by noise or other interference factors can be reduced, and the reliability of the detection results can be ensured.

[0074] In step S340, if the minimum similarity is lower than or equal to a set threshold, the segmentation time point between two speech segments in the speech segment set corresponding to the minimum similarity is determined, so as to determine the time point of speech data change of different speakers in the first speech data.

[0075] It should be noted that the first voice data may contain the voice data of multiple speakers. For example, in the scenario of an online meeting, different participants will speak in turn. Therefore, the voice data of the online meeting will contain the voice data of multiple participants. Thus, the time point of change of the voice data of different speakers in the first voice data refers to the time point when the voice data of different speakers changes, that is, the time point of speaker switching in the first voice data.

[0076] Optionally, the segmentation time point between two speech segments in the set of speech segments corresponding to the lowest similarity can be used as the time point of change of speech data of different speakers in the first speech data.

[0077] It should be noted that in this embodiment, by setting a reasonable threshold, and only when the minimum similarity is lower than or equal to the threshold, it is considered that there is a speaker change time point, which can further reduce the possibility of false detection of speaker change time points and effectively ensure the reliability of the detection results.

[0078] Figure 6 A flowchart illustrating a voice data processing method according to an embodiment of this application is shown. This voice data processing method can be executed by a computer device, which may be a server or other device. (Refer to...) Figure 6 As shown, the voice data processing method includes at least steps S610 to S640, which are described in detail below:

[0079] In step S610, the second speech data is divided into at least two speech segments.

[0080] In some optional embodiments, the second speech data can be divided according to a set speech segment length to obtain at least two speech segments.

[0081] In step S620, the voiceprint feature vector corresponding to each of the at least two speech segments obtained from the division of the second speech data is extracted.

[0082] In some optional embodiments, when extracting the speaker feature vector of a speech segment, the speech segment can be divided into multiple speech signal frames. Then, the audio features corresponding to each of these multiple speech signal frames are extracted. These audio features can then be input into a network model to obtain the frame-level speaker feature vector output by the network model for each speech signal frame. Finally, based on the frame-level speaker feature vectors corresponding to the multiple speech signal frames, the speaker feature vector corresponding to the speech segment can be generated. Specific processing details can be found in the technical solutions of the foregoing embodiments and will not be repeated here.

[0083] In step S630, at least two speech segments are clustered according to their respective voiceprint feature vectors to obtain multiple speech segment clusters.

[0084] In some optional embodiments, at least two speech segments can be clustered using a bottom-up hierarchical clustering (AHC) method. Specifically, at the start of clustering, each speech segment is treated as a separate cluster. To determine which speech segments should be merged into the same cluster, a distance metric needs to be defined; commonly used distance metrics include Euclidean distance and cosine similarity. Optionally, cosine similarity can be used to measure the similarity between two voiceprint feature vectors. In each iteration, the two closest (i.e., most similar) clusters are found and merged into a new cluster. This process is repeated until a stopping condition is met, such as reaching a predetermined number of clusters or a minimum similarity threshold. The entire clustering process can be visualized as a tree diagram, where each leaf node represents an original speech segment, and each internal node represents a merge operation. As the hierarchy increases, more and more speech segments are grouped into the same cluster, hence the name bottom-up hierarchical clustering.

[0085] In step S640, the speech segment cluster containing the most speech segments among multiple speech segment clusters is taken as the speaker speech segment cluster in the second speech data.

[0086] It should be noted that, generally speaking, in a piece of audio data, the number of speech segments of the target speaker is usually greater than that of other interfering speakers. Therefore, the speech segment cluster containing the most speech segments among multiple speech segment clusters can be taken as the speaker's speech segment cluster in the second audio data. That is, the technical solution of this application embodiment can remove interfering speech segments from audio data, thereby accurately extracting the speech segments of the target speaker.

[0087] In some optional embodiments, after determining the speaker speech segment clusters in the second speech data, a speaker voiceprint feature vector corresponding to the second speech data can be generated based on the voiceprint feature vectors corresponding to the speech segments in the speaker speech segment clusters. For example, the voiceprint feature vectors corresponding to the speech segments in the speaker speech segment clusters can be averaged to obtain the speaker voiceprint feature vector corresponding to the second speech data. Optionally, when averaging the voiceprint feature vectors corresponding to the speech segments in the speaker speech segment clusters, the voiceprint feature vectors corresponding to the speech segments in the speaker speech segment clusters can be average pooled to obtain the speaker voiceprint feature vector corresponding to the second speech data.

[0088] In some optional embodiments, after obtaining the speaker's voiceprint feature vector corresponding to the second voice data, corresponding business processing can be performed based on the speaker's voiceprint feature vector. For example, the speaker's identity can be identified based on the speaker's voiceprint feature vector, or registration processing can be performed based on the speaker's voiceprint feature vector, that is, the speaker's voiceprint feature vector is associated with the speaker's identity and stored, so as to facilitate subsequent identification of the speaker's identity based on the speaker's voiceprint feature vector, etc.

[0089] The following combination Figure 7 and Figure 8 Taking the extraction of voiceprint feature vectors through convolutional neural networks as an example, the implementation details of the technical solution of this application embodiment are described in detail:

[0090] In the extraction of voiceprint feature vectors, LSTM models are typically used. LSTM selectively remembers or forgets information through special gating mechanisms (input gate, forget gate, output gate). This mechanism means that LSTM is influenced by all previous frames when processing each frame. For voiceprint recognition tasks, this characteristic may introduce unnecessary contextual information, leading to inaccurate capture of speaker features in the current frame, thus affecting the discriminative power of the voiceprint vector. Furthermore, the relatively complex structure of LSTM networks with a large number of parameters makes model training more difficult and prone to overfitting, especially in small sample sizes. LSTM may fail to generalize well to unseen data, further limiting its ability to provide high discriminative power among different speakers.

[0091] Based on this, the technical solution of this application proposes to use a convolutional neural network to obtain frame-level voiceprint feature vectors in a streaming manner, and then use the extracted frame-level voiceprint vectors for tasks such as non-perceptible voiceprint registration and speaker change point detection.

[0092] Specifically, such as Figure 7 As shown, for an input frame of speech signal, for example, in an audio communication scenario (such as online meetings, voice calls, etc.), every 20ms of audio stream input is considered as one frame of speech signal. Then, audio features are extracted, and the extracted audio features are input into a convolutional neural network, thereby obtaining the frame-level voiceprint feature vector output by the convolutional neural network.

[0093] Optionally, the audio feature extraction process can be as follows: first, the input speech time-domain signal is segmented and windowed; then, FFT frequency domain features are extracted from the frame signal; then, the FFT frequency domain features are processed using a Mel filter bank to extract the energy of a specific frequency band; then, the energy output of each filter is calculated, and the resulting Fbank features are used as the audio features corresponding to the speech signal.

[0094] Because of differences in size, shape, and function among human vocal organs—including the vocal cords, soft palate, tongue, teeth, and lips; and the pharyngeal cavity, oral cavity, and nasal cavity—even minute differences in these organs can alter the airflow during speech, resulting in variations in timbre and tone. This leads to each person having their own unique speaking style, and generally, a person's speaking characteristics remain relatively stable after adulthood. Frame-level voiceprint feature vector acquisition involves inputting the audio features of a single frame of speech signal into a convolutional neural network to obtain the speaker's voiceprint feature vector. This voiceprint feature vector can be a vector of several hundred dimensions, implicitly containing the speaker's unique speaking characteristics. Different speakers have their own unique voiceprint feature vectors, making them easily distinguishable from one another.

[0095] In some optional embodiments, the extraction of frame-level voiceprint feature vectors can be implemented using a convolutional neural network. The input of the convolutional neural network can be the audio features of a frame of speech signal, and the output is a frame-level voiceprint feature vector. Optionally, the structure of the convolutional neural network can be as follows: Figure 5A As shown, it includes residual network blocks, frame-level embedding layers, average pooling layers, and fully connected layers. The structure of the residual network blocks can be as follows: Figure 5B As shown, in order to achieve streaming extraction of frame-level voiceprint feature vectors, Figure 5B The convolutional layer shown can be padded only on the left side and not on the right side. This makes the convolutional layer causal, which can maintain the temporal alignment of the output and ensure that the convolutional operation is always based on past and current data. This enables real-time extraction of streaming voiceprint vectors, and whenever a new audio signal frame arrives, the system can immediately process it and generate the corresponding voiceprint feature vector without waiting for subsequent audio signals.

[0096] Optionally, the stride of the convolutional layer in the time dimension can be set to 1. Here, stride refers to the step size by which the convolutional kernel moves across the input. When the stride is set to 1, it means that the convolutional kernel moves forward only one time step at a time. This ensures that the temporal resolution of the output sequence of the convolutional layer is consistent with the input sequence; that is, there is a corresponding output for each time step. This is crucial for frame-level speaker feature vector extraction, because each frame of audio signal input can generate a corresponding speaker feature vector.

[0097] In applications involving the acquisition of voiceprint feature vectors in audio communication, when a speaker finishes speaking for 30 seconds, related technologies may require using 15 seconds of the audio signal to extract the voiceprint feature vector. However, the technical solution of this application embodiment can achieve frame-level voiceprint feature vector extraction. In other words, when a person is speaking, this application embodiment can extract frame-level voiceprint feature vectors in real time. Thus, when the speaker finishes speaking for 30 seconds, the technical solution of this application embodiment only needs to process the frame-level voiceprint feature vector (such as average pooling) to obtain the speaker's segment-level voiceprint feature vector, achieving zero-latency, real-time voiceprint feature vector extraction.

[0098] In some optional embodiments, the goal of the convolutional neural network model in this application during training is to achieve speaker classification as accurately as possible. Therefore, the convolutional neural network model can use the cross-entropy loss function as the model's loss function during training. That is, the training objective of the convolutional neural network model is:

[0099]

[0100] Among them, t i Indicates the actual speaker's label; o i is the probability of the i-th class label output by the convolutional neural network model after processing by the normalized exponential function (softmax); n represents the number of labels; the training objective of the convolutional neural network model is to make the network output speaker classification results as consistent as possible with the real answers.

[0101] The performance evaluation of the frame-level voiceprint feature vectors extracted by convolution processing in this embodiment is shown in Table 1:

[0102] system EER LSTM-based frame-level speaker feature vector extraction system 3.52％ Frame-level voiceprint feature vector extraction system based on convolution 0.58％

[0103] Table 1

[0104] Referring to Table 1, EER (Equal Error Rate) is a commonly used performance evaluation metric in voiceprint recognition, used to measure the accuracy of the system. EER represents the error rate when the False Positive Rate (FPR) and False Negative Rate (FNR) are equal. Experimental results show that the EER of the convolution-based frame-level voiceprint feature vector extraction system is significantly lower than that of the LSTM-based system, indicating that the convolution-based system has a significant advantage.

[0105] In some optional embodiments, the frame-level voiceprint feature vectors extracted in this application can be applied in non-perceptible voiceprint registration scenarios. For example, such as Figure 8 As shown, after obtaining the speaker's voice data, the frame-level voiceprint feature vector can be extracted using the technical solution of the aforementioned embodiment. Then, the segment-level voiceprint feature vector is generated based on the frame-level voiceprint feature vector. Subsequently, speaker clustering is performed based on the segment-level voiceprint feature vector to obtain the speaker's voiceprint feature vector.

[0106] Alternatively, the segment-level voiceprint feature vector can be obtained by average pooling of multiple extracted frame-level voiceprint feature vectors. This process has low computational cost and will not increase processing time.

[0107] Optionally, speaker clustering is mainly because among the N segment-level voiceprint feature vectors, there may be voiceprint feature vectors of interfering persons due to the presence of other interfering persons in the communication. Therefore, the speech segments of the target speaker can be clustered by the bottom-up hierarchical clustering method (AHC), and the speech segments of interfering persons can be removed. Then, the voiceprint feature vector of the target speaker can be calculated.

[0108] For example, the speech segment cluster containing the most speech segments among multiple speech segment clusters obtained by clustering can be taken as the speech segment cluster of the target speaker, and then the voiceprint feature vector of the target speaker can be extracted from the speech segment cluster.

[0109] Optionally, after calculating the voiceprint feature vector of the target speaker, registration can be performed based on the speaker's voiceprint feature vector, that is, the speaker's voiceprint feature vector is associated with the speaker's identity and stored, so that the speaker's identity can be identified in the future based on the speaker's voiceprint feature vector. This achieves a seamless voiceprint registration process.

[0110] In some optional embodiments, the frame-level speaker feature vectors extracted in this application can be applied to speaker change point detection scenarios. The role of speaker change point detection is to detect whether there are speaker changes in a speech segment. If so, the time points of speaker changes are found, which can then be used for processing such as speaker separation, dialogue management, and content annotation.

[0111] Specifically, taking a 2-second audio segment as an example, in the embodiments of this application, the 2-second audio segment can be divided into two segments with a step size of 0.1 seconds. Then the lengths of the first segment and the second segment are (0.1 seconds, 1.9 seconds), (0.2 seconds, 1.8 seconds), (0.3 seconds, 1.7 seconds), ... (1.7 seconds, 0.3 seconds), (1.8 seconds, 0.2 seconds), (1.9 seconds, 0.1 seconds).

[0112] Based on the obtained frame-level voiceprint feature vector, the voiceprint feature vectors of the first segment and the second segment can be calculated. For example, the frame-level voiceprint feature vectors contained in the first segment can be averaged to obtain the voiceprint feature vector of the first segment, and the frame-level voiceprint feature vectors contained in the second segment can be averaged to obtain the voiceprint feature vector of the second segment.

[0113] Then, the cosine similarity between the first and second voiceprint vectors can be calculated. Since there are multiple segmentation methods, multiple cosine similarities can be calculated. Based on this, the minimum similarity value and the corresponding segmentation time point T can be obtained. If the minimum similarity value is less than a set threshold, then it is considered that a speaker change point has been detected, and the speaker change time point is T.

[0114] In summary, the technical solution of this application embodiment achieves high-precision frame-level voiceprint feature vector extraction, which can then be applied to application scenarios such as imperceptible voiceprint registration and accurate speaker change point detection. It not only improves the accuracy of voiceprint feature vector extraction, but also provides a more efficient and reliable solution for speech data processing, and has broad application prospects and practical value.

[0115] The following describes an apparatus embodiment of this application, which can be used to execute the voice data processing method in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the voice data processing method described above.

[0116] Figure 9 A block diagram of a voice data processing apparatus according to an embodiment of the present application is shown. The voice data processing apparatus can be applied to a computer device, which may be a server or other devices.

[0117] Reference Figure 9 As shown, a voice data processing apparatus 900 according to an embodiment of this application includes: a segmentation unit 902, an extraction unit 904, a calculation unit 906, and a processing unit 908.

[0118] The segmentation unit 902 is configured to segment the first speech data according to at least two segmentation methods to obtain at least two speech segment sets, wherein each speech segment set corresponds to one of the at least two segmentation methods and each speech segment set contains two speech segments; the extraction unit 904 is configured to extract the voiceprint feature vectors corresponding to the two speech segments for each speech segment set in the at least two speech segment sets; the calculation unit 906 is configured to calculate the similarity between the voiceprint feature vectors corresponding to the two speech segments for each speech segment set in the at least two speech segment sets, obtain the similarity corresponding to each speech segment set, and select the lowest similarity from the similarity corresponding to the multiple speech segment sets; the processing unit 908 is configured to determine the segmentation time point between the two speech segments in the speech segment set corresponding to the lowest similarity if the lowest similarity is lower than or equal to a set threshold, so as to determine the speaker change time point in the first speech data.

[0119] In some embodiments of this application, based on the foregoing scheme, the segmentation unit 902 is configured to: segment the first speech data according to a first segmentation duration to obtain a speech segment set corresponding to the first segmentation duration; gradually increase the first segmentation duration according to a set step size to obtain at least one second segmentation duration, and segment the first speech data according to the at least one second segmentation duration to obtain a speech segment set corresponding to the at least one second segmentation duration.

[0120] In some embodiments of this application, based on the foregoing scheme, the segmentation unit 902 is further configured to: divide the second speech data into at least two speech segments; the extraction unit 904 is further configured to: extract the voiceprint feature vector corresponding to each of the at least two speech segments; and the processing unit 908 is further configured to: perform clustering processing on the at least two speech segments according to the voiceprint feature vectors corresponding to the at least two speech segments respectively, to obtain multiple speech segment clusters, and take the speech segment cluster containing the most speech segments in the multiple speech segment clusters as the speaker speech segment cluster in the second speech data.

[0121] In some embodiments of this application, based on the foregoing scheme, the processing unit 908 is further configured to: generate a speaker voiceprint feature vector corresponding to the second speech data according to the voiceprint feature vector corresponding to the speech segment in the speaker speech segment cluster.

[0122] In some embodiments of this application, based on the aforementioned scheme, the segmentation unit 902 is configured to: divide the second speech data according to a set speech segment length to obtain the at least two speech segments.

[0123] In some embodiments of this application, based on the foregoing scheme, the partitioning unit 902 is further configured to: if it is necessary to extract the voiceprint feature vector corresponding to a specified speech segment, divide the specified speech segment into multiple speech signal frames; the extraction unit 904 is further configured to: extract the audio features corresponding to each of the multiple speech signal frames; input the audio features corresponding to each speech signal frame into a network model to obtain the frame-level voiceprint feature vector corresponding to each speech signal frame output by the network model; and generate the voiceprint feature vector corresponding to the specified speech segment based on the frame-level voiceprint feature vectors corresponding to the multiple speech signal frames respectively.

[0124] In some embodiments of this application, based on the aforementioned scheme, the segmentation unit 902 is configured to: segment the specified speech segment according to a set frame length, and perform correction processing on the segmented speech signals through a set window function to obtain the multiple speech signal frames.

[0125] In some embodiments of this application, based on the foregoing scheme, the extraction unit 904 is configured to: convert each of the plurality of speech signal frames into a corresponding frequency domain representation; convert the frequency domain representation corresponding to each speech signal frame into an energy output through a Mel filter bank; and generate an audio feature corresponding to each speech signal frame based on the energy value corresponding to each speech signal frame output by the Mel filter bank.

[0126] In some embodiments of this application, based on the aforementioned scheme, the extraction unit 904 is configured to perform average pooling processing on the frame-level voiceprint feature vectors corresponding to the plurality of speech signal frames respectively to obtain the voiceprint feature vector corresponding to the specified speech segment.

[0127] In some embodiments of this application, based on the foregoing scheme, the network model includes a convolutional neural network, which contains residual network blocks, and the residual network blocks contain convolutional layers. During the convolution process, the convolutional layers pad data on the left side of the input data and refuse to pad data on the right side of the input data.

[0128] In some embodiments of this application, based on the foregoing scheme, the segmentation unit 902 is configured to: if the specified speech segment is a speech segment recorded by the speaker, then during the process of receiving the specified speech segment recorded by the speaker, the speech signal of a set length received each time is determined as a segmented speech signal frame.

[0129] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the computer device of the present application is shown. The computer device may be the server in the foregoing embodiments.

[0130] It should be noted that, Figure 10 The computer system 1000 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0131] like Figure 10 As shown, the computer system 1000 may include a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 1002 or a program loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0132] The following components can be connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0133] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0134] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a computer program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and a computer program.

[0136] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0137] In another aspect, this application also provides a computer-readable medium, which may be included in the computer device described in the above embodiments; or it may exist independently and not assembled into the computer device. The computer-readable medium carries one or more computer programs, which, when executed by the computer device, cause the computer device to perform the methods described in the above embodiments.

[0138] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0139] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computer device to execute the method according to the embodiments of this application. For example, it can execute... Figure 3 or Figure 6 The speech data processing method shown.

[0140] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0141] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A voice data processing method, characterized in that, include: The first speech data is segmented according to at least two segmentation methods to obtain at least two speech segment sets, wherein each speech segment set corresponds to one of the at least two segmentation methods, and each speech segment set contains two speech segments; For each of the at least two speech segment sets, extract the voiceprint feature vectors corresponding to the two speech segments respectively; For each speech segment set in the at least two speech segment sets, the similarity between the voiceprint feature vectors corresponding to the two speech segments is calculated to obtain the similarity corresponding to each speech segment set, and the lowest similarity is selected from the similarity corresponding to the multiple speech segment sets. If the minimum similarity is lower than or equal to a set threshold, then the segmentation time point between the two speech segments in the speech segment set corresponding to the minimum similarity is determined, so as to determine the time point of speech data change of different speakers in the first speech data.

2. The voice data processing method according to claim 1, characterized in that, The first speech data is segmented using at least two different segmentation methods to obtain at least two sets of speech segments, including: The first speech data is segmented according to the first segmentation duration to obtain a set of speech segments corresponding to the first segmentation duration; The first segmentation duration is gradually increased according to a set step size to obtain at least one second segmentation duration. The first speech data is then segmented according to the at least one second segmentation duration to obtain speech segment sets corresponding to the at least one second segmentation duration.

3. The voice data processing method according to claim 1, characterized in that, The voice data processing method further includes: Divide the second speech data into at least two speech segments; Extract the voiceprint feature vector corresponding to each of the at least two speech segments; Based on the voiceprint feature vectors corresponding to the at least two speech segments respectively, the at least two speech segments are clustered to obtain multiple speech segment clusters; The speech segment cluster containing the most speech segments among the multiple speech segment clusters is taken as the speaker speech segment cluster in the second speech data.

4. The voice data processing method according to claim 3, characterized in that, The voice data processing method further includes: Based on the voiceprint feature vectors corresponding to the voice segments in the speaker's voice segment cluster, the speaker's voiceprint feature vector corresponding to the second voice data is generated.

5. The voice data processing method according to claim 3, characterized in that, The second speech data is divided into at least two speech segments, including: The second voice data is divided according to the set voice segment length to obtain the at least two voice segments.

6. The voice data processing method according to any one of claims 1 to 5, characterized in that, The voice data processing method further includes: If it is necessary to extract the voiceprint feature vector corresponding to a specified speech segment, the specified speech segment is divided into multiple speech signal frames; Extract the audio features corresponding to each of the multiple speech signal frames; The audio features corresponding to each speech signal frame are input into the network model to obtain the frame-level speaker feature vector corresponding to each speech signal frame output by the network model. Based on the frame-level voiceprint feature vectors corresponding to the multiple speech signal frames, a voiceprint feature vector corresponding to the specified speech segment is generated.

7. The voice data processing method according to claim 6, characterized in that, Dividing the specified speech segment into multiple speech signal frames includes: The specified speech segment is divided according to the set frame length, and the resulting multiple speech signals are corrected using a set window function to obtain the multiple speech signal frames.

8. The voice data processing method according to claim 6, characterized in that, Extracting the audio features corresponding to each of the multiple speech signal frames, including: Each of the plurality of speech signal frames is converted into a corresponding frequency domain representation; The frequency domain representation corresponding to each speech signal frame is converted into energy output using a Mel filter bank; Based on the energy value corresponding to each speech signal frame output by the Mel filter bank, audio features corresponding to each speech signal frame are generated.

9. The voice data processing method according to claim 6, characterized in that, Based on the frame-level speaker feature vectors corresponding to the multiple speech signal frames, a speaker feature vector corresponding to the specified speech segment is generated, including: The frame-level voiceprint feature vectors corresponding to the multiple speech signal frames are subjected to average pooling to obtain the voiceprint feature vector corresponding to the specified speech segment.

10. The voice data processing method according to claim 6, characterized in that, The network model includes a convolutional neural network, which contains residual network blocks, and the residual network blocks contain convolutional layers. During the convolution process, the convolutional layers pad data on the left side of the input data and refuse to pad data on the right side of the input data.

11. The voice data processing method according to claim 6, characterized in that, Dividing the specified speech segment into multiple speech signal frames includes: If the specified voice segment is a voice segment recorded by the speaker, then during the process of receiving the specified voice segment recorded by the speaker, each received voice signal of a set length is determined as a segmented voice signal frame.

12. A voice data processing device, characterized in that, include: The segmentation unit is configured to segment the first speech data according to at least two segmentation methods to obtain at least two speech segment sets, wherein each speech segment set corresponds to one of the at least two segmentation methods, and each speech segment set contains two speech segments; The extraction unit is configured to extract the voiceprint feature vectors corresponding to the two speech segments for each speech segment set in the at least two speech segment sets; The calculation unit is configured to calculate the similarity between the voiceprint feature vectors corresponding to the two speech segments for each speech segment set in the at least two speech segment sets, obtain the similarity corresponding to each speech segment set, and select the lowest similarity from the similarity corresponding to the multiple speech segment sets respectively. The processing unit is configured to determine the segmentation time point between two speech segments in the speech segment set corresponding to the lowest similarity if the lowest similarity is lower than or equal to a set threshold, so as to determine the time point of speech data change of different speakers in the first speech data.

13. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the voice data processing method according to any one of claims 1 to 11.

14. A computer device, characterized in that, include: One or more processors; A memory for storing one or more computer programs, which, when executed by the one or more processors, cause the computer device to implement the voice data processing method according to any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, wherein a processor of a computer device reads from the computer-readable storage medium and executes the computer program, causing the computer device to perform the voice data processing method according to any one of claims 1 to 11.

Images