A voiceprint identification method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a voiceprint identification method and device, electronic equipment and storage medium. By performing phoneme-level segmentation and time alignment on the to-be-identified voice segment and the reference voice segment, respectively generating spectrograms and adjusting image parameters, and then performing image superposition, a visualized formant difference graph is formed. The difference graph highlights the formant difference between the voice segments through color, transparency and other parameter settings, and intuitively displays the frequency band, intensity and trend difference. If there is a formant trend difference, the identification result is determined in combination with the frequency band and intensity difference values, thereby improving the accuracy of voiceprint recognition.

Description

Technical Field

[0001] This disclosure relates to the field of voiceprint recognition technology, and more specifically, to a voiceprint identification method, apparatus, electronic device, and storage medium. Background Technology

[0002] Speech is one of the important biometrics for individual identification and behavioral analysis. In traditional voiceprint recognition systems, numerical features such as Mel-frequency cepstral coefficients (MFCC), spectrogram convolutional features, or x-vector acoustic embeddings are typically used to construct speech models. However, these features exist in the form of high-dimensional vectors, lacking intuitiveness and hindering manual visual analysis and verification, especially in scenarios with high interpretability requirements such as forensic identification and medical speech analysis.

[0003] In voiceprint identification, formants are key biometric carriers: Stability: The formant patterns of adults are stable over the long term (annual change rate <5%) and are not easily changed by short-term physiological or environmental changes; The frequency and shape (such as curve trend and distribution range) of formants vary significantly among different people, and it is difficult to replicate the fine formant structure even when imitating other people's pronunciation; Compared with speech content, formant features are more robust to background noise and speech rate changes and can penetrate some environmental interference for identity recognition.

[0004] Existing methods for identifying the same person using formant techniques mainly focus on comparing vowel formant frequencies. Vowel formant extraction primarily relies on Linear Predictive Coding (LPC). LPC is based on the assumption of an all-pole model for speech generation, assuming that a speech signal can be generated by an all-pole filter. However, in reality, speech generation is very complex, and actual speech signals do not perfectly conform to the all-pole model. In many cases, speech signals may contain zero-point components, i.e., anti-resonance. The LPC model cannot accurately represent this zero-point information, leading to inaccurate modeling of the speech signal. The information filtered out by LPC may reflect an individual's pronunciation characteristics (such as a heavy nasal tone). Summary of the Invention

[0005] This disclosure provides at least one voiceprint identification method, device, electronic device, and storage medium. It can perform detection based on a graphic overlay method, which overlays the formants of one or more audio segments to be compared. By setting different parameters such as color and transparency, the morphological differences between segments can be highlighted, and the differences in formant morphology between different audio segments can be seen intuitively, thereby improving the accuracy of voiceprint identification.

[0006] This disclosure provides a voiceprint identification method, including:

[0007] Obtain the speech segment to be identified and the reference speech segment, segment the speech segment to be identified into multiple phoneme segments to be identified, and segment the reference speech segment into multiple reference phoneme segments;

[0008] Align the phoneme segment to be identified with the reference phoneme segment in the time dimension, and determine the spectrogram of the segment to be identified and the reference spectrogram after phoneme alignment respectively.

[0009] The image parameters corresponding to the spectrogram to be identified and the reference spectrogram are adjusted respectively, and the spectrogram to be identified after parameter adjustment is superimposed on the reference spectrogram to generate a visual formant difference map.

[0010] The differences in frequency band energy distribution of the resonant peaks, the differences in energy intensity of the resonant peaks, and the presence of differences in the trend of the resonant peaks are detected in the visualized resonant peak difference map. If the differences in the trend of the resonant peaks exist, the voiceprint identification result is determined based on the differences in frequency band energy distribution of the resonant peaks and the differences in energy intensity of the resonant peaks.

[0011] In one optional implementation, the image parameters corresponding to the spectrogram to be identified and the reference spectrogram are adjusted respectively, and the parameter-adjusted spectrogram to be identified and the reference spectrogram are superimposed to generate a visual formant difference map, specifically including:

[0012] Align the main content of the spectrogram to be identified with that of the reference spectrogram;

[0013] Adjust the opacity and feather radius of the spectrogram to be identified and the reference spectrogram, and create a gradient transition or semi-transparent background to adjust the layer overlay effect between the spectrogram to be identified and the reference spectrogram;

[0014] The saturation, brightness, hue, and mixed color bands of the spectrogram to be identified and the reference spectrogram are unified to generate the visualized formant difference map.

[0015] In one optional implementation, detecting the frequency band difference values ​​of the resonance peak energy distribution, the difference values ​​of the resonance peak energy intensity, and the existence of resonance peak trend differences displayed in the visualized resonance peak difference map specifically includes:

[0016] In the visualized formant difference diagram, the difference in the vertical axis frequency range between the spectrogram to be identified and the reference spectrogram is determined, and the difference in the vertical axis frequency range is determined as the difference in the frequency band of the formant energy distribution.

[0017] In the visualized formant difference map, the pixel area difference between the spectrogram to be identified and the reference spectrogram is determined, and the pixel area difference is determined as the formant energy intensity difference value.

[0018] Determine the overall trend direction between the spectrogram to be identified and the reference spectrogram in the visualized formant difference diagram;

[0019] If there is a difference in the overall trend direction corresponding to the dynamic trajectory between the spectrogram to be identified and the reference spectrogram, then it is determined that there is a difference in the trend of the formant peaks.

[0020] In an optional implementation, before determining the voiceprint identification result based on the frequency band difference value of the resonant peak energy distribution and the frequency band difference value of the resonant peak energy intensity, the method further includes:

[0021] The frequency band difference value of the resonance peak energy distribution is compared with the preset first threshold range to determine the first difference level to which the frequency band difference value of the resonance peak energy distribution belongs. The first difference level includes a large difference level, a medium difference level, and a small difference level in the distribution frequency band.

[0022] The energy intensity difference value of the resonance peak is compared with the preset second threshold range to determine the second difference level to which the energy intensity difference value of the resonance peak belongs. The second difference level includes a large energy intensity difference level, a medium energy intensity difference level, and a small energy intensity difference level.

[0023] In one optional implementation, the voiceprint identification result is determined based on the difference between the frequency band of the resonant peak energy distribution and the difference between the resonant peak energy intensity, specifically including:

[0024] When the large difference level of the distribution frequency band belongs to the large difference level of the distribution frequency band, and the energy intensity difference value of the formant peak belongs to the large difference level of the energy intensity, it is determined that the speech segment to be identified and the reference speech segment belong to the same person.

[0025] When the large difference level of the distribution frequency band belongs to the medium difference level of the distribution frequency band or the small difference level of the energy intensity, and the energy intensity difference value of the resonance peak belongs to the medium difference level of the energy intensity or the small difference level of the energy intensity, it is determined that the speech segment to be identified and the reference speech segment do not belong to the same person.

[0026] This disclosure also provides a voiceprint identification device, including:

[0027] The phoneme segmentation module is used to acquire the speech segment to be identified and the reference speech segment, segment the speech segment to be identified into multiple phoneme segments to be identified, and segment the reference speech segment into multiple reference phoneme segments.

[0028] The phoneme alignment module is used to align the phoneme segment to be identified with the reference phoneme segment in the time dimension, and to determine the spectrogram to be identified and the reference spectrogram after phoneme alignment, respectively.

[0029] The image adjustment and overlay module is used to adjust the image parameters corresponding to the spectrogram to be identified and the reference spectrogram respectively, and overlay the spectrogram to be identified after parameter adjustment with the reference spectrogram to generate a visual formant difference map.

[0030] The difference detection module is used to detect the frequency band difference value of the energy distribution of the resonance peak, the energy intensity difference value of the resonance peak, and whether there is a difference in the trend of the resonance peak displayed in the visualized resonance peak difference map. If the difference in the trend of the resonance peak exists, the voiceprint identification result is determined based on the frequency band difference value of the energy distribution of the resonance peak and the energy intensity difference value of the resonance peak.

[0031] In one optional implementation, the image adjustment and overlay module is specifically used for:

[0032] Align the main content of the spectrogram to be identified with that of the reference spectrogram;

[0033] Adjust the opacity and feather radius of the spectrogram to be identified and the reference spectrogram, and create a gradient transition or semi-transparent background to adjust the layer overlay effect between the spectrogram to be identified and the reference spectrogram;

[0034] The saturation, brightness, hue, and mixed color bands of the spectrogram to be identified and the reference spectrogram are unified to generate the visualized formant difference map.

[0035] In one optional implementation, the difference detection module is specifically used for:

[0036] In the visualized formant difference diagram, the difference in the vertical axis frequency range between the spectrogram to be identified and the reference spectrogram is determined, and the difference in the vertical axis frequency range is determined as the difference in the frequency band of the formant energy distribution.

[0037] In the visualized formant difference map, the pixel area difference between the spectrogram to be identified and the reference spectrogram is determined, and the pixel area difference is determined as the formant energy intensity difference value.

[0038] Determine the overall trend direction between the spectrogram to be identified and the reference spectrogram in the visualized formant difference diagram;

[0039] If there is a difference in the overall trend direction corresponding to the dynamic trajectory between the spectrogram to be identified and the reference spectrogram, then it is determined that there is a difference in the trend of the formant peaks.

[0040] This disclosure also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the above-described voiceprint identification method or any possible implementation of the above-described voiceprint identification method.

[0041] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the above-described voiceprint identification method or any possible implementation thereof.

[0042] This disclosure also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-described voiceprint identification method, or the steps in any possible implementation of the above-described voiceprint identification method.

[0043] This disclosure provides a voiceprint identification method, apparatus, electronic device, and storage medium. The method involves acquiring a speech segment to be identified and a reference speech segment; segmenting the speech segment to be identified into multiple phoneme segments to be identified; segmenting the reference speech segment into multiple reference phoneme segments; aligning the phoneme segments to be identified and the reference phoneme segments in the time dimension; determining the aligned spectrograms to be identified and the reference spectrograms; adjusting the image parameters corresponding to the spectrograms to be identified and the reference spectrograms; and overlaying the adjusted spectrograms to be identified and the reference spectrograms to generate a visualized formant difference map; detecting the frequency band difference value of the formant energy distribution, the difference value of the formant energy intensity, and the existence of formant trend differences displayed in the visualized formant difference map; and determining the voiceprint identification result based on the frequency band difference value of the formant energy distribution and the difference value of the formant energy intensity if the formant trend differences exist. Detection can be performed based on graphic overlay. One or more audio segments to be compared are superimposed, and by setting different parameters such as color and transparency, the morphological differences between segments can be highlighted. The differences in the formant morphology of different audio segments can be seen intuitively, thus improving the accuracy of voiceprint identification.

[0044] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0046] Figure 1 A flowchart of a voiceprint identification method provided by an embodiment of this disclosure is shown;

[0047] Figure 2 A flowchart of another voiceprint identification method provided by an embodiment of this disclosure is shown;

[0048] Figure 3 A schematic diagram of a voiceprint identification device provided in an embodiment of this disclosure is shown;

[0049] Figure 4 A schematic diagram of an electronic device provided in an embodiment of the present disclosure is shown. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0051] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0052] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0053] Research has found that existing methods for identifying the same person using formant technology mainly focus on comparing vowel formant frequencies, with vowel formant extraction primarily relying on Linear Predictive Coding (LPC). LPC is based on the assumption of an all-pole model for speech generation, assuming that speech signals can be generated by an all-pole filter. However, in reality, speech generation is very complex, and actual speech signals do not perfectly conform to the all-pole model. In many cases, speech signals may contain zero-point components, i.e., anti-resonance. The LPC model cannot accurately represent this zero-point information, leading to inaccurate modeling of the speech signal. The information filtered out by LPC may reflect an individual's pronunciation characteristics (such as a heavy nasal tone).

[0054] Based on the above research, this disclosure provides a voiceprint identification method, apparatus, electronic device, and storage medium. The method involves acquiring a speech segment to be identified and a reference speech segment; segmenting the speech segment to be identified into multiple phoneme segments to be identified; segmenting the reference speech segment into multiple reference phoneme segments; aligning the phoneme segments to be identified and the reference phoneme segments in the time dimension; determining the aligned spectrograms to be identified and the reference spectrograms; adjusting the image parameters corresponding to the spectrograms to be identified and the reference spectrograms; and overlaying the adjusted spectrograms to be identified and the reference spectrograms to generate a visualized formant difference map; detecting the frequency band difference value of the formant energy distribution, the difference value of the formant energy intensity, and the existence of formant trend differences displayed in the visualized formant difference map; if the formant trend differences exist, determining the voiceprint identification result based on the frequency band difference value of the formant energy distribution and the difference value of the formant energy intensity. Detection can be performed based on graphic overlay. One or more audio segments to be compared are superimposed, and by setting different parameters such as color and transparency, the morphological differences between segments can be highlighted. The differences in the formant morphology of different audio segments can be seen intuitively, thus improving the accuracy of voiceprint identification.

[0055] To facilitate understanding of this embodiment, a detailed description of the voiceprint identification method disclosed in this disclosure is provided first. The execution entity of the voiceprint identification method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this voiceprint identification method can be implemented by a processor calling computer-readable instructions stored in memory.

[0056] See Figure 1 The diagram shows a flowchart of a voiceprint identification method provided in an embodiment of this disclosure. The method includes steps S101 to S104, wherein:

[0057] S101. Obtain the speech segment to be identified and the reference speech segment, divide the speech segment to be identified into multiple phoneme segments to be identified, and divide the reference speech segment into multiple reference phoneme segments.

[0058] In this embodiment, the voice segment to be identified and the reference voice segment are first acquired using a voiceprint acquisition device (such as a microphone array, mobile terminal, or voice recorder). The voice segment to be identified can originate from real-time voice collected on-site, historical voice samples, or audio files, while the reference voice segment is pre-stored and verified voice data. Next, voice preprocessing operations are performed on both voice segments.

[0059] Here, speech preprocessing operations include, but are not limited to: noise suppression using spectral subtraction, Wiener filtering, or deep neural network noise reduction models to remove background noise from the original speech; speech enhancement to improve the clarity and separability of speech, for example, through short-time energy normalization, speech activity detection (VAD), etc.; sampling rate standardization to unify the speech sampling rate to 16kHz or other set standard values; framing and windowing to perform short-time framing of the speech signal and apply window functions such as Hamming windows to preserve local speech stability features.

[0060] Among them, a phoneme is a basic concept in phonetics. It refers to the smallest unit of speech that is divided according to the natural properties of speech. From the perspective of individual pronunciation, the same phonemes in the speech produced by an individual have a relatively stable acoustic structure. This structure has a certain degree of interpersonal variability, and therefore can be used in voiceprint identification and recognition.

[0061] Furthermore, after preprocessing, the speech segments are segmented using a forced phoneme alignment tool. In this embodiment, the Montreal Forced Aligner (MFA) is preferably used, which automatically annotates the input speech based on a pre-trained HMM-GMM acoustic model and dictionary.

[0062] Specifically, the speech segment to be identified and the corresponding text of the speech content are input into the MFA. The MFA supports Pinyin, IPA phonetic symbols, or other phoneme transcription formats. The MFA internally uses the Viterbi decoding algorithm to calculate the maximum likelihood path between the speech signal and the phoneme sequence, thereby outputting the start and end times of each phoneme. Based on the annotation results, the entire speech signal is divided into multiple phoneme segments to be identified, and each phoneme segment corresponds to a minimum unit of speech articulation. Similarly, the above operation is performed on the reference speech segment to obtain multiple reference phoneme segments.

[0063] Here, each phoneme segment contains the time interval of the audio signal and phoneme category labels (such as / a / , / i / , / u / , / b / , / d / , / s / , etc.), and can be further used for subsequent spectrogram generation and alignment analysis.

[0064] It should be noted that, to improve alignment accuracy, the phoneme boundary segmentation effect can be optimized by adjusting phoneme model parameters, introducing multi-speaker acoustic models, and introducing dynamic duct length modeling. Furthermore, in specific applications, alignment schemes based on end-to-end deep models, such as CTC, Attention-based Alignment, or DurationPredictor, can be used for phoneme segmentation.

[0065] S102. Align the phoneme segment to be identified with the reference phoneme segment in the time dimension, and determine the spectrogram to be identified and the reference spectrogram after phoneme alignment respectively.

[0066] In specific implementation, after obtaining the phoneme segment to be identified and the reference phoneme segment, this embodiment further includes aligning the two phoneme segments in the time dimension and generating the aligned spectrograms to be identified and the reference spectrograms, respectively. Since speech segments from different sources differ in pronunciation speed, speech rate rhythm, and oral cavity architecture, even the same phoneme may have different durations. Direct spectrogram comparison may lead to formant misalignment or distortion. Therefore, it is necessary to align the phoneme segments in the time dimension.

[0067] Here, the Dynamic Time Warping (DTW) algorithm can be used for time alignment of phoneme segments. DTW is a nonlinear registration method based on dynamic programming that can effectively align two time-series signals of unequal length but with similar patterns.

[0068] Specifically, acoustic features such as Short-Time Fourier Transform (STFT), Mel-frequency cepstral coefficients (MFCC), or Mel-spectrogram are extracted for each phoneme segment to form a two-dimensional time-frequency matrix. The feature distance between the phoneme segment to be identified and the reference phoneme segment at each frame time point is calculated to form a cumulative distance matrix (usually using Euclidean distance or cosine distance). Using the DTW dynamic programming algorithm, a time alignment path that minimizes the cumulative distance is searched, compressing longer phonemes in the time dimension and expanding shorter phonemes to obtain an aligned time frame index mapping. Based on the alignment path, the acoustic features of the phoneme segment to be identified and the reference phoneme segment are resampled to ensure that they have the same frame number and time scale, thereby achieving synchronization in the time dimension.

[0069] In some implementations, end-to-end attention alignment models (such as Transformer-based monotonic alignment) or neural network-level temporal alignment based on differentiable long-term modeling mechanisms (such as DurIAN and FastSpeech) can be combined to improve robustness and boundary accuracy.

[0070] Furthermore, after completing the time alignment of phoneme segments, corresponding spectrograms are generated for the aligned phoneme segments to be identified and the reference phoneme segments, respectively. The audio signal of each frame is transformed using Short Time Fourier Transform (STFT) or Mel filter bank to obtain its spectral representation, with time on the horizontal axis, frequency on the vertical axis, and chroma or luminance representing energy intensity. Based on the time mapping relationship obtained after DTW alignment, it is ensured that the phoneme content represented by the two spectrograms corresponds in each frame. The spectral information is converted into a two-dimensional image form to form the spectrogram to be identified and the reference spectrogram, which facilitates subsequent image overlay and difference analysis.

[0071] In practical applications, spectrograms can be normalized, log-scaled, or weighted smoothed to enhance the clarity of formant boundaries. Furthermore, auxiliary information such as phoneme boundaries and frequency scales can be annotated in the image to facilitate manual or automated analysis by experts or algorithms.

[0072] S103. Adjust the image parameters corresponding to the spectrogram to be identified and the reference spectrogram respectively, and overlay the spectrogram to be identified after parameter adjustment with the reference spectrogram to generate a visual formant difference map.

[0073] In practice, after obtaining the aligned spectrogram to be identified and the reference spectrogram, in order to achieve a more intuitive and clear visualization analysis of formant differences, this embodiment further adjusts the image parameters of the two spectrograms respectively, and superimposes the adjusted images to generate a visualized formant difference map.

[0074] Here, since the spectrograms generated from different speech segments may vary significantly in terms of image attributes such as color intensity, brightness, saturation, and transparency, direct superposition may result in indistinguishable formant overlap areas or severe color interference, affecting visual resolution. Therefore, it is necessary to unify and adjust the image parameters.

[0075] In this embodiment, the following image parameters can be adjusted: First, use image editing tools (such as affine transformation and free transformation) to geometrically align the main body (e.g., the main formant region) of the spectrogram to be identified and the reference spectrogram to ensure that the center frequency position and the start and end points of the phoneme time axis correspond. Set the opacity of the two images so that they can be superimposed in one image, forming a transparent overlapping area. The default setting range is 0%-100%. Generally, the reference spectrogram is set to a higher opacity (e.g., 80%-100%), while the spectrogram to be identified is set to a lower opacity (e.g., 30%-60%) to create a distinct overlapping effect. Unify the color intensity and brightness of the two images to avoid one layer appearing too dark or too bright after image superposition. The default saturation and brightness adjustment range is 0%-100%, which can be fine-tuned according to the actual color dynamic range. Adjust the dominant color tone of the spectrograms to ensure the two images use distinguishable color channels; for example, use blue-purple tones for the baseline image and yellow-red tones for the image to be identified, thus enhancing contrast. The hue adjustment range is 0°-360°, with complementary color pairs being preferred. Feather the image edges to smooth transitions and reduce recognition interference from sharp abrupt changes in overlapping boundaries. The default feather radius range is 0–250 pixels, which should be adjusted according to image resolution. Configure the blending method between image layers on different brightness or color channels; for example, use a luminance channel mask similar to Photoshop to partially mask layers and highlight areas of difference.

[0076] Furthermore, after adjusting the parameters mentioned above, the two images are overlaid. The reference spectrogram is placed at the bottom layer, and the spectrogram to be identified is placed on top layer. The adjusted opacity and blending mode are applied. Image blending modes such as "overlay," "difference," and "screen" are used to fuse the formant regions, ensuring that the formant contours are clear and distinguishable, and that the different energy distributions have obvious differentiation in color contrast. The final output image shows the differences between the two spectrograms in the formant regions, including frequency position shifts, energy intensity differences, and inconsistent formant trends, which can be directly used for subsequent difference detection and voiceprint analysis.

[0077] In other alternative embodiments, post-processing operations such as edge enhancement, color level mapping, and logarithmic compression can be performed on the image to further improve the recognition and data representation of the visualized image.

[0078] As one possible implementation, the main content in the spectrogram to be identified and the reference spectrogram are aligned; the opacity and feather radius of the spectrogram to be identified and the reference spectrogram are adjusted, and a gradient transition or semi-transparent background is created to adjust the layer overlay effect between the spectrogram to be identified and the reference spectrogram; the saturation, brightness, hue and mixed color bands of the spectrogram to be identified and the reference spectrogram are unified to generate a visual formant difference map.

[0079] In practice, firstly, the main content of the spectrogram to be identified and the reference spectrogram are spatially aligned to determine the main formant regions (such as the frequency bands F1, F2, and F3) in each of the two images; then, image free transformation tools (such as affine transformation and horizontal / vertical scaling) are used to adjust the spectrogram to be identified to be aligned with the reference spectrogram on the frequency and time axes; finally, the time axis range is finely registered according to the start and end time points of the phoneme boundaries.

[0080] Optionally, reference lines can be set for different formant main frequency bands to assist in alignment, ensuring that the spectral peaks of the same phoneme segment are spatially comparable.

[0081] Here, to make the visual effect of layer overlay more discernible, this embodiment processes the layer transparency and boundary transition features. The reference spectrogram is set to a higher opacity (e.g., 80%-100%) to preserve its original information for reference; the spectrogram to be identified is set to a lower opacity (e.g., 40%-60%) to form an overlay layer and show the differences; feathering is applied to the edge areas of the spectrogram to be identified (e.g., a radius of 20-80 pixels) to achieve a natural transition of layer edges and avoid visual interference caused by boundary jumps; the image transparency changes are controlled by linear or radial gradient methods to make the formant information in different areas present a sense of hierarchy, making it easier to observe the commonalities and differences in overlapping areas.

[0082] To further enhance the contrast and recognizability of the overlay images, the color parameters of the images are uniformly adjusted, and a uniform saturation level (e.g., 60%-80%) is set to ensure consistent color intensity and avoid difficulty in comparison due to differences in color intensity. The image brightness is uniform to enhance the clarity of the formant peaks on the background. Complementary tones are set for different images, for example, the reference spectrogram is a cool tone (blue-purple) and the spectrogram to be identified is a warm tone (orange-red), which enhances the ability to distinguish differences through color differentiation. When overlaying images, a "mixing color band" is set to selectively mix images of different brightness ranges based on brightness or color channels, thereby highlighting the differences between bright and dark areas in the overlapping region.

[0083] In this way, through the coordinated adjustment of the aforementioned image parameters, the generation of the visualized formant difference map is completed. The final image presents the formant feature differences between the speech to be identified and the reference speech in the same phoneme segment in an intuitive visual manner, providing an interpretable basis for subsequent speaker recognition based on image differences.

[0084] S104. Detect the frequency band difference value of the energy distribution of the resonance peak, the energy intensity difference value of the resonance peak, and whether there is a difference in the trend of the resonance peak displayed in the visualized resonance peak difference map. If there is a difference in the trend of the resonance peak, determine the voiceprint identification result based on the frequency band difference value of the energy distribution of the resonance peak and the energy intensity difference value of the resonance peak.

[0085] In specific implementation, after generating the visualized formant difference map, in order to realize the automatic comparison and judgment of the formant morphology between the speech segment to be identified and the reference speech segment, this embodiment further includes a difference detection and voiceprint identification result judgment step. In the visualized formant difference map, formant difference analysis is performed on three dimensions: the frequency band difference value of formant energy distribution, the difference value of formant energy intensity, and the difference of formant trend.

[0086] Specifically, for detecting the frequency band difference in formant energy distribution, the pixel range of the significant color region on the vertical axis (frequency axis) of the difference map is scanned, and the main formant frequency bands at each phoneme position of the spectrogram to be identified and the reference spectrogram are extracted respectively. The upper and lower limits of the main formant frequency range of the corresponding phoneme are compared, and the frequency interval difference is calculated. This difference is used as the frequency band difference value of formant energy distribution to measure the frequency shift phenomenon caused by vocal tract deformation. For detecting the difference in formant energy intensity, the area occupied by the image color saturation or brightness of the formant region in each phoneme position is counted. The number of high-intensity regions (such as the number of pixels above the RGB threshold) in the spectrogram to be identified and the reference spectrogram are calculated respectively. The area difference between the two is obtained as the formant energy intensity difference value, reflecting the difference in sound source energy and vocal organ tension. To detect differences in formant trends, the evolution of the formant centerline or trajectory over time is analyzed in the visualized formant difference graph. If there are significantly different undulation directions or dynamic transition characteristics (such as different formant starting frequencies or rising / falling trends), then differences in formant trends are confirmed. In particular, the transition states of main formants such as F1 and F2 between phonemes are discriminated to reflect the dynamic characteristics of the speech flow.

[0087] Specifically, in the visualized formant difference map, the difference in the vertical frequency range between the spectrogram to be identified and the reference spectrogram is determined, and this difference in the vertical frequency range is defined as the frequency band difference value of the formant energy distribution; the difference in pixel area between the spectrogram to be identified and the reference spectrogram is determined, and this difference in pixel area is defined as the difference value of the formant energy intensity; the overall trend direction between the spectrogram to be identified and the reference spectrogram is determined in the visualized formant difference map; if there is a difference in the overall trend direction corresponding to the dynamic trajectory between the spectrogram to be identified and the reference spectrogram, then a difference in the formant trend is determined to exist.

[0088] In practice, the distribution range of significant formant regions on the vertical axis (i.e., the frequency axis) is extracted from the visualized formant difference map. For each phoneme segment to be compared, the frequency start and end values ​​of the formants in the spectrogram to be identified and the reference spectrogram are identified. The formant frequency range of each phoneme segment is calculated, and the frequency interval difference between the two is taken. This frequency interval difference is defined as the formant energy distribution frequency band difference value, which is used to reflect the physiological / phonological differences between the two speech segments in terms of formant frequency configuration.

[0089] To measure the energy characteristics of the sound source and the changes in resonance intensity, this embodiment further compares the differences in brightness or intensity regions of the image in space. Contour recognition is performed on the formant regions of each phoneme segment in the visualized formant difference map, and high-energy regions are extracted using a threshold segmentation method. The pixel areas of the bright regions at the same position in the spectrogram to be identified and the reference spectrogram are counted (e.g., the total number of pixels that meet a specific saturation / brightness). The difference between the two area values ​​is calculated. This difference is defined as the formant energy intensity difference value, which is used to quantify the difference in vocal tract energy output between the two speech samples during pronunciation.

[0090] Furthermore, based on the positional changes of the formant image trajectories (e.g., the main formant trajectories such as F1, F2, and F3) along the time axis, their trend segments are extracted; using linear fitting, Bézier curve modeling, and other methods, trend models are established for the formant trajectories of the spectrogram to be identified and the benchmark spectrogram, respectively; their trend slopes, fluctuation patterns, and upward / downward trends are compared to determine whether their overall trend directions are consistent; if there is a significant deviation in the trajectory direction (e.g., one is an upward trend while the other is a downward trend), then it is determined that there is a difference in the formant trend.

[0091] As one possible implementation method, see Figure 2 The diagram shows a flowchart of a voiceprint identification method provided in an embodiment of this disclosure. The method includes steps S201 to S202, wherein:

[0092] S201. Compare the frequency band difference value of the resonance peak energy distribution with the preset first threshold range to determine the first difference level to which the frequency band difference value of the resonance peak energy distribution belongs, wherein the first difference level includes a large difference level, a medium difference level, and a small difference level.

[0093] S202. Compare the energy intensity difference value of the resonance peak with the preset second threshold range to determine the second difference level to which the energy intensity difference value of the resonance peak belongs, wherein the second difference level includes a large energy intensity difference level, a medium energy intensity difference level, and a small energy intensity difference level.

[0094] In the specific implementation, after extracting the frequency band difference value of the resonant peak energy distribution and the energy intensity difference value of the resonant peak, in order to further improve the sensitivity and fault tolerance of voiceprint comparison, this embodiment introduces a difference level classification mechanism to quantify and classify the difference values.

[0095] Here, the extracted frequency band difference values ​​of the resonance peak energy distribution are compared with the preset first threshold interval, and divided into different levels according to the degree of difference to reflect the physiological differences in the frequency band: if the difference value is ≥ the first high threshold (e.g., ≥200Hz), it is determined to be a large difference level in the frequency band; if the difference value is between the first medium threshold and the first high threshold (e.g., 100Hz ≤ difference value < 200Hz), it is determined to be a medium difference level in the frequency band; if the difference value is < the first medium threshold (e.g., < 100Hz), it is determined to be a small difference level in the frequency band.

[0096] Similarly, the energy intensity difference value of the resonance peak (e.g., represented by pixel area difference or gray-scale integral difference) is compared with the interval divided by the second threshold to be classified into the corresponding energy difference level: if the difference value is ≥ the second high threshold (e.g., pixel area difference ≥ 1500 pixel units), it is classified as a large energy intensity difference level; if the difference value is between the second medium threshold and the second high threshold (e.g., 800 ≤ difference value < 1500), it is classified as a medium energy intensity difference level; if the difference value is < the second medium threshold (e.g., < 800), it is classified as a small energy intensity difference level.

[0097] Furthermore, when the large difference level of the distribution frequency band belongs to the large difference level of the distribution frequency band, and the difference value of the formant energy intensity belongs to the large difference level of energy intensity, it is determined that the speech segment to be identified and the reference speech segment belong to the same person; when the large difference level of the distribution frequency band belongs to the medium difference level or the small difference level of energy intensity, and the difference value of the formant energy intensity belongs to the medium difference level or the small difference level of energy intensity, it is determined that the speech segment to be identified and the reference speech segment do not belong to the same person.

[0098] Here, when a difference in formant trend is detected, it indicates that the phoneme has a significantly different vocal dynamic trajectory during the time evolution process, and will enter the difference level determination stage. Combined with the other two difference dimensions, if either the formant energy distribution frequency band difference value or the energy intensity difference value reaches the preset significant difference threshold (such as the difference exceeding a certain frequency offset value or the area difference ratio exceeding a set percentage), it can be directly determined that the speech segment and the reference speech segment do not belong to the same speaker; if both are at a low difference level, but the trend difference exists, they can also be marked as samples to be further confirmed; if all three differences are not significant or only slight differences exist, they can be determined to belong to the same speaker.

[0099] In this way, by classifying the two types of difference values, continuous numerical values ​​can be transformed into discrete semantic indicators, thus providing a logical basis for the final voiceprint comparison decision. For example, if both difference levels are "large difference level", it indicates that the voiceprint features are significantly different and they are determined to be different people; if both difference levels are "small difference level", it is considered a high match and it is presumed to be the same person; if there is a conflict in the difference levels, the results of the formant trend difference judgment can be combined for comprehensive analysis, or the situation can be transferred to the manual review process.

[0100] This disclosure provides a voiceprint identification method that involves acquiring a speech segment to be identified and a reference speech segment; segmenting the speech segment to be identified into multiple phoneme segments to be identified, and segmenting the reference speech segment into multiple reference phoneme segments; aligning the phoneme segments to be identified and the reference phoneme segments in the time dimension to determine the aligned spectrograms to be identified and the reference spectrograms; adjusting the image parameters corresponding to the spectrograms to be identified and the reference spectrograms, and overlaying the adjusted spectrograms to be identified and the reference spectrograms to generate a visualized formant difference map; detecting the frequency band difference value of the formant energy distribution, the difference value of the formant energy intensity, and the existence of formant trend differences displayed in the visualized formant difference map; if the formant trend differences exist, determining the voiceprint identification result based on the frequency band difference value of the formant energy distribution and the difference value of the formant energy intensity. Detection can be performed based on graphic overlay. One or more audio segments to be compared are superimposed, and by setting different parameters such as color and transparency, the morphological differences between segments can be highlighted. The differences in the formant morphology of different audio segments can be seen intuitively, thus improving the accuracy of voiceprint identification.

[0101] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0102] Based on the same inventive concept, this disclosure also provides a voiceprint identification device corresponding to the voiceprint identification method. Since the principle of the device in this disclosure is similar to the voiceprint identification method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0103] Please see Figure 3 , Figure 3 This is a schematic diagram of a voiceprint identification device provided in an embodiment of this disclosure. Figure 3 As shown in the figure, the voiceprint identification device 300 provided in this embodiment includes:

[0104] The phoneme segmentation module 310 is used to acquire the speech segment to be identified and the reference speech segment, segment the speech segment to be identified into multiple phoneme segments to be identified, and segment the reference speech segment into multiple reference phoneme segments.

[0105] The phoneme alignment module 320 is used to align the phoneme segment to be identified with the reference phoneme segment in the time dimension, and to determine the spectrogram to be identified and the reference spectrogram after phoneme alignment.

[0106] The image adjustment and overlay module 330 is used to adjust the image parameters corresponding to the spectrogram to be identified and the reference spectrogram respectively, and overlay the spectrogram to be identified after parameter adjustment with the reference spectrogram to generate a visual formant difference map.

[0107] The difference detection module 340 is used to detect the frequency band difference value of the energy distribution of the resonance peak, the energy intensity difference value of the resonance peak, and whether there is a difference in the trend of the resonance peak displayed in the visualized resonance peak difference map. If the difference in the trend of the resonance peak exists, the voiceprint identification result is determined based on the frequency band difference value of the energy distribution of the resonance peak and the energy intensity difference value of the resonance peak.

[0108] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0109] This disclosure provides a voiceprint identification device that acquires a speech segment to be identified and a reference speech segment. The speech segment to be identified is segmented into multiple phoneme segments to be identified, and the reference speech segment is segmented into multiple reference phoneme segments. The phoneme segments to be identified and the reference phoneme segments are aligned in the time dimension to determine the aligned spectrograms to be identified and the reference spectrograms. The image parameters corresponding to the spectrograms to be identified and the reference spectrograms are adjusted, and the adjusted spectrograms to be identified and the reference spectrograms are superimposed to generate a visualized formant difference map. The device detects the frequency band difference value of the formant energy distribution, the difference value of the formant energy intensity, and the existence of formant trend differences displayed in the visualized formant difference map. If the formant trend differences exist, the voiceprint identification result is determined based on the frequency band difference value of the formant energy distribution and the difference value of the formant energy intensity. Detection can be performed based on graphic overlay. One or more audio segments to be compared are superimposed, and by setting different parameters such as color and transparency, the morphological differences between segments can be highlighted. The differences in the formant morphology of different audio segments can be seen intuitively, thus improving the accuracy of voiceprint identification.

[0110] Corresponding to Figure 1In addition to the voiceprint identification method, this disclosure also provides an electronic device 400, such as... Figure 4 The diagram shown is a structural schematic of an electronic device 400 provided in an embodiment of this disclosure, including:

[0111] Processor 41, memory 42, and bus 43; memory 42 is used to store execution instructions, including main memory 421 and external memory 422; the main memory 421, also called internal memory, is used to temporarily store the computational data in processor 41, as well as the data exchanged with external memory 422 such as hard disk. Processor 41 exchanges data with external memory 422 through main memory 421. When the electronic device 400 is running, processor 41 and memory 42 communicate through bus 43, enabling processor 41 to execute... Figure 1 and Figure 2 The steps of the voiceprint identification method in China.

[0112] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the voiceprint identification method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0113] This disclosure also provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they can perform the steps of the voiceprint identification method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0114] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0115] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0116] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0117] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0118] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for voiceprint identification, characterized in that, include: Obtain the speech segment to be identified and the reference speech segment, segment the speech segment to be identified into multiple phoneme segments to be identified, and segment the reference speech segment into multiple reference phoneme segments; Align the phoneme segment to be identified with the reference phoneme segment in the time dimension, and determine the spectrogram of the segment to be identified and the reference spectrogram after phoneme alignment respectively. The image parameters corresponding to the spectrogram to be identified and the reference spectrogram are adjusted respectively, and the spectrogram to be identified after parameter adjustment is superimposed on the reference spectrogram to generate a visual formant difference map. The frequency band difference value of the energy distribution of the resonance peaks, the energy intensity difference value of the resonance peaks, and the existence of resonance peak trend differences are detected in the visualized resonance peak difference map. If the resonance peak trend differences exist, the voiceprint identification result is determined based on the frequency band difference value of the energy distribution of the resonance peaks and the energy intensity difference value of the resonance peaks. Adjust the image parameters corresponding to the spectrogram to be identified and the reference spectrogram respectively, and then overlay the parameter-adjusted spectrogram to be identified and the reference spectrogram to generate a visual formant difference map, specifically including: Align the main content of the spectrogram to be identified with that of the reference spectrogram; Adjust the opacity and feather radius of the spectrogram to be identified and the reference spectrogram, and create a gradient transition or semi-transparent background to adjust the layer overlay effect between the spectrogram to be identified and the reference spectrogram; The saturation, brightness, hue, and mixed color bands of the spectrogram to be identified and the reference spectrogram are unified to generate the visual formant difference map; The detection of the frequency band differences in energy distribution of the resonance peaks, the differences in energy intensity of the resonance peaks, and the presence of differences in the trend of the resonance peaks displayed in the visualized resonance peak difference map specifically includes: In the visualized formant difference diagram, the difference in the vertical axis frequency range between the spectrogram to be identified and the reference spectrogram is determined, and the difference in the vertical axis frequency range is determined as the difference in the frequency band of the formant energy distribution. In the visualized formant difference map, the pixel area difference between the spectrogram to be identified and the reference spectrogram is determined, and the pixel area difference is determined as the energy intensity difference of the formant. Determine the overall trend direction between the spectrogram to be identified and the reference spectrogram in the visualized formant difference diagram; If there is a difference in the overall trend direction corresponding to the dynamic trajectory between the spectrogram to be identified and the reference spectrogram, then it is determined that there is a difference in the trend of the formant peaks.

2. The method according to claim 1, characterized in that, Before determining the voiceprint identification result based on the frequency band difference value of the resonance peak energy distribution and the frequency band difference value of the resonance peak energy intensity, the method further includes: The frequency band difference value of the resonance peak energy distribution is compared with the preset first threshold range to determine the first difference level to which the frequency band difference value of the resonance peak energy distribution belongs. The first difference level includes a large difference level, a medium difference level, and a small difference level in the distribution frequency band. The energy intensity difference value of the resonance peak is compared with the preset second threshold range to determine the second difference level to which the energy intensity difference value of the resonance peak belongs. The second difference level includes a large energy intensity difference level, a medium energy intensity difference level, and a small energy intensity difference level.

3. The method according to claim 2, characterized in that, The voiceprint identification result is determined based on the difference in frequency band of the resonance peak energy distribution and the difference in resonance peak energy intensity, specifically including: When the large difference level of the distribution frequency band belongs to the large difference level of the distribution frequency band, and the energy intensity difference value of the formant peak belongs to the large difference level of the energy intensity, it is determined that the speech segment to be identified and the reference speech segment belong to the same person. When the large difference level of the distribution frequency band belongs to the medium difference level of the distribution frequency band or the small difference level of the energy intensity, and the energy intensity difference value of the resonance peak belongs to the medium difference level of the energy intensity or the small difference level of the energy intensity, it is determined that the speech segment to be identified and the reference speech segment do not belong to the same person.

4. A voiceprint identification device, characterized in that, include: The phoneme segmentation module is used to acquire the speech segment to be identified and the reference speech segment, segment the speech segment to be identified into multiple phoneme segments to be identified, and segment the reference speech segment into multiple reference phoneme segments. The phoneme alignment module is used to align the phoneme segment to be identified with the reference phoneme segment in the time dimension, and to determine the spectrogram to be identified and the reference spectrogram after phoneme alignment, respectively. The image adjustment and overlay module is used to adjust the image parameters corresponding to the spectrogram to be identified and the reference spectrogram respectively, and overlay the spectrogram to be identified after parameter adjustment with the reference spectrogram to generate a visual formant difference map. The difference detection module is used to detect the difference values ​​of the frequency band of the energy distribution of the resonance peak, the difference values ​​of the energy intensity of the resonance peak, and whether there are differences in the trend of the resonance peak displayed in the visualized resonance peak difference map. If there are differences in the trend of the resonance peak, the voiceprint identification result is determined based on the difference values ​​of the frequency band of the energy distribution of the resonance peak and the difference values ​​of the energy intensity of the resonance peak. The image adjustment and overlay module is specifically used for: Align the main content of the spectrogram to be identified with that of the reference spectrogram; Adjust the opacity and feather radius of the spectrogram to be identified and the reference spectrogram, and create a gradient transition or semi-transparent background to adjust the layer overlay effect between the spectrogram to be identified and the reference spectrogram; The saturation, brightness, hue, and mixed color bands of the spectrogram to be identified and the reference spectrogram are unified to generate the visual formant difference map; The difference detection module is specifically used for: In the visualized formant difference diagram, the difference in the vertical axis frequency range between the spectrogram to be identified and the reference spectrogram is determined, and the difference in the vertical axis frequency range is determined as the difference in the frequency band of the formant energy distribution. In the visualized formant difference map, the pixel area difference between the spectrogram to be identified and the reference spectrogram is determined, and the pixel area difference is determined as the energy intensity difference of the formant. Determine the overall trend direction between the spectrogram to be identified and the reference spectrogram in the visualized formant difference diagram; If there is a difference in the overall trend direction corresponding to the dynamic trajectory between the spectrogram to be identified and the reference spectrogram, then it is determined that there is a difference in the trend of the formant peaks.

5. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the voiceprint identification method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the voiceprint identification method as described in any one of claims 1 to 3.

Images