Voice voiceprint region tracing method, device and equipment and storage medium

Abstract

The invention provides a voiceprint regional traceability method and device of voice, equipment and a storage medium, and the method comprises the steps: carrying out the audio preprocessing of to-be-analyzed voice, extracting acoustic features, and generating an international phonetic symbol phoneme sequence; classifying the international phonetic symbol phoneme sequence based on a pre-constructed multi-level linguistic feature knowledge base and a statistical machine learning model, and determining a plurality of phonetic feature fragments; wherein each sound class feature fragment comprises a sound class type, a time interval and a confidence coefficient; and inputting the plurality of voice class feature fragments into a pre-trained regional attribute determination model, and outputting a regional attribute tracing result of the to-be-analyzed voice by modeling a conditional probability relationship between the voice class features and a dialect region. Regional traceability is realized through multi-level linguistic knowledge modeling, statistical machine learning driven feature extraction and a regional attribute determination model, and the problems of inaccuracy and low efficiency in the existing voiceprint identification technology are fundamentally overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech recognition, in particular to a voiceprint regional tracing method and device, equipment and a storage medium. BACKGROUND

[0002] Voiceprint identification is an important means to determine the identity of the speaker by analyzing the individual sound characteristics in the speech signal. With the wide application of speech data in criminal cases, investigation, electronic evidence and other scenes, voiceprint identification has become one of the indispensable forensic science technologies. Among them, the listening analysis as the core link of voiceprint identification directly affects the scientificity and accuracy of the identification conclusion. The current mainstream voiceprint identification workstation adopts the "same word search" listening method, that is, based on the speech transcription result, the same words or phrases in the sample and the sample are searched to form a candidate segment list for manual comparison and playback by the identification personnel. Although this method improves the keyword search efficiency, it still has the following defects: first, the matching granularity is limited to the word level, and it is difficult to compare the pronunciation characteristics at the phoneme level; second, it relies on manual input of keywords for passive search, and it is difficult to actively find potential comparable phoneme segments; third, it lacks support for phonetics rules, and cannot distinguish between "comparable" and "incomparable" phonemes (such as / a / in "big" and / a / in "flower" are not comparable due to different contexts), which increases the risk of misjudgment. More importantly, the existing technology generally ignores the guiding role of linguistic knowledge in voiceprint analysis. In summary, the existing voiceprint identification technology has obvious shortcomings in single feature extraction dimension, lack of linguistic support in analysis logic, lack of regional attribute reasoning ability, and insufficient human-computer interaction adaptation, and an urgent need for a voiceprint regional tracing method to improve the accuracy of voiceprint identification. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a voiceprint regional tracing method, device, equipment and storage medium, which fundamentally overcomes the inaccuracy and low efficiency of the existing voiceprint identification technology by multi-level linguistic knowledge modeling, statistical machine learning driven feature extraction and regional attribute determination model.

[0004] The voiceprint regional tracing method provided by the present application comprises: audio preprocessing of the speech to be analyzed, extracting acoustic features and generating an international phonetic alphabet phoneme sequence; classifying the international phonetic alphabet phoneme sequence based on a pre-constructed multi-level linguistic feature knowledge base and a statistical machine learning model to determine a plurality of phoneme class feature segments; wherein each phoneme class feature segment comprises a phoneme class type, a time interval and a confidence level. The plurality of sound class feature segments are input into a pre-trained regional attribute determination model, and the regional attribute tracing result of the speech to be analyzed is output by modeling the conditional probability relationship between the sound class features and the dialect area.

[0005] In one possible implementation, the audio preprocessing of the speech to be analyzed, the extraction of acoustic features, and the generation of the International Phonetic Alphabet (IPA) phoneme sequence include: The speech to be analyzed is subjected to noise reduction processing, and the speech to be analyzed after noise reduction is framed according to a fixed time window to obtain a speech frame sequence. At least one acoustic feature is extracted for each speech frame; wherein the acoustic feature includes at least one of a Mel-frequency cepstrum coefficient, a filter bank energy, and a prosodic feature. Based on a speech transcription model fused with a dialect dictionary, the speech frame sequence and its corresponding acoustic features are classified frame by frame to output a timestamped text transcription result and a corresponding International Phonetic Alphabet (IPA) phoneme sequence; wherein the speech transcription model is constructed based on a Wav2Vec2.0 architecture and is obtained by fine-tuning training on a large-scale speech corpus containing multi-dialect pronunciation samples.

[0006] In one possible implementation, the multi-level linguistic feature knowledge base is constructed by the following steps: Dialect speech samples and corresponding text transcription data covering multiple geographical regions are collected; The basic sound class features, regional specificity features, and context association features are extracted from the dialect speech samples and correlated to construct a multi-level linguistic feature knowledge base; The basic sound class features are used to describe common pronunciation variation types in speech and define the comparability rules between phonemes, the regional specificity features are used to establish the mapping relationship between pronunciation variation patterns and dialect areas, and the context association features are used to represent the variation rules of the same sound class in different language use environments.

[0007] In one possible implementation, the classification of the International Phonetic Alphabet (IPA) phoneme sequence based on the pre-constructed multi-level linguistic feature knowledge base and the statistical machine learning model includes: The International Phonetic Alphabet (IPA) phoneme sequence is classified using a bidirectional long short-term memory network, the sound class features in the multi-level linguistic feature knowledge base are matched, and candidate sound class segments are generated; The XGBoost model is used to combine the context association features and the regional distribution prior probability to reorder the confidence of the candidate sound class segments, retain the effective sound class feature segments with high confidence, and determine the plurality of sound class feature segments.

[0008] In a possible implementation, the inputting of the plurality of phonetic feature segments into the pre-trained regional attribute determination model comprises modeling a conditional probability relationship between the phonetic feature and the dialect area, and outputting a regional attribute tracing result of the speech to be analyzed. calculating a response probability of each target dialect area to the combination of the phonetic features, to generate a probability distribution of the regional attribution of the speaker; According to the contribution of each phonetic feature to the dialect area, the key evidence path supporting the probability distribution of the regional attribution of the speaker is reversely analyzed to form an interpretable regional attribute inference conclusion, and a key evidence chain explanation is outputted. Based on the probability distribution of the regional attribution of the speaker and the key evidence chain explanation supporting the conclusion, the regional attribute tracing result is constructed.

[0009] In a possible implementation, the regional attribute determination model is determined by the following steps: Collecting a plurality of speech sample sets of dialect areas; Constructing a Bayesian network model, wherein the input nodes are the phonetic features in each speech sample and their occurrence frequency and confidence, the hidden layer nodes represent the potential dialect language habit patterns, and the output nodes are the attribution categories of each dialect area; Based on the statistics of the co-occurrence frequency of each phonetic feature in different dialect areas in the speech sample set, the Bayesian network model is trained by using maximum likelihood estimation; Using cross-validation method to determine the accuracy rate of regional attribute prediction of the Bayesian network model on unknown samples, and adjusting the network structure or smoothing the prior parameters according to the feedback until the preset performance threshold is met, to determine the regional attribute determination model.

[0010] In a possible implementation, after the outputting of the regional attribute tracing result of the speech to be analyzed, the voiceprint regional tracing method further comprises: Providing the regional attribute tracing result to an interface to support multi-dimensional display of phonetic segments, geographic information system visualization, synchronous playback comparison of evidence and samples, spectrum analysis and intelligent annotation function, and generating an identification auxiliary file meeting the archiving requirements.

[0011] The voiceprint regional tracing device comprises: An audio processing module configured to perform audio preprocessing on the speech to be analyzed, extract acoustic features, and generate an international phonetic alphabet phoneme sequence; The classification module is configured to classify the international phonetic alphabet phoneme sequence based on a pre-constructed multi-level linguistic feature knowledge base and a statistical machine learning model, and determine a plurality of sound class feature segments; each of the sound class feature segments includes a sound class type, a time interval, and a confidence level. The determination module is configured to input the plurality of sound class feature segments into a pre-trained regional attribute determination model, output a regional attribute tracing result of the speech to be analyzed by modeling a conditional probability relationship between the sound class feature and the dialect area.

[0012] The application also provides an electronic device, which includes a processor, a memory, and a bus. The memory stores machine readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus. The machine readable instructions are executed by the processor to perform the steps of the voice acoustic fingerprint regional tracing method.

[0013] The application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the voice acoustic fingerprint regional tracing method are performed.

[0014] The voice acoustic fingerprint regional tracing method, device, equipment, and storage medium provided by the application include the following steps: performing audio preprocessing on speech to be analyzed, extracting acoustic features, and generating an international phonetic alphabet phoneme sequence; classifying the international phonetic alphabet phoneme sequence based on a pre-constructed multi-level linguistic feature knowledge base and a statistical machine learning model, and determining a plurality of sound class feature segments; each of the sound class feature segments includes a sound class type, a time interval, and a confidence level; inputting the plurality of sound class feature segments into a pre-trained regional attribute determination model, and outputting a regional attribute tracing result of the speech to be analyzed by modeling a conditional probability relationship between the sound class feature and the dialect area. The regional tracing is realized by multi-level linguistic knowledge modeling, statistical machine learning driven feature extraction, and regional attribute determination model, which fundamentally overcomes the problems of inaccuracy and low efficiency in existing acoustic fingerprint identification technology.

[0015] To make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0017] Figure 1 A flow chart of a voice voiceprint regional tracing method provided by an embodiment of the present application; Figure 2 A structure schematic diagram of a voice voiceprint regional tracing device provided by an embodiment of the present application; Figure 3 A structure schematic diagram of a voice voiceprint regional tracing device provided by an embodiment of the present application; Figure 4 A structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, every other embodiment obtained by those skilled in the art without creative labor belongs to the scope of the present application.

[0019] Firstly, the application scenarios applicable to the present application are introduced. The present application can be applied to the field of voice recognition technology.

[0020] It is found through research that voiceprint identification is an important means of determining the identity of a speaker by analyzing individual sound characteristics in a voice signal. With the widespread application of voice data in criminal cases, investigations, electronic evidence, etc., voiceprint identification has become one of the indispensable technologies of forensic science. Among them, the listening recognition analysis as the core link of voiceprint identification directly affects the scientificity and accuracy of the identification conclusion. The current mainstream voiceprint identification workstation mostly uses the "same word search" listening recognition method, that is, based on the voice transcription result, the same words or phrases in the sample and the sample are searched to form a candidate segment list for manual comparison and playback by the identification personnel. Although this method improves the keyword search efficiency, it still has significant defects: first, the matching granularity is limited to the word level, and it is difficult to compare the pronunciation characteristics at the phoneme level; second, it relies on manual input of keywords for passive search, and it is difficult to actively find potential comparable phoneme segments; third, it lacks support for phonetic rules, and cannot distinguish between "comparable" and "incomparable" phonemes (such as / a / in "big" and / a / in "flower" which are not comparable due to different contexts), increasing the risk of misjudgment. More importantly, existing technologies generally ignore the guiding role of linguistic knowledge in voiceprint analysis. In summary, the existing voiceprint identification technology has obvious shortcomings in single feature extraction dimension, lack of linguistic support in analysis logic, lack of regional attribute reasoning ability, and insufficient human-computer interaction adaptation, and an urgent need for a voiceprint regional tracing method to improve the accuracy of voiceprint identification.

[0021] Based on this, the voiceprint regional tracing method provided by the embodiment of the present application is implemented by multi-level linguistic knowledge modeling, statistical machine learning driven feature extraction and regional attribute determination model to fundamentally overcome the problems of inaccuracy and low efficiency in existing voiceprint identification technology.

[0022] Please refer to Figure 1 , Figure 1 The flowchart of the voiceprint regional tracing method provided by the embodiment of the present application is shown in Figure 1 The voiceprint regional tracing method provided by the embodiment of the present application includes the following steps: S101: performing audio preprocessing on the voice to be analyzed, extracting acoustic features and generating an international phonetic alphabet phoneme sequence.

[0023] In this step, the audio preprocessing is performed on the voice to be analyzed, and the acoustic features of the audio preprocessed voice to be analyzed are extracted and the international phonetic alphabet phoneme sequence is generated.

[0024] In one possible implementation, the audio preprocessing of the voice to be analyzed, the extraction of acoustic features and the generation of the international phonetic alphabet phoneme sequence include: A: denoising the speech to be analyzed, and dividing the denoised speech into frames according to a fixed time window to obtain a sequence of speech frames.

[0025] It should be noted that spectral subtraction is applied to the effective speech frame for noise reduction. Specifically, the background noise power spectrum is estimated in the silent section, the noise estimate is subtracted from the short-time Fourier transform (STFT) spectrum of the noisy speech, and the time-domain signal is recovered by inverse transform. Alternatively, Wiener filtering or a deep neural network (DNN) based speech enhancement module can also be used to further improve the signal-to-noise ratio.

[0026] Here, the denoised speech signal is processed by frame segmentation, with a frame length of 20 ms and a frame shift of 10 ms to ensure sufficient overlap between adjacent frames and avoid information loss. Each frame of data is multiplied by a Hamming window to reduce spectral leakage.

[0027] B: extracting at least one acoustic feature from each speech frame; wherein the acoustic features include at least one of mel-frequency cepstral coefficients, filter bank energies and prosodic features.

[0028] Here, after completing the frame segmentation, the system extracts multi-dimensional acoustic features from each frame of speech to form a feature vector set for subsequent modeling. The main features extracted include mel-frequency cepstral coefficients (MFCC), filter bank energies (FBank), and fundamental frequency (F0) and prosodic features. All the above features are arranged in time sequence to form a two-dimensional feature matrix, which is used as input representation for subsequent speech transcription and phoneme classification.

[0029] C: based on a dialect dictionary fusion speech transcription model, classifying the speech frame sequence and its corresponding acoustic features frame by frame, and outputting a timestamped text transcription result and corresponding international phonetic alphabet phoneme sequence; wherein the speech transcription model is based on the Wav2Vec2.0 architecture and is obtained by fine-tuning training on a large-scale speech corpus containing multi-dialect pronunciation samples.

[0030] Here, on the basis of obtaining high-quality acoustic features, the system calls a pre-trained multi-dialect fusion speech transcription model to convert the speech content into text and its corresponding International Phonetic Alphabet (IPA) phoneme sequence, with the time boundary information (start / end timestamp) of each phoneme. The speech transcription model is fine-tuned based on the Wav2Vec2.0 architecture, which includes a convolutional feature encoder that maps the original waveform to context-dependent hidden states, a Transformer context network that captures long-range dependencies, and a connection time classification (CTC) output layer that directly outputs phoneme-level predictions. The model introduces a large-scale annotated dataset covering major dialect areas in China (such as Southwest Mandarin, Cantonese, Minnan, Wu, etc.) during the training phase, and integrates dialect pronunciation dictionary constraints to ensure good recognition of dialect variants (such as [n] / [l] misreading, tone retention, etc.).

[0031] It should be noted that the output results of the speech transcription model include: text transcription results: standard Chinese characters or pinyin text, IPA phoneme sequence, and time alignment information corresponding to the speech.

[0032] S102: Based on the pre-constructed multi-level linguistic feature knowledge base and statistical machine learning model, the international phonetic alphabet phoneme sequence is classified to determine a plurality of phonetic feature segments; wherein each phonetic feature segment includes phonetic type, time interval and confidence.

[0033] In this step, the international phonetic alphabet phoneme sequence is classified based on the pre-constructed multi-level linguistic feature knowledge base and statistical machine learning model to determine a plurality of phonetic feature segments.

[0034] In one possible implementation, the multi-level linguistic feature knowledge base is constructed by the following steps: Collect dialect speech samples and corresponding text transcription data covering multiple geographic regions; extract basic phonetic features, region-specific features, and context-related features from the dialect speech samples to correlate with each other, and construct a multi-level linguistic feature knowledge base; wherein the basic phonetic features are used to describe common pronunciation variation types in speech and define the comparability rules between phonemes, the region-specific features are used to establish the mapping relationship between pronunciation variation patterns and dialect regions, and the context-related features are used to represent the variation rules of the same phonetic class in different language use environments.

[0035] Here, the multi-level linguistic feature knowledge base specifically includes: a basic phonetic feature layer: according to the standard of Chinese phonetics, 12 types of key pronunciation variation categories are defined, such as: initial dissimilation category: [n] / [l] not divided, [s] / [ ]mixing (flat and flat tongue not divided); vowel mutual conversion class: [e]→[a], [y]→[i]; tone deviation class: entering tone retention, tone value deviation, etc. Each class is annotated with its phonetic form, typical example and comparability rule (such as the open call in “big” / a / , only with “he” / a / has comparability, not with the close call in “flower” / a / ), to avoid misjudgment across contexts. Regional specificity feature layer: integrate multiple dialect area language records and measured phonetic database, and establish “sound class-dialect area” mapping relationship table. For example: Southwest Mandarin dialect area: high probability of [n] / [l] mixed reading; Wu dialect area: common voice stop retention, short and sharp entering tone; Min dialect area: [tʃ] / [t] substitution phenomenon exists; Cantonese area: nine-tone six-tone system, complete retention of nasal final [-m][-n][-ŋ]. Context-related feature layer: build “sound variation occurrence condition” knowledge items to describe the variation rules of specific sound classes in different parts of speech, sentence patterns, speech speed or adjacent phoneme environments. For example: “[n] / [l] mixed reading” is more common in fast spoken language.

[0036] The present application builds a multi-level linguistic feature knowledge base, introduces a three-dimensional feature system of “basic sound class-regional specificity-context association”, and clearly defines the comparability rules between various phonemes, so that the system follows linguistic principles when automatically extracting sound classes. Combined with the two-stage model of LSTM+XGBoost, not only the accurate classification of phoneme sequence is realized, but also the output result is dynamically optimized according to the context and confidence, effectively avoiding the wrong association between non-comparable phonemes.

[0037] In one possible implementation, the international phonetic alphabet phoneme sequence is classified based on a pre-constructed multi-level linguistic feature knowledge base and a statistical machine learning model, and a plurality of sound class feature segments are determined, including: a: using a bidirectional long short-term memory network to classify the international phonetic alphabet phoneme sequence, matching the sound class features in the multi-level linguistic feature knowledge base, and generating candidate sound class segments.

[0038] Here, the previously generated timestamped IPA phoneme sequence is input into a bidirectional long short-term memory network model, which uses the sound class features in the hierarchical linguistic feature knowledge base to capture the context-dependent relationships between phonemes and preliminarily identify potential candidate sound class feature segments.

[0039] b: using an XGBoost model to combine context-related features and regional distribution prior probabilities to reorder the confidence of candidate sound class segments, retain high-confidence effective sound class feature segments, and determine a plurality of sound class feature segments.

[0040] Here, the context-related feature layer is queried to obtain the probability preference of the current sound class in different language environments, and is encoded as a numerical feature.

[0041] The "phonetic class and dialect area mapping table" in the system call knowledge base is used to obtain the statistical distribution of the current phonetic class in each major dialect area in the country and convert it into a quantitative priori.

[0042] It should be noted that the phonetic class and dialect area mapping table is constructed by linguistics experts using CRF (Conditional Random Field) and other models to analyze the relationship between phoneme sequences and regional labels, automatically correct the deviations in the initial rules, dynamically adjust the mapping weights, and retrain new samples to build.

[0043] Here, the XGBoost model is introduced to combine contextual correlation features and regional distribution probabilities to sort the confidence of candidate phonetic class segments, filter out invalid segments with a confidence lower than 0.7, and determine multiple phonetic class feature segments.

[0044] Among them, the existing technology relies on manual playback of audio segments or passive search based on keywords, which is cumbersome and easy to miss key pronunciation segments. The present application introduces a bidirectional LSTM and XGBoost fusion model combined with a multi-level language knowledge base to actively and batch extract phonetic class feature segments (such as [n] / [l] misreading, tongue tip back sound to tongue surface sound, etc.) with regional discrimination significance from the input audio, without the need for the user to pre-input search terms, significantly reducing the time for invalid manual screening, and significantly improving the work efficiency of the listening and discrimination preparation stage.

[0045] S103: inputting the multiple phonetic class feature segments into a pre-trained regional attribute determination model, establishing a conditional probability relationship between the phonetic class features and the dialect area, and outputting the regional attribute tracing result of the speech to be analyzed.

[0046] In this step, after identifying multiple high-confidence phonetic class feature segments in the audio, a Bayesian network-based regional attribute determination model is further constructed and applied to realize scientific inference from "pronunciation features" to "speaker regional attribution". This model learns the statistical distribution of typical phonetic class features in each local dialect area, establishes a conditional probability relationship between phonetic class features and dialect areas, and finally outputs the most likely regional attribute and its probability distribution of the speech to be analyzed, providing interpretable auxiliary evidence support for voiceprint identification.

[0047] Here, the regional attribute tracing result includes the probability distribution of the speaker's regional attribution and the key evidence chain explanation, where the key evidence chain explanation is the subset of phonetic class features that most affect the probability distribution of the speaker's regional attribution.

[0048] It should be noted that the regional attribute determination model adopts a Bayesian network as the core inference engine, and the structure is a directed acyclic graph (DAG), wherein the nodes represent random variables, and the edges represent the dependency relationship between variables. The entire network contains two types of nodes: evidence nodes corresponding to the input phonetic feature segments, such as “[n] / [l] mixed reading”, “flat and tilted tongue not distinguished”, “retained entering tone”, etc., and each node takes the frequency and comprehensive confidence of its occurrence in the current audio; the target node represents the dialect area category to which the speaker belongs, covering multiple dialect subareas, including Mandarin (subdivided into Southwest, Northwest, North China, Jianghuai, etc.), Wu dialect, Cantonese, Min dialect, Xiang dialect, Gan dialect, Hakka dialect, etc.

[0049] In one possible implementation, the plurality of phonetic feature segments are input into a pre-trained regional attribute determination model, the conditional probability relationship between the phonetic features and the dialect area is modeled, and the regional attribute tracing result of the voice to be analyzed is output, including: (1) Calculate the response probability of each target dialect area to the combination of phonetic features for the phonetic feature segments, and generate the probability distribution of the regional attribution of the speaker.

[0050] Here, the phonetic features of the phonetic feature segments are mapped to the corresponding observation nodes in the Bayesian network, and the weighted evidence strength is calculated according to the occurrence frequency and comprehensive confidence. The joint tree algorithm (Joint Tree Algorithm) or belief propagation (Belief Propagation) method is used to perform probability inference in the Bayesian network, update the posterior probability distribution of the target node “dialect area”, and obtain the probability distribution of the regional attribution of the speaker.

[0051] (2) According to the contribution degree of each phonetic feature to the dialect area, the key evidence path supporting the probability distribution of the regional attribution of the speaker is reversely analyzed to form an interpretable regional attribute inference conclusion, and the key evidence chain explanation is output. Based on the probability distribution of the regional attribution of the speaker and the key evidence chain explanation supporting the conclusion, the regional attribute tracing result is constructed.

[0052] Here, the feature contribution quantification mechanism is introduced, and the sensitivity analysis and gradient backtracking method are used to identify the phonetic feature subset that has the most influence on the final judgment, i.e. the “key evidence chain explanation”.

[0053] In one possible implementation, the regional attribute determination model is determined by the following steps: A voice sample set of multiple dialect areas is collected. A Bayesian network model is constructed, wherein an input node is a phonetic feature in each voice sample and its occurrence frequency and confidence, a hidden layer node represents a potential dialect language habit pattern, and an output node is a belonging category of each dialect area. Based on the voice sample set, a co-occurrence frequency of each phonetic feature in different dialect areas is counted, the Bayesian network model is trained by using maximum likelihood estimation, an accuracy rate of regional attribute prediction of the Bayesian network model on unknown samples is determined by using a cross-validation method, and a network structure or a prior parameter is adjusted according to feedback until a preset performance threshold is met, so that the regional attribute determination model is determined.

[0054] In a possible implementation, after the regional attribute tracing result of the voice to be analyzed is output, the voiceprint regional tracing method further includes: The regional attribute tracing result is provided to an interactive interface, multi-dimensional display of phonetic fragments, geographic information system visualization, synchronous playback comparison of evidence and samples, spectrum analysis, and intelligent labeling functions are supported, and an identification auxiliary file meeting archiving requirements is generated.

[0055] Here, the interactive interface is deployed on a voiceprint identification workstation or a special voice analysis platform, runs on a Windows / Linux operating system, supports high-resolution display and professional audio equipment access. The front end is developed by using an Electron+React framework, the back end communicates with a core algorithm engine (such as phonetic identification and Bayesian inference) through a RESTful API, and a local encrypted database (SQLite) and a cloud synchronization dual-mode mechanism are used for data storage, so that information security and compliance audit are ensured.

[0056] Among them, the left side area of the system main interface on the interactive interface is an "audio class feature panel", which lists all detected valid audio class feature segments in table form and supports multiple sorting and filtering methods: multi-dimensional column item display: audio class type, occurrence time, context text, IPA phoneme sequence, and comprehensive confidence. The right side of the interface is embedded with a dynamic GIS map module, which is realized based on Leaflet or ArcGIS Engine, and is used to visually display the high-probability dialect distribution area pointed to by the current voice feature. Heat map rendering: according to the probability distribution output by Bayesian inference, a dialect attribution heat map is drawn on the China map; the color depth represents the possibility of a certain area as the source of the speaker (such as the Sichuan Basin, which is represented by deep red, representing 85% high probability); when hovering over a province, a pop-up box will appear, displaying the main audio class features and support in that area. The "listening and identifying work area" is set in the middle of the system, which supports fine voice comparison between the case sample and the reference sample. The system has a built-in "structured annotation toolbar" that provides preset label templates to reduce subjective description bias. After completing all the analysis, the system automatically generates a standardized voiceprint regional attribute identification auxiliary report, which supports one-key export of multiple formats to generate an identification auxiliary file that meets the archiving specifications.

[0057] The present application integrates GIS map visualization, intelligent annotation templates, context display, and spectrum comparison functions to build a special interactive interface, so that even a junior technician can complete the standardized audio class recognition and annotation process under the guidance of the system. The intelligent annotation log is automatically generated and supports export in PDF / XML format, meeting the archiving requirements and helping to promote the standardization of voiceprint identification work.

[0058] The voiceprint regional tracing method provided by the embodiment of the present application comprises: performing audio preprocessing on the voice to be analyzed, extracting acoustic features, and generating an international phonetic alphabet phoneme sequence; classifying the international phonetic alphabet phoneme sequence based on a pre-constructed multi-level linguistic feature knowledge base and a statistical machine learning model, and determining a plurality of audio class feature segments; wherein each audio class feature segment comprises an audio class type, a time interval, and a confidence; inputting the plurality of audio class feature segments into a pre-trained regional attribute determination model, and outputting a regional attribute tracing result of the voice to be analyzed by modeling the conditional probability relationship between the audio class feature and the dialect area. Through multi-level linguistic knowledge modeling, statistical machine learning driven feature extraction, and regional attribute determination model, regional tracing is realized, which fundamentally overcomes the problems of inaccuracy and low efficiency in existing voiceprint identification technology.

[0059] Please refer to Figure 2 , Figure 3 , Figure 2One of the structural schematic diagrams of a voiceprint-based geographic tracing device provided in this application embodiment; Figure 3 This is a second schematic diagram of a voiceprint-based geographic tracing device provided in an embodiment of this application. Figure 2 As shown, the voiceprint geographic tracing device 200 includes: The audio processing module 210 is used to perform audio preprocessing on the speech to be analyzed, extract acoustic features, and generate an IPA phoneme sequence. The classification module 220 is used to classify the International Phonetic Alphabet phoneme sequence based on a pre-built multi-level linguistic feature knowledge base and a statistical machine learning model, and to determine multiple sound class feature segments; wherein, each sound class feature segment includes sound class type, time interval and confidence level; The determination module 230 is used to input multiple sound category feature segments into a pre-trained regional attribute determination model, and output the regional attribute tracing result of the speech to be analyzed by modeling the conditional probability relationship between sound category features and dialect area.

[0060] Furthermore, the audio processing module 210 is used to perform audio preprocessing on the speech to be analyzed, extract acoustic features, and generate an IPA phoneme sequence: The speech to be analyzed is denoised, and the denoised speech to be analyzed is divided into frames according to a fixed time window to obtain a speech frame sequence. For each speech frame, at least one acoustic feature is extracted; wherein the acoustic feature includes at least one of Mel frequency cepstral coefficients, filter bank energy, and prosodic features; The speech transcription model based on the integrated dialect dictionary classifies the speech frame sequence and its corresponding acoustic features frame by frame, and outputs the text transcription result with timestamp and the corresponding International Phonetic Alphabet phoneme sequence; wherein, the speech transcription model is built on the Wav2Vec2.0 architecture and fine-tuned and trained on a large-scale speech corpus containing pronunciation samples of multiple dialects.

[0061] Furthermore, such as Figure 3 As shown, the voiceprint regional tracing device 200 also includes a knowledge base construction module 240, which constructs the multi-level linguistic feature knowledge base through the following steps: Collect dialect speech samples and corresponding text transcription data covering multiple geographical regions; Basic sound category features, regional specific features, and contextual association features are extracted from the dialect speech samples and correlated with each other to construct a multi-level linguistic feature knowledge base; The basic phonetic class feature is used to describe common pronunciation variation types in speech and define comparability rules between phonemes of different classes, the region-specific feature is used to establish a mapping relationship between pronunciation variation patterns and dialect regions, and the context-related feature is used to represent variation rules of the same phonetic class in different language use environments.

[0062] Further, the classification module 220 is configured to classify the international phonetic alphabet phoneme sequence based on the pre-constructed multi-level linguistic feature knowledge base and the statistical machine learning model, and determine the multiple phonetic class feature segments, including: using a bidirectional long short-term memory network to classify the international phonetic alphabet phoneme sequence, matching the phonetic class features in the multi-level linguistic feature knowledge base, and generating candidate phonetic class segments; using an XGBoost model to combine context-related features and regional distribution prior probabilities to reorder the confidence of the candidate phonetic class segments, retain effective phonetic class feature segments with high confidence, and determine the multiple phonetic class feature segments.

[0063] Further, the determination module 230 is configured to input the multiple phonetic class feature segments into a pre-trained regional attribute determination model, output the regional attribute tracing result of the speech to be analyzed by modeling the conditional probability relationship between the phonetic class features and the dialect regions, and output the regional attribute tracing result of the speech to be analyzed by modeling the conditional probability relationship between the phonetic class features and the dialect regions. calculating the response probability of each target dialect region to the phonetic class feature combination for the phonetic class feature segments, and generating a probability distribution of the speaker's regional attribution; and according to the contribution of each phonetic class feature to the dialect region, the key evidence path supporting the probability distribution of the speaker's regional attribution is reversely analyzed to form an interpretable regional attribute inference conclusion, and a key evidence chain explanation is output; Based on the probability distribution of the speaker's regional attribution and the key evidence chain explanation supporting the conclusion, the regional attribute tracing result is constructed.

[0064] Further, as shown in Figure 3 The voiceprint regional tracing device 200 further includes a model training module 250, which determines the regional attribute determination model by the following steps: collecting a plurality of dialect region speech sample sets; constructing a Bayesian network model, wherein the input nodes are the phonetic class features in each speech sample and their occurrence frequency and confidence, the hidden layer nodes represent the potential dialect language habit patterns, and the output nodes are the attribution categories of each dialect region; Based on the statistics of the co-occurrence frequency of each phonetic class feature in different dialect regions in the speech sample set, the Bayesian network model is trained using maximum likelihood estimation; The cross-validation method is used to determine the accuracy of the regional attribute prediction of the Bayesian network model on unknown samples, and the network structure or the smoothing prior parameter is adjusted according to the feedback until the preset performance threshold is met, and the regional attribute determination model is determined.

[0065] Further, as shown in Figure 3 The voiceprint regional tracing device 200 further includes an interaction module 260, which is configured to: The regional attribute tracing result is provided to an interface, and functions such as multi-dimensional display of sound class fragments, geographic information system visualization, synchronous playback comparison of samples and samples, spectrum analysis, and intelligent labeling are supported, and an identification auxiliary file meeting the archiving requirements is generated.

[0066] The voiceprint regional tracing device provided by the embodiment of the present application comprises: an audio processing module configured to perform audio preprocessing on a voice to be analyzed, extract acoustic features, and generate an international phonetic alphabet phoneme sequence; a classification module configured to classify the international phonetic alphabet phoneme sequence based on a pre-constructed multi-level linguistic feature knowledge base and a statistical machine learning model, and determine a plurality of sound class feature fragments; wherein each sound class feature fragment comprises a sound class type, a time interval, and a confidence level; and a determination module configured to input the plurality of sound class feature fragments into a pre-trained regional attribute determination model, output a regional attribute tracing result of the voice to be analyzed by modeling a conditional probability relationship between the sound class features and the dialect area. The regional tracing is realized by multi-level linguistic knowledge modeling, statistical machine learning driven feature extraction, and a regional attribute determination model, which fundamentally overcomes the problems of inaccuracy and low efficiency in existing voiceprint identification technology.

[0067] Please refer to Figure 4 , Figure 4 A structural schematic diagram of an electronic device provided by the embodiment of the present application is shown in Figure 4 The electronic device 400 comprises a processor 410, a memory 420, and a bus 430.

[0068] The memory 420 stores machine readable instructions executable by the processor 410, and when the electronic device 400 is running, the processor 410 and the memory 420 communicate through the bus 430. The machine readable instructions executed by the processor 410 can perform the steps of the voiceprint regional tracing method of the method embodiment shown in the above Figure 1 The specific implementation can be referred to the method embodiment, which will not be described here.

[0069] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the voiceprint regional tracing method of the method embodiment shown in the aboveFigure 1 The steps of the voiceprint regional tracing method of the voice in the method embodiment are specifically implemented as described in the method embodiment, which will not be repeated here.

[0070] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the system, device and unit described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0071] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0072] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0073] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0074] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various storage medium that can store program codes.

[0075] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any skilled person in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features, within the technical scope disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for tracing the geographical origin of speech, characterized in that, The voiceprint geographic tracing method includes: The speech to be analyzed is preprocessed to extract acoustic features and generate a sequence of International Phonetic Alphabet phonemes. The IPA phoneme sequence is classified based on a pre-built multi-level linguistic feature knowledge base and a statistical machine learning model to determine multiple sound class feature segments; wherein each sound class feature segment includes sound class type, time interval and confidence level; Multiple sound feature segments are input into a pre-trained regional attribute determination model. By modeling the conditional probability relationship between sound features and dialect areas, the regional attribute tracing results of the speech to be analyzed are output.

2. The voiceprint geographic tracing method according to claim 1, characterized in that, The audio preprocessing of the speech to be analyzed, extracting acoustic features and generating an IPA phoneme sequence includes: The speech to be analyzed is denoised, and the denoised speech to be analyzed is divided into frames according to a fixed time window to obtain a speech frame sequence. For each speech frame, at least one acoustic feature is extracted; wherein the acoustic feature includes at least one of Mel frequency cepstral coefficients, filter bank energy, and prosodic features; The speech transcription model based on the integrated dialect dictionary classifies the speech frame sequence and its corresponding acoustic features frame by frame, and outputs the text transcription result with timestamp and the corresponding International Phonetic Alphabet phoneme sequence; wherein, the speech transcription model is built on the Wav2Vec2.0 architecture and fine-tuned on a large-scale speech corpus containing pronunciation samples of multiple dialects.

3. The voiceprint geographic tracing method according to claim 1, characterized in that, The multi-level linguistic feature knowledge base is constructed through the following steps: Collect dialect speech samples and corresponding text transcription data covering multiple geographical regions; Basic sound category features, regional specific features, and contextual association features are extracted from the dialect speech samples and correlated with each other to construct a multi-level linguistic feature knowledge base; Among them, the basic phoneme features are used to describe common pronunciation variation types in speech and define the comparability rules between various phonemes; the regional specific features are used to establish the mapping relationship between pronunciation variation patterns and dialect regions; and the context-related features are used to characterize the variation patterns of the same phoneme in different language use environments.

4. The voiceprint geographic tracing method according to claim 1, characterized in that, The International Phonetic Alphabet (IPA) phoneme sequence is classified based on a pre-built multi-level linguistic feature knowledge base and a statistical machine learning model, identifying multiple sound category feature segments, including: The International Phonetic Alphabet (IPA) phoneme sequence is classified using a bidirectional long short-term memory network, and matched with the sound class features in the multi-level linguistic feature knowledge base to generate candidate sound class segments. By using the XGBoost model to combine contextual features and prior probabilities of geographical distribution, candidate sound class segments are reordered by confidence, retaining effective sound class feature segments with high confidence, and identifying multiple sound class feature segments.

5. The voiceprint geographic tracing method according to claim 1, characterized in that, The process involves inputting multiple sound feature segments into a pre-trained regional attribute determination model. By modeling the conditional probability relationship between sound features and dialect regions, the model outputs the regional attribute tracing results of the speech to be analyzed, including: Calculate the response probability of each target dialect region to the combination of sound features for the sound feature segments, and generate the probability distribution of the speaker's regional affiliation; Based on the contribution of each sound category feature to the dialect area, the key evidence path supporting the probability distribution of the speaker's regional affiliation is reverse-analyzed, forming an interpretable conclusion on regional attributes and outputting a key evidence chain explanation. Based on the probability distribution of the speaker's geographical affiliation and the key evidence chain supporting this conclusion, the geographical attribute tracing result is constructed.

6. The voiceprint geographic tracing method according to claim 1, characterized in that, The regional attribute determination model is determined through the following steps: Collect speech sample sets from multiple dialect regions; Construct a Bayesian network model where the input nodes are the sound class features, their frequency and confidence in each speech sample, the hidden layer nodes represent the potential dialect language habit patterns, and the output nodes are the categories to which each dialect region belongs. Based on the statistical analysis of the co-occurrence frequency of each sound category feature in different dialect areas in the speech sample set, the Bayesian network model is trained using maximum likelihood estimation. The accuracy of the Bayesian network model in predicting regional attributes on unknown samples is determined by cross-validation. The network structure is adjusted or the prior parameters are smoothed based on the feedback until a preset performance threshold is met, thus determining the regional attribute determination model.

7. The voiceprint geographic tracing method according to claim 1, characterized in that, After outputting the regional attribute source tracing results of the speech to be analyzed, the voiceprint regional source tracing method further includes: The results of the regional attribute tracing are provided to the interface, supporting multi-dimensional display of audio segments, visualization of geographic information systems, synchronous playback and comparison of evidence and samples, spectrum analysis, and intelligent annotation functions, and generating identification auxiliary documents that meet the archiving requirements.

8. A voiceprint-based regional tracing device, characterized in that, The voiceprint geolocation tracing device includes: The audio processing module is used to preprocess the speech to be analyzed, extract acoustic features, and generate a sequence of International Phonetic Alphabet phonemes. The classification module is used to classify the International Phonetic Alphabet phoneme sequence based on a pre-built multi-level linguistic feature knowledge base and a statistical machine learning model, and to determine multiple sound class feature segments; wherein, each sound class feature segment includes sound class type, time interval and confidence level; The determination module is used to input multiple sound category feature segments into a pre-trained regional attribute determination model, and output the regional attribute tracing result of the speech to be analyzed by modeling the conditional probability relationship between sound category features and dialect area.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the voiceprint localization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the voiceprint-based regional tracing method for speech as described in any one of claims 1 to 7.

Images